Integration Tests
Integration tests verify the system’s behavior as a whole — they run the actual binaries (xretractor, xqry, xtrdb) and compare their output against patterns, or check specific properties of the output files. This differs from unit tests, which use the GTest framework to test isolated classes and functions of the rdb and retractor libraries (e.g. payload, descriptor, crsMath, compiler), don’t require a running server, and don’t produce artifacts on disk. Integration tests are run with the command ninja test (or ctest) in the build/Debug/ directory; a single test can be run with ctest -R <name> -V.
Integration tests are split into two directories based on their concurrency requirements. Tests in the IntegrationTest_serial directory run the xretractor server in IPC mode — they use the shared lock file /tmp/xretractor_service.lock and Boost shared-memory segments. To avoid conflicts between concurrent instances, CMake forces RUN_SERIAL TRUE for them (one after another). Tests in the IntegrationTest_parallel directory do not start the IPC server — they compile queries (xretractor -c) or perform file operations via xtrdb — and can safely run in parallel.
Serial tests — IntegrationTest_serial
| Test name | Description |
|---|---|
agse1 | The @(start, length) time-window operator — forward variants @(1,4), backward @(1,-4), various lengths. See: AGSE Sliding Data Window. |
agse2 | Window @(n,m) combinations on a 3-field stream, alignment and rate conversion at ratios 1:1, 1:2, 2:3, 2:4. See: AGSE Sliding Data Window. |
agse3 | The @(n,m) operator when the output rate is lower than the input rate (source rate 0.1) — windows @(3,2), @(3,3), @(3,-3). See: AGSE Sliding Data Window. |
consistency | Read consistency: two streams read the same source; their difference must always equal 100. See: Data and Control Flow. |
issue113_meta_internal | Structure of the .meta sidecar file: header size (8 B), entry size (18 B), sampling interval, null bitsets for records with and without nulls. See: Data Storage Format — Files. |
issue113_meta_xtrdb | Verification via xtrdb that after running xretractor+xqry, the .meta file is created and reported correctly (meta: temp/str_null.meta). See: Data Storage Format — Artifact Analysis. |
issue113_null_skip | The -n flag in xqry — rows that are entirely null are skipped; without the flag, all rows (including all-null ones) must be present. See: Command-Line Options — xqry. |
issue113_null_xqry | Nulls transmitted over IPC: null values from the source file are shown as null in xqry output. See: Command-Line Options — xqry. |
issue121_isnull | The isnull(field) function — returns 1 when the field is null, 0 otherwise. See: Aggregate Operators and Expression Functions. |
issue121_null_propagation | Propagation of null values through SELECT to the output stream. See: SELECT Command. |
issue128_numeric_to_string | Conversion of INTEGER/FLOAT to STRING via the to_string() function with a declared field width; verification of the output descriptor. See: Aggregate Operators and Expression Functions. |
issue128_string_to_numeric | Conversion of STRING to numeric types: to_integer(), to_float(), to_double(); null propagation through the conversion. See: Aggregate Operators and Expression Functions. |
issue167_dedup_cascaded | Cascaded absorption of substrates via deduplicateSubstrats() — multi-step rewriting of PUSH_ID tokens. See: Substrates. |
issue167_dedup_field_names | Substrate deduplication without comparing schema field names — merging when field types are equivalent, regardless of names. See: Substrates. |
issue167_dedup_nonzero_offset | Updating PUSH_ID in the consumer’s lSchema for a non-zero offset of the absorbed substrate; coverage of the code path in compiler.cpp. See: Substrates. |
issue167_dedup_positive | The basic deduplication case: a substrate merged with a named stream with an equivalent program and field types. See: Substrates. |
issue167_triarg | Multi-argument stream expressions: s1+s2+s3, (s1#s2)#s3, s1+(s2#s3), s1+s2+s3+s4; in-memory and on-disk substrates. See: Substrates, Operation Sequencing. |
issue42_rule | The RULE command — conditional DUMP and SYSTEM actions triggered on stream values; runs xretractor and reads the result via xtrdb. See: RULE Command. |
issue56_timeshift | The > filter operator on joined streams — only records satisfying the condition end up in the output stream. See: SELECT Command — Sequencing. |
issue61_tmpmem | The in-memory substrate SUBSTRAT 'memory' — intermediate data kept in RAM instead of on disk. See: STORAGE Types. |
issue6_adhoc | Ad-hoc query mode: xqry -a 'SELECT ...' — defining and executing a query on the fly, without an .rql file. See: Ad Hoc Queries. |
operations | The # (HASH merge) operator on two streams with different rates — verifying the ratio of record counts in the output. See: Interleaving Operation Sequencing. |
rotation_test | The stream binary-file rotation mechanism (ROTATION) — file count after two xretractor -m 2 cycles. See: File Rotation Mechanism. |
simple | A smoke test for arithmetic on joined streams (core0 rate 0.1 + core1 rate 0.2), read via xtrdb. See: SELECT Command. |
simple_max | The .max operator on a stream — the maximum value, joined with the original stream. See: Aggregate Operators and Expression Functions. |
xqry_elem_limit | The -m N parameter in xqry — limits the number of received records to exactly N, regardless of the source’s length. See: Command-Line Options — xqry. |
Parallel tests — IntegrationTest_parallel
| Test name | Description |
|---|---|
dsp | Regression for the FIR filter pipeline: sliding window @(1,25), array multiplication with the _ index, .sumc reduction, joining the signal with the output. See: Signal Filter Implementation. |
issue113_meta | xtrdb operations after two appends — record list and hexdump of the binary file compared against a pattern. See: Data Storage Format — Artifact Analysis. |
issue113_meta_autocreate | Automatic creation of the .meta sidecar file after the first append; size >16 B; xtrdb reports the correct path. See: Data Storage Format — Files. |
issue113_null_txtsrc | The rread/getpos commands in xtrdb on a TEXTSOURCE stream containing nulls. See: Data Storage Format — Artifact Analysis. |
issue153_storagemap_meta_cases | The xtrdb -s storage map for a plain file and a retractordb-style file: slot markers, segment list, rotated files, .meta/.shadow references. See: Inspection Tool xtrdb -s. |
issue31_doc | Generating DOT/SVG graphs via xretractor -c -d ... for documentation examples at three levels of detail. See: Compilation Debugging. |
issue42_rule | Compilation of RULE syntax — the -c stage only, without starting the server. See: RULE Command. |
issue56_timeshift | Compilation of the > filter operator — the -c stage only. See: SELECT Command — Sequencing. |
issue61_tmpmem | Compilation of a query with SUBSTRAT 'memory' — the -c stage only. See: STORAGE Types. |
issue95_loopInCompile | Cycle detection in the query graph by the compiler — expected “Circular dependency” error and a non-zero exit code. See: Loop Detection in Compilation. |
issue96_no_substrat_reduction | User-defined streams with an identical structure are NOT merged; only automatic substrates are subject to merging. See: Substrates. |
issue96_substrat_reference | A generated substrate shared by two user streams — correct references in the dependency tree. See: Substrates. |
Pattern1 | Compilation of the # (HASH-merge) operator, field selection with an offset, and stream joining with +. See: Interleaving and Summation Operation Sequencing. |
Pattern2 | Compilation of queries on BYTE streams from /dev/urandom: SELECT, arithmetic, stream joining. See: SELECT Command. |
Pattern3 | Compilation of SELECT * (unfold) with an output-file declaration and retention. See: Asterisk Expansion. |
Pattern4 | Compilation of the Crc(bits, seed) function in 16-bit and 8-bit variants on a 2-field stream. See: Aggregate Operators and Expression Functions. |
Pattern5 | xtrdb operations on a multi-type record (STRING, INTEGER, BYTE, FLOAT): append, read, list/rlist, input, write. See: Artifact Analysis. |
Pattern6 | Compilation of the window operator @(n,m) forward @(1,10) and backward @(1,-10), plus a leak-free valgrind run. See: AGSE Sliding Data Window. |
Pattern7 | Compilation with identical field names across multiple streams (issue #17) — correct field identification via the stream index. See: DECLARE Command, Aliasing. |
retention | xtrdb operations on a file with a RETENTION parameter: open, purge, append, list, write for a specific record. See: Data Storage Format — File Rotation Mechanism. |
simple | Compilation of a basic arithmetic query + DOT graph + valgrind; uses data from IntegrationTest_serial/simple. See: SELECT Command. |
simple_max | Compilation of a query with .max + DOT graph + valgrind; uses data from IntegrationTest_serial/simple_max. See: Aggregate Operators and Expression Functions. |
subquery | Compilation of nested subqueries: (a#b)>1 (hash-merge inside a filter) and (a>1)#b (filter inside a hash-merge). See: Dependency Tree Construction. |
txtsrc | xtrdb operations descc/rread/printt on a TEXTSOURCE stream (a text file as a data source). See: Data Storage Format. |