Known limits
Known limits and deferred decisions
This page lists what rqm does not do today, and what has been left for later on purpose. Where a limit has a workaround, it is given.
Platforms
rqm is built and tested on Linux. It is untested on Windows and on macOS.
- The file layer and the process layer have code for Windows. The process layer was studied by hand on one Windows machine (ARM64), and the file layer was written without one. The Windows tests of both layers are written, and none of them has run in CI, because no Windows runner is used.
- A release carries a Windows binary. It is built from Linux and never run, and it is as untested as the rest of the Windows code.
- There is no macOS binary, and the macOS code is the same as the Linux code, without any run on a Mac.
Platforms says what is guaranteed on each.
Tests and reports
- A test that is not in the report is not seen. rqm does not prove that a report accounts for every test declared in the sources, because that would need the runner’s own selection. It does list the annotated tests that are absent from the report, in
rqm coverage. nextest leaves an ignored test out of the report of a normal run, so an annotated ignored test is listed there. - Generated tests. A test that a macro generates, and a test behind
include!, is not in the scan, so it cannot carry an attribute. A selector can bind it. Forrstestandtest_case, the generated names in a nextest report are matched to the declaration by the prefix rule, and by nothing looser. - A test with a name that is not ASCII has no identity. It is the error
E-ID-NONASCII, and that error stopsrqm test. Renaming the test is the remedy. - Shared report paths. Two runs that share one fixed report path, which is what nextest allows, are not covered by the baseline check. A report path with
{run_id}avoids the problem where the runner takes its path on the command line. - The other languages. Only Rust is scanned. A test of another runner is bound with a selector, and a selector that matches nothing leaves its requirement incomplete.
The reader
- No suggestion for an unknown field. The reader reports an unknown top-level field at its key, but does not suggest the nearest known name, although the acceptance criterion of the requirement says it should.
- A blank title. The reader accepts a
titlethat holds only spaces, andrqm createrefuses to write one. - Fuzzing. The fuzz targets compile, and no fuzzing session has been run.
- Limits. Requirement YAML and Rust sources have no size limit of rqm’s own. Only the limits of the parser apply, and a breach is a named diagnostic. Report files and captured output have limits that
rqm.yamlcan change. - Anchors and aliases in YAML are not specially handled. The parser accepts them within its budget, hashes are computed over the parsed model, and a document that relies on them is outside what rqm promises.
The baseline
- Promotion defaults that could be reversed. A change of policy removes the records that the run did not refresh, where the alternative is to refuse unless the run is full. A promotion that changes nothing does not rewrite the file. A change to a source file is only a warning, also when the code of an annotated test changed. The messages of relatedness name requirements by uuid.
- Invalidation is global. Any change of the policy fingerprint makes every record stale. The subset fingerprint of each requirement is recorded, so scoping the invalidation later is a comparison and not a migration.
Performance
There is no numeric budget. Every command reads and hashes every file on every run, and rqm has no cache. If measurements show that this is too slow, the fix is a cache behind the narrow parse function, keyed by content hash, or a better scan. It will never be a weaker rule.
Left for later, on purpose
- Attribution from report properties. Today a requirement is credited only through the static mapping. If a runner turns out to be unable to express its tests statically, an opt-in relaxation can be added without changing existing projects.
- The
verification_conflictlabel. If it confuses people, it can be split into a regressed and an unconfirmed variant, and no other rule changes. - Scanning other languages. A scanner for another language needs a parser, an identity grammar and a list of supported forms.
- Prebuilt binaries for macOS.