The JSON output
The JSON output of rqm
--format json is the stable contract. Every command prints one JSON object followed by one newline, to standard output, and prints nothing to standard error. Keys inside data are sorted. A change that removes or renames a key, or changes what one means, raises schema. A key may be added without that.
The envelope
| Key | Type | Meaning |
|---|---|---|
schema |
number | The version of this layout. It is 1. |
command |
string | The command that ran, or rqm when the command line could not be understood. |
ok |
boolean | True when the command succeeded, judged as if --strict were not given. |
data |
object or null | The result of the command. It is null when the command has none, for example when there is no project. |
diagnostics |
array | Every diagnostic, in output order. docs/DIAGNOSTICS.md lists the codes. |
truncated |
boolean | True when the cap removed diagnostics. E-DIAG-CAP is then the last one. |
The exit code is 0 when everything is fine, 1 when a gate failed, 2 for invalid input and 3 for an operational error. --strict makes a warning exit 2 and does not change the output.
Paths in the output are relative to the project root and use /. No output holds a directory of the machine, the time or a user name, so the same project gives the same bytes wherever it is.
files
Every command that reads the project puts files in its data:
| Key | Meaning |
|---|---|
discovered |
How many requirement files were found. |
read |
How many of them could be read. |
loaded |
How many became requirements. |
When loaded is less than discovered, the diagnostics say why, and the data holds only what loaded.
list
rqm list [--kind <name>]... [--status <word>]... prints the requirements by id number. A flag may be repeated: a requirement matches when it has any of the values of that flag, and it must match every flag that was given. An unknown kind or status is E-USAGE, not an empty list.
| Key | Meaning |
|---|---|
files |
See above. |
requirements |
An array with one object per requirement shown. |
counts |
total is the number of loaded requirements, and shown is the number that passed the filters. |
A requirement object has id, uuid, kind, status, priority and title. priority is the resolved priority, or null when neither the chain of parents nor the project sets one.
show
rqm show <id-or-uuid> prints one requirement. An id or uuid that no loaded requirement has is E-REQ-NOT-FOUND, and data is null.
| Key | Meaning |
|---|---|
files |
See above. |
requirement |
The requirement, below. |
run |
What a run proved about the requirement, below, or null when there is no run record. |
baseline |
What the baseline says about the requirement, below. |
manual_evidence |
The manual evidence the requirement declares, below. |
presentation |
The one label for the requirement, below. |
boundary |
A sentence that says what the label does and does not mean. It is the same in every output that shows a label. |
The requirement object:
| Key | Meaning |
|---|---|
id, uuid, kind, status, title, text |
As written. |
acceptance |
The acceptance criteria, in the order written. |
verification |
The methods, sorted. |
evidence, owner, rationale, notes |
As written, or null. |
labels |
As written. |
tests |
The test selectors, in the order written. |
priority |
null, or a value object (below). |
fields |
The fields of the kind that have a value, each a value object, keyed by name. |
parent |
null, or a linked object (below). |
children |
The requirements whose parent this is, by id number. |
links |
The requirements this one links to, in the order written. |
linked_from |
The requirements that link to this one, by id number. |
file |
The path of the file. |
bytes_hash |
The hash of the bytes of the file. |
effective_hash |
The hash of the meaning, as docs/CANON.md defines it. |
A value object has value, provenance and from. provenance is own when the requirement wrote the value, inherited when it comes from an ancestor and default when it comes from the project or from the default of the field. from is the id of the ancestor for inherited and null otherwise.
A linked object has id, uuid and title.
The run object tells what the latest run, or the run that --run <id> names, proved about the requirement. It has run_id, finished_at, evaluation, scope, counted and reasons. evaluation is satisfied, incomplete, unknown or unsatisfied, and none when the run did not evaluate the requirement. scope is current when the run evaluated the requirement as it is now, superseded when its effective hash has changed since the run, and null when evaluation is none. counted is the number of logical cases that counted as evidence, and reasons holds objects with kind and detail, as in test. A run is not a verification: whether the evidence is kept is for the baseline to say. A run record that cannot be used is E-RUN-RECORD, and a run that has no record is E-RUN-NOT-FOUND. The requirement is shown in both cases and run is null.
The baseline object has evaluation, stale_reason and record. evaluation is current (the baseline has a record whose effective hash, hash algorithm and policy fingerprint are the ones of the project now), stale, absent (a baseline was read and has no record for the requirement), invalid (the baseline cannot be used, and nothing in it is trusted) or disabled (no baseline is configured, or the requirement is a draft). stale_reason is set for a stale record and is algo_changed, policy_changed or content_changed, the first of the three that applies in that order. record is the record that the evaluation was made from, or null, and has result, run_id, verified_at, effective_hash, suites, cases and flaky_cases. A record that is stale is still shown, so that it can be read next to the run.
The manual_evidence object has declared, edited_since_record, hash, description and note. declared is true when a method other than test is listed. edited_since_record is true when the declaration differs from the one the baseline recorded, false when it is the same, and null when there is nothing to compare with. hash covers the methods and the evidence text and is null when nothing is declared. note always says that the evidence is an assertion by the author, because rqm cannot check it. Manual evidence never produces verified.
The presentation object has label, rule, pending_baseline, baseline_stale and explanation. label is one of not_applicable, not_gated, unknown, verified, verified_draft, verification_conflict, stale, pending_baseline, failed, incomplete and unverified. rule is the number of the rule that produced it (1, 1a, 2 to 11). pending_baseline is true when the run satisfied the requirement and the baseline record is stale. baseline_stale repeats stale_reason for every label, so that it is visible when the label comes from the run. explanation is a sentence for the labels that a bare word would mislead about, and null otherwise. For verification_conflict it says when the baseline recorded the requirement as verified and what the latest run says now. The label is computed from the stored facts and nothing else (see Presentation).
check
rqm check validates the whole project and runs nothing. Every diagnostic of reading, resolving and discovery is reported, and E-REQUIRED-UNBOUND is added for a required requirement that lists test in verification and has nothing bound to it. It exits 2 when any error was found.
With --baseline it is also the release gate. It reads the configured baseline strictly and says whether the baseline backs every required requirement. The gate reads no run record. A required requirement with test among its verification methods passes only when its label is verified; one without test passes only when its label is not_applicable and it declares manual evidence. The command exits 1 when any required requirement does not pass, and 2 when the project or the baseline has an error.
| Key | Meaning |
|---|---|
files |
See above. |
requirements |
How many requirements were checked. |
release |
The result of the release gate. It is absent when --baseline was not given. |
The release object has baseline, requirements, failing and boundary:
| Key | Meaning |
|---|---|
baseline |
The configured baseline path. |
requirements |
An array with one object per required requirement, by id number. |
failing |
How many of those requirements do not pass the gate. |
boundary |
The sentence that says what a label does and does not mean, as in show. |
A release row has id, uuid, verification, label, rule, baseline, stale_reason, manual_declared, explanation and passes. id and uuid name the requirement, verification lists its methods, and label and rule are the presentation of the requirement with no run in scope. baseline is the baseline evaluation (current, stale, absent or invalid) and stale_reason is set when it is stale. manual_declared says whether the requirement declares manual evidence, and explanation is the sentence that goes with the label, or null. passes is true when the row satisfies the rule of the gate.
coverage
rqm coverage [--run <id>] reports how many gated requirements fall into each of four classes: those with no tests bound to them (unbound), those with tests that did not run (unreported_source_test), those with tests that failed (not_passing), and those without a baseline record (not_recorded). Each class can be evaluated (when all required inputs are available) or not evaluated (when data is missing). A gated requirement is one with status approved or required, verification method test, and status != draft.
| Key | Meaning |
|---|---|
files |
See above. |
gated |
The number of requirements that are gated. |
run |
The run that the run classes come from (the latest run, or the one selected with --run), or null when there is no run record. |
baseline |
configured when a baseline is configured and usable, disabled otherwise. |
classes |
An array with one object per class, in order: unbound, unreported_source_test, not_passing, not_recorded. |
legend |
Two strings that explain the meaning of classes and evaluation states. |
boundary |
A sentence that says what the label does and does not mean, as in show. |
selected |
An array of class names that were selected with --class, empty when every class is shown. |
A class object has class, state, missing, requirements and count. class is one of unbound, unreported_source_test, not_passing, not_recorded. state is evaluated when all required inputs are available, not_evaluated otherwise. missing names the missing input when not evaluated (e.g., run or baseline), or is null when evaluated. count is the number of requirements in the class when evaluated, or null when not evaluated. requirements is an array of requirement objects in the class, empty and meaningless when not evaluated.
A requirement object in a class has id, uuid, and detail. detail explains why the requirement is in the class: for unreported_source_test it is null, for not_passing it is the run evaluation word (none, incomplete, unsatisfied, unknown, or superseded when the run evaluated an older version), and for not_recorded it is the baseline evaluation (stale, absent, or invalid). For stale records, the reason is appended as a suffix (e.g., stale_content_changed). A requirement may also have tests, an array of bound test objects with suite, case and annotated fields, but only for unreported_source_test class entries (listing the tests absent from the report) and only when data is available.
tests
rqm tests lists the Rust tests that the scan found, and the requirements that claim each of them. It reads sources and runs nothing. A project without a Cargo.toml has no targets and no tests, and that is not an error.
| Key | Meaning |
|---|---|
files |
See above. |
targets |
An array with one object per Cargo target that has tests to scan, sorted by package, kind and name. |
tests |
An array with one object per test declaration, sorted by identity. |
counts |
targets and tests are the lengths of the two arrays, and claimed is the number of tests that at least one requirement claims. |
A target object has package, kind (lib, proc-macro, bin or test), name, crate_name, binary_id and root. crate_name is the first segment of the identity of its tests, without the ending that two targets of one crate name need. binary_id is the id that nextest gives to the test binary of the target. root is the root file of the target, relative to the project root.
A test object has id (the identity, see docs/IDENTITY.md), form (test, tokio::test, rstest, test_case or quickcheck), ignored, file, line, col and requirements. line and col give the position of the name of the function. requirements holds one { "id": ..., "via": [...] } object for each requirement that claims the test, by id number, where via lists annotation when a marker claims it and selector when the tests list of the requirement names it. A selector that names a test that no declaration has is not listed here.
affected
rqm affected [PATH]... lists the requirements whose file is among the changed paths, and the requirements bound by annotation or selector to tests declared in a changed file. Without a path it reads one path per line from standard input. Paths are relative to the project root; a path that cannot be inside the project is W-AFFECTED-PATH-IGNORED and skipped. The result does not depend on the order or the repetition of the paths, and the command starts no program except the source scan.
| Key | Meaning |
|---|---|
files |
See above. |
paths |
The paths that were accepted, normalized, sorted and without repetition. |
configuration_changed |
True when rqm.yaml is among the paths. |
requirements |
An array with one object per affected requirement, by id number and then uuid. |
An affected requirement object has id, uuid, title and reasons. A reason object has kind, path, test and line. kind is requirement_file when the file of the requirement changed, test when a changed file declares a test that the requirement claims by annotation or selector, and configuration when the configuration file changed and the requirement is gated (it has the test method and is approved or required). path is the changed file. For test, test is the identity of the test and line is the line of its declaration; otherwise both are null. Reasons are sorted and hold no repetition.
trace
rqm trace <id-or-uuid> [--run <id>] lists the tests bound to one requirement, where each is declared, how it is bound, and what a run said about it. It reads sources and a run record and runs nothing. An id or uuid that no loaded requirement has is E-REQ-NOT-FOUND, an argument that is neither is E-USAGE, and a run that has no record is E-RUN-NOT-FOUND.
| Key | Meaning |
|---|---|
files |
See above. |
requirement |
The requirement that is traced, below. |
tests |
An array with one object per bound test, by identity. |
run |
The run the results come from, below, or null when there is no run record. |
links |
The requirements this one links to, in the order written, as linked objects. It is what the requirement authored, and it never adds a reciprocal link. |
boundary |
The sentence from show, which is part of every output that shows a label. |
A traced requirement object has id, uuid, title, status and verification.
A traced test object has id, file, line, via, result and cases. file and line are the declaration of the test, or null when the scan has no declaration of it. via lists annotation and selector, in that order. result is the result of the test in the run: passed, flaky, skipped, failed, errored, or not_in_run when the run credited no case of the test to the requirement. It is null when there is no run record. cases is the number of cases of the test that the run credited to the requirement. The worst final outcome wins, in the order errored, failed, skipped, passed, and a passed result with a flaky case is flaky.
A trace run object has run_id, scope and evaluation. scope is current or superseded, as in show, and null when the run did not evaluate the requirement. evaluation is the run evaluation of the requirement: satisfied, incomplete, unknown or unsatisfied, or none when the run did not evaluate it.
test
rqm test [--suite <id>]... [--exclude <id>]... [--promote] [--create] [--prune] runs the suites, reads their reports and says what the run proves. It writes the run record and never the baseline itself: with --promote it hands the run it just made to the promotion of baseline update, which writes the baseline at once. It starts nothing and writes nothing when the project has an error or the suite selection is not usable.
| Key | Meaning |
|---|---|
files |
See above. |
run_id |
The id of the run, 15 characters. It differs from run to run. |
record |
The run record, relative to the project root, or null when it could not be written. |
selection |
suites are the suites that ran and excluded the suites that were left out, both in the order of rqm.yaml. |
suites |
An array with one object per suite that ran, in the order they ran. |
requirements |
An array with one object per requirement that has a run evaluation, in uuid order. |
counts |
How many requirements got each run evaluation: satisfied, incomplete, unknown and unsatisfied. |
gate |
An array with one object per requirement that fails the status gate. Empty when none does. |
not_run |
The annotated tests that no clean report ran, by identity. |
failed_cases |
The failing cases of the mandatory suites that were clean, as “suite: class::name”. Skipped cases and cases that passed on a later attempt are not listed. The command fails with exit 1 because of them only with –fail-on-test-failure. |
promotion |
What the promotion with --promote reported, or null without --promote and when the promotion was refused or failed. It has the data of baseline update: files, run_id, path, created, written, bytes_hash, full_run, promoted, retracted, untouched, orphaned and records, where retracted and untouched hold objects with id and reason, and orphaned holds objects with id and dropped. |
A suite object has id, execution (mandatory or optional), clean, fault, cases, ended, exit_code, signal, log and report. clean is true when the process ended normally, the report could be read, the report is not older than the start of the run and it holds enough cases. fault is null for a clean suite, and otherwise the first thing that failed: process, report, stale or too_few_cases. cases is the number of logical cases in the report. ended says how the process ended: exited, timeout, timeout_with_survivors, left_processes or not_started. exit_code and signal are kept apart, and either is null when it does not apply. log and report are paths relative to the project root.
A requirement object has id, status, gated, run, counted, reasons, baseline and presentation. run is the run evaluation: satisfied, incomplete, unknown or unsatisfied. gated is true when the requirement has the test method and is approved or required. counted is the number of logical cases that count as evidence. reasons holds one { "kind": ..., "detail": ... } object for each reason, where kind is one of case_failed, case_doubt, suite_unclean, selector_unmatched, too_few_cases, ambiguous_case or not_gated, and detail names the case, suite or selector, or is null. baseline is the baseline evaluation of the requirement (current, stale, absent, invalid or disabled), and presentation is the object described under show, with label, rule, pending_baseline, baseline_stale and explanation. A baseline state never changes the exit code of test: a new or edited requirement is pending_baseline or stale until it is promoted. boundary is the sentence from show, which is part of every output that shows a label.
A gate object has id, status, run and wanted. wanted is satisfied for a required requirement and anything but unsatisfied for an approved one.
baseline update
rqm baseline update [--run <id>] [--create] [--prune] promotes the evidence of a run into the baseline. It is the only command that writes the baseline. It refuses, and writes nothing, when the project has an error, when the baseline is not the one the run was made against, or when the project changed in a way the evidence depends on.
| Key | Meaning |
|---|---|
files |
See above. |
run_id |
The run that was promoted. |
path |
The baseline, relative to the project root. |
created |
True when the file did not exist and was created. |
written |
True when the file was replaced. It is false when the plan changes nothing, and then the file is left as it was. |
bytes_hash |
The hash of the bytes that were written, or null when nothing was written. |
full_run |
True when every configured suite was selected and clean. Only a full run may retract a record because evidence is incomplete. |
promoted |
The ids of the requirements whose record was written or refreshed. |
retracted |
An array of objects with id and reason: unsatisfied, unknown, incomplete or policy_changed. |
untouched |
An array of objects with id and reason, for records the run cannot speak for. |
orphaned |
An array of objects with id and dropped, for records of requirements that are gone or no longer gated. dropped is true with --prune. |
records |
The number of records the baseline has after the update. |
init
rqm init --default-kind <name> [--force] [--no-gitignore] writes rqm.yaml. The default kind is mandatory: it is declared in the file, with no fields, and new requirements have it unless --kind says otherwise. It refuses with E-INIT-EXISTS when a project exists at or above the working directory, unless --force replaces the nearest rqm.yaml.
| Key | Meaning |
|---|---|
config |
The file that was written, relative to the working directory. |
default_kind |
The default kind that was declared. |
flavor |
cargo when a Cargo.toml is next to it, and generic otherwise. |
replaced |
True when an existing file was replaced. |
gitignore |
created, updated, unchanged, no_repository or skipped. |
next_steps |
The things to do next, in order, as text. |
create
rqm create --title <text> [--kind <name>] [--text <text>] [--acceptance <text>]... [--field name=value]... writes the file of a new draft requirement.
| Key | Meaning |
|---|---|
id |
The new id: one more than the highest number in the project. |
uuid |
The new uuid, version 7. It is the only value of any command that differs from run to run. |
kind |
The kind of the requirement. |
file |
The file that was written, relative to the project root. |
files |
See above. It describes the project as it was before the file was written. |
rename
rqm rename <REQUIREMENT> <NEW-ID> changes the id of a requirement and every reference to it that is written as the old id. It refuses a project with errors, a new id that is not an id, the same id, and an id that another requirement has (E-ID-DUPLICATE), and writes nothing in those cases. References written as a uuid, the test selectors and the name of the file are not changed.
| Key | Meaning |
|---|---|
from |
The old id. |
to |
The new id. |
uuid |
The uuid of the requirement, which does not change. |
files |
An array with one object per changed file, sorted by path. |
references |
The number of places changed in all files, the requirement’s own id included. |
A file object has path (relative to the project root), kind (requirement or source) and places (the number of places changed in that file).
edit
rqm edit <REQUIREMENT> opens the requirement in the editor named by $VISUAL or $EDITOR, checks the result and reports the diagnostics.
| Key | Meaning |
|---|---|
id |
The id of the requirement. |
uuid |
The uuid of the requirement. |
file |
The path of the file. |
changed |
False when the editor left the copy as it was, so nothing was written. |
valid |
False when the edited file has errors. The file keeps the edited text. |