Working with edhil
Working with edhil
edhil is the tool for the last two steps of a stage-gated way of working: the work is designed and refined by a person with an assistant, and then implemented by an agent under the control of the tool. This chapter shows how the stages map onto the commands. The way of working itself (the stages, the documents and the gates) is a separate document; this chapter only summarizes it.
The loop
design -> plan -> for each epic:
refine it into work items (documents, outside edhil)
|
edhil import (documents -> forge issues)
|
edhil implement (issues -> commits)
|
review the code, edhil log, edhil work-items
|
findings -> work items (edhil import --epic)
|
the next epic, refined from the code as it is now
Everything above the line edhil import is writing and agreeing. edhil does not design, plan or refine anything. It takes the finished work items and turns them into commits.
Stages 1 and 2: design and plan
The design document says what is being built and why, and the plan splits it into milestones and epics. They live in your repository or wherever your team keeps documents. edhil does not read them. The only thing to take from them is how big an epic should be: an epic is one chunk of work that you can review in one sitting, with work items that each become one commit.
Stage 3: refining an epic
Each epic is refined on its own, from the code as it is after the previous epic. The refinement has three steps, each with a gate where a person says yes:
- A pre-study, only when there are open technical questions that change the scope: a benchmark, an API test, a dependency check. It is done and reported before the epic is refined.
- A breakdown: the work items with one line each, for you to check the scope and the order.
- Detailed work items: the document that edhil imports. It is the specification the agent works from.
The last one is the part that matters for edhil. A work item is implemented from its description alone: the agent gets the title and the whole text of the description, the rules, and nothing else. Whatever it needs to know has to be in the work item. A good work item therefore has:
- A high level summary: the goal in plain sentences, then what the work item does. Self-contained.
- Steps: the files to change with their paths, the code as a diff or snippet, and the unit tests to write, specific enough that the tests check real behavior.
- A definition of done: statements that someone other than the author can check.
- Verification steps: the commands that edhil runs after the agent is done, one per line in one
verifyblock. They are the check that the work item is finished, so they belong to the work item and not to the project.
The template in the epic template has this shape, and the format is strict: The epic document format lists the rules and edhil import the command. An epic also names its project in a Project line, the number of the project issue in the forge. Three more things make the difference in practice:
- Order the work items by dependency, and number them in that order.
implementruns an epic in ascending order of issue numbers, andimportcreates the issues in the order of the document, so the order of the document is the order of the run. - Choose verification commands that can fail for the right reason. Each command runs without a shell, in the repository root, in the order written, and the first failure stops the verification. Run the quick ones first. Which commands may run, and how long each may take, is decided by the project in
integration.verification(see Configuration). - Put what the whole epic decides in the epic, not in a work item, and repeat in a work item what it relies on. If a work item needs a design decision that is not written down, the agent answers that it is blocked and the run pauses; that is the intended behavior, and the fix is to write the decision down.
Importing the epic
Check the document first. The preview parses it exactly as the import does and creates nothing:
edhil import epic-3.md --dry-run
The same check is available without edhil, for a pipeline or for the assistant that writes the document: python3 tools/edhil-validate/edhil_validate.py epic-3.md.
Look at the epic title and at the titles of the work items. Then import it:
edhil import epic-3.md
edhil creates the epic, one issue for each work item, and the links, and prints their numbers. Look at one of them in the forge: the description is the text of the block. From now on the issues are the source of truth for the epic, and you can edit them there before they are implemented. Keep the format when you edit: edhil implement reads the verification commands back from the description, and refuses a work item whose description is not in the format. Issues that were made before the format existed have to be edited into it before they can be implemented.
Stage 4: implementing
Look at what is open, then implement the epic:
edhil work-items
edhil implement 123
123 is the number of the epic that import printed. The plan shows the commands that will verify each work item and asks before it starts. All the open work items run in order, each ending in a commit on your current branch. Start from a clean tree on the branch you want the commits on. Watch the first run: you learn what the agent does well and badly on your project, and the output is shown as it happens.
When a run pauses, look at what happened (the block names the work item and the step), and look at the tree. Then answer:
- retry when the cause is gone or random: a flaky test, a timeout, a command you fixed in another terminal;
- ignore when the failure does not matter for this work item and you accept the change as it is;
- discard when the commit failed or git failed and you do not want the change at all: edhil restores the tracked files and removes the new ones (the
.edhil/directory is kept) and goes on with the next work item; - abort when you want to fix it yourself. The change stays in the working tree.
After an abort you can fix the code by hand and commit it with edhil commit 123, which drafts the message, so the history looks the same as for the work items that edhil committed. Then go on with the rest. The work item you fixed by hand is still open in the forge and edhil does not know that it is done, so an epic number would run it again: name the remaining work items instead, for example edhil implement 125 126. (The work items that edhil committed are marked in edhil work-items, and an epic number leaves them out.) Close the issue yourself when you are satisfied.
To see what happened afterwards:
edhil log
edhil log 12
edhil log 12 --item 124
edhil log 12 --item 124 --step 1
The first lists the runs, the second reports one run, the third one work item with its commit message and events, and the last one step, with the prompt, the transcript and the full output. The journal keeps the patch of every commit too (--patch).
The review
After the epic is implemented, review the code. The commits are ordinary commits on your branch, so use your usual tools. A review ends in one of four decisions, and each has a command:
| Decision | What you do |
|---|---|
| No change is needed | Nothing. Go on to the next epic. |
| Work items are added to this epic | Write them in a document, with no epic heading, and run edhil import findings.md --epic 123. Then edhil implement the new numbers. |
| A new epic is written | Write it like any epic and edhil import it. |
| The work items go first in the next epic | Write them as the first work items of the next epic when you refine it. |
Small fixes you would rather make by hand are just commits: use edhil commit to get the same message format.
Starting the next epic
Hand the code as it is now to whoever refines the next epic: the diff of the commits (git log -p from the commit before the first run), or a snapshot, with the reports of the runs if the review needs them. The next epic is refined against that, not against the original design, because the code may have moved in ways that change the scope.
Rules of thumb
- One run per epic, started from a clean tree, on the branch the commits are meant for.
- Keep the work items small enough that the verification takes minutes and the diff can be read in one go.
- A verification command that is slow or flaky makes every work item slow or flaky. Make it fast and reliable first, and give the slow ones a
timeout_minutes. - The agent cannot ask. Anything the work item leaves open will be decided by the agent or end in a pause.
- Nothing is pushed and no issue is closed by edhil. Review, push and close when you are satisfied.
The epic template
This is a document that edhil import reads. Copy it and fill it in. The headings must keep their form (see edhil import): the epic heading # Epic N - Title followed by the Project line, the work item headings ## WI #N.M - Title, the four sections of a work item with exactly one verify block, and a line of three hyphens (---) between the blocks.
# Epic 1 - A title that says what the epic delivers
**Project:** #161
**Milestone:** the milestone this epic belongs to
The Project line is the number of the project issue in the forge that this epic belongs to. Two or three paragraphs: the goal of the epic, what is decided for the whole epic, and the order of the work items. Anything that every work item depends on belongs here, because the agent only sees the work item it is implementing.
---
## WI #1.1 - A title that says what the work item does
**Depends on:** nothing
### High level summary
The goal of the work item in plain sentences, then what the work item does. It must make sense without any other document.
### Steps
The files to change, with their paths, and the code to insert or change as a diff or a full snippet. The unit tests to write and what each one checks.
### Definition of done
1. A criterion that someone other than the author can check.
2. The checks of the project pass.
### Verification steps
One command per line, run in the repository root, in order. Each must succeed.
```verify
yet cleanify
cargo test -p cratename
yet prepush
WI #1.2 - The next work item
Depends on: WI #1.1
High level summary
Builds on WI #1.1. The reference is rewritten to the real issue number when the document is imported.
Steps
…
Definition of done
- …
Verification steps
yet prepush
```