Agents Lookup

Use cases / Software development

Scripted multi-step AI workflows for repeatable code tasks

Define a fixed sequence of AI and shell steps in a file so tasks like grading tests or generating fixtures run the same way each time.

DifficultyIntermediate
Setup timeHalf a day
CategorySoftware development

The use case

Free-form agent chats are hard to repeat. Shopify open-sourced Roast, a framework for writing AI workflows as files: each step is either a model prompt or a plain command, run in order with outputs passed along. Shopify used it for jobs such as grading the quality of test files across a large codebase. Because the steps are fixed, results are easier to compare between runs and to review. Engineers still check outputs on a sample before acting on them, and keep deterministic steps like running tests as plain commands rather than asking the model.

Example workflow

  1. Write the task as ordered steps in a workflow file
  2. Mix prompt steps with shell or test commands
  3. Run the workflow on a few targets
  4. Check results on a sample
  5. Run across the codebase and review findings

Tools

Risks and guardrails

Sources

Written by Agents Lookup. Credits and official pages for the tools this use case names: