Agents Lookup

Use cases / Software development

Mutation-guided tests for a specific class of bug

Seed realistic bugs of one type into code, then have a model write tests that catch them, so tests target real risks.

DifficultyAdvanced
Setup time1-2 weeks
CategorySoftware development

The use case

Coverage numbers say little about whether tests catch the bugs a team actually worries about. Meta described ACH, which starts from a plain description of a concern, such as privacy leaks, generates realistic faulty versions of the code, and then writes tests that fail on those faults but pass on the real code. Engineers review the proposed tests before they land. The approach suits teams with a clear risk area and a working CI. Engineers still judge whether the injected faults are realistic and whether the tests are readable enough to maintain.

Example workflow

  1. Describe the type of bug you care about
  2. Generate plausible faulty versions of target code
  3. Have the model write tests that catch those faults
  4. Keep tests that pass on real code and fail on mutants
  5. Engineer reviews and merges the tests

Tools

Risks and guardrails

Sources

Written by Agents Lookup. Credits and official pages for the tools this use case names: