Automated small-scale ML research experiments
An agent proposes ML research ideas, writes and runs experiment code, and drafts a write-up, which researchers vet carefully.
The use case
Machine learning researchers run many small experiments to test ideas. Sakana AI's open-source AI Scientist generates research ideas from a template, writes and runs experiment code, analyzes results and drafts a paper, with an automated reviewer scoring it. Sakana itself notes flaws, such as experiments that do not support conclusions. It is best used to explore many small ideas cheaply. Researchers still check the code and results, rerun anything promising by hand, and never submit or publish generated papers without full human authorship and review.
Example workflow
- Choose a template and research area
- Agent proposes ideas and runs experiments
- Researcher reviews code and results
- Rerun promising results independently
- Write up real findings yourself
Tools
- AI Scientist
- Python
- PyTorch
Risks and guardrails
- Generated code may run outside a sandbox; isolate it
- Conclusions may not follow from results
- Do not submit generated papers as your own work
Sources
Written by Agents Lookup. Credits and official pages for the tools this use case names:
- Sakana AI: The AI Scientist: sakana.ai/ai-scientist