DevPik Logo
ai agentsresearchclaude codereproducibilityopen source

OpenResearch: Give Every Experiment Its Own Agent and Its Own Worktree

A local-first workspace that gives each research direction its own agent session and its own git worktree, then tracks every run in a git-native experiment tree. How it differs from Karpathy's autoresearch, and the shared-cluster gotcha its own README admits to.

ByMuhammad Tayyab12 min read
All open source picks
alphaXiv/OpenResearch
The official repository — this write-up is not affiliated with the project.
5.2kRustMIT

What OpenResearch Is

OpenResearch is a local-first workspace that turns Claude Code, Codex, OpenCode or Cursor into a research agent. Not a chatbot that summarises papers, but something that reviews literature, forms a hypothesis, changes code, runs the experiment, reads the evidence and decides what to try next.

It comes from alphaXiv, the team behind the arXiv discussion layer of the same name. It is written in Rust, MIT licensed, hit number one on GitHub Trending, and passed 5,200 stars roughly three months after the repository was created.

The pitch that matters is not "AI does science". It is a much narrower and more defensible claim: research is already an embarrassingly parallel search over ideas, and the tooling has never treated it that way.

This Is Not Karpathy's autoresearch

The README uses the word "autoresearch", and that word now belongs to a different project, so it is worth separating them before going further.

karpathy/autoresearch is Andrej Karpathy's roughly 630-line loop that hands an AI agent a small LLM training setup and lets it experiment overnight on a single GPU. It modifies code, trains for a few minutes, checks the result, and iterates. It is deliberately minimal, and it is the thing most of the current writing about "autoresearch" is describing.

alphaXiv/OpenResearch is a workspace built around the same insight at a much larger scale. Instead of one loop on one GPU, it runs many directions at once, each with its own agent session and its own isolated checkout, and keeps a permanent record of how every run relates to every other.

The relationship is roughly script to IDE. Karpathy's version demonstrates that the loop works. OpenResearch is the answer to "fine, now run forty of these and tell me which branch of the search actually went anywhere".

If you are choosing between them, the honest split is: one GPU and one idea, take the 630 lines. Multiple directions, multiple machines, or any need to reconstruct what you did last week, take the workspace.

One Direction, One Agent, One Worktree

The central mechanism is simple enough to describe in a sentence: every research direction gets an independent agent session and an isolated git worktree.

If you have run several coding agents at once, you already know the failure this prevents. Two agents in one checkout will overwrite each other's edits, fight over the branch, and leave you unable to tell which change produced which result. The usual fix is to clone the repository several times, which wastes disk and detaches the histories.

A git worktree is the right primitive instead. It is a separate working directory with its own branch, sharing the same repository history and remote as your main checkout. Isolation without duplication.

Anthropic ships worktree support in Claude Code for exactly this reason, and the parallel-agent workflow has become common enough that it has a name. OpenResearch takes the same primitive and moves it up a level: the unit of parallelism is not "a task" but "a hypothesis", and the worktree lives as long as the direction does rather than as long as the session does.

That difference sounds small and is not. A task finishes. A research direction gets revisited three weeks later when a different branch fails and you want to know whether the thing you abandoned was actually worse.

The Experiment Tree

The second idea is where the real value sits, and it is the one most likely to survive even if you never install this.

OpenResearch tracks experiment variants in a git-native experiment tree, and every run receives an immutable archive of the exact commit it was recorded against. Logs, diffs, files, results and artifacts stay attached to the work that produced them.

Anyone who has tried to reproduce a result six months later knows why this matters. The usual state of affairs is a results table in a notebook, a directory of checkpoints named v2_final_REAL, and no reliable way to recover which code produced which number. Experiment trackers exist to solve this and mostly solve half of it: they record metrics faithfully and record the code loosely, usually as a commit hash that may or may not still exist on a branch somebody has since rebased.

Making the version control system the experiment tracker, rather than bolting a tracker alongside it, closes that gap. The lineage of an idea and the lineage of the code become the same graph.

When you do write the results up, a markdown table generator saves a lot of manual alignment, and the loss values and learning rates that come out of these runs are usually easier to read through a scientific notation converter than as raw floats.

Autoresearch: The Full Loop

With the worktrees and the experiment tree in place, the autonomous mode follows naturally. OpenResearch can run the whole cycle without a human in it: propose an idea, change the code, launch an experiment, inspect the evidence, decide what to try next.

Multiple agents can explore different directions simultaneously while the experiment tree preserves their lineage. That last clause is doing the work. Autonomous exploration without recorded lineage produces a pile of results nobody can interpret, which is worse than no results, because it looks like progress.

Be realistic about what this buys you. The agent is not going to have the idea that makes the paper. What it can do is the part researchers actually lose weeks to: running the ablation you already knew you should run, sweeping the parameter you were avoiding, and checking the baseline you assumed was fine. That is a large fraction of real research time and almost none of the interesting part.

The failure mode to watch for is a well-recorded search through an uninteresting space. The tool will happily run overnight in a direction that was never going to work, and the experiment tree will document it beautifully.

Run Anywhere

The same committed source snapshot runs locally, over SSH, or on Slurm, Kubernetes, Ray, Hugging Face Jobs, Modal, Tinker, or alphaXiv's managed compute. Publishing the repository is not required, which matters for unpublished work.

The Slurm and Kubernetes support is the tell that this was built by people who have actually done research in an academic environment rather than in a startup with a company AWS account. Slurm is what university clusters run, and tooling that ignores it ignores most of academia.

There is a nice pattern for the common setup where your GPUs are somewhere else and your browser is not:

sh
orx up --remote user@host

That runs the workspace next to the remote GPUs while you drive it from the browser on your laptop. SSH config aliases and custom ports work. Local models are supported too, through LM Studio, oMLX, Ollama or a custom endpoint with OpenCode, so the agent itself does not have to be a hosted frontier model.

Experiment configuration in this world is YAML more often than not, and a YAML formatter is a quicker way to find a broken indentation level than reading a stack trace from a job that died four minutes into a queue slot.

Installing It

Two commands on macOS or Linux:

sh
curl -LsSf https://openresearch.sh/install.sh | sh
orx up

orx up opens the local dashboard at http://127.0.0.1:4791. Windows is a beta download and needs Git for Windows installed first. macOS needs 11 or later.

To let your coding agent drive it directly, orx install-skills installs the OpenResearch skill into supported agents. After that the CLI is the interface:

CommandWhat it does
orx projectsList projects
orx runs <project-id>List runs for a project
orx logs <run-id>Read a run's logs
orx exp run <experiment-id>Launch an experiment
orx discover keyword <query>Search the literature
orx paper <arxiv-id-or-doi>Pull a specific paper

orx discover and orx paper are the alphaXiv heritage showing through. Literature search is a first-class command rather than something you tab away to a browser for, which is the correct place for it in a loop that is supposed to run without you.

An account at openresearch.sh is optional and only used for service-owned features: organisations and managed compute. Everything else works signed out.

Local by Default, With One Caveat They Actually Tell You About

OpenResearch runs on 127.0.0.1 with a local SQLite store. Creating a project or launching a run does not publish your code. For unpublished research that is the whole argument, and it is a meaningfully different posture from tools that require a cloud account before you can start.

Then there is this line in the README, which most coverage of this repo will skip:

The remote service binds to loopback and has no application-level authentication, so other users on that host can reach it.

Read that carefully, because the reassuring half and the dangerous half are in the same sentence. Loopback binding is normally a security property. It stops the network reaching you. It does not stop anyone already logged into the same machine.

Now think about where research compute lives. Shared university clusters. Slurm login nodes. A lab server with fifteen accounts on it. On a single-user laptop, loopback means private. On a shared host, loopback means "anyone with a shell here", and the service has no authentication of its own to fall back on.

So if you run orx up --remote user@host against a multi-user box, treat the workspace as readable by every other user on it. Use a machine you control, or an SSH tunnel to a loopback port nobody else can reach, and do not assume the bind address is doing work it is not doing.

Credit where it is due: they wrote this down plainly instead of letting people discover it. That is rarer than it should be, and it is a reasonable signal about the rest of the engineering.

Telemetry, and How to Turn It Off

Official release builds send opt-out usage events tied to a random installation ID. The exclusion list is specific rather than vague: no code, no prompts, no file contents or paths, no repository names, no tokens, no emails, and no project or experiment identifiers.

sh
orx telemetry off
orx telemetry status
orx <command> --no-telemetry

Builds from source send nothing at all.

Opt-out rather than opt-in will annoy some people, and that is a fair objection. What makes it defensible here is that the exclusion list is written as a list of specific things rather than a promise to be careful, and the off switch is a single command that is documented in the README rather than buried in a settings page.

Who This Is Actually For

Not everyone doing research needs this, and the honest sorting is fairly clear.

It fits if you run ML experiments, you have more ideas than time, you already juggle several agent sessions, and you have at some point failed to reproduce your own result from last month. The parallel-worktree and experiment-tree combination is aimed precisely at that person.

It does not fit if your research is not code-and-experiment shaped. A literature review, a theory paper, or qualitative work gets very little from an experiment tree. If you want an agent to read and summarise papers, alphaXiv's own platform is closer to what you want than this is.

It is probably overkill if you have one GPU and one idea. Karpathy's autoresearch is 630 lines and will teach you whether the loop suits how you work, at a much lower cost than adopting a workspace.

The thing worth taking from it regardless of whether you install it is the design claim: the unit of research is a direction, not a task, and version control is already the right substrate for tracking directions. That idea outlives any particular tool built on it.

Frequently Asked Questions

Is OpenResearch the same as Karpathy's autoresearch?

No. karpathy/autoresearch is a roughly 630-line loop that lets an agent run overnight ML experiments on a single GPU. alphaXiv/OpenResearch is a full local-first workspace applying the same idea at scale: many directions in parallel, each with its own agent session and isolated git worktree, plus a git-native experiment tree that records how every run relates to every other. The relationship is roughly script to IDE. One GPU and one idea suits the 630 lines; multiple directions or a need to reconstruct past work suits the workspace.

Which coding agents does OpenResearch work with?

Claude Code, Codex, OpenCode and Cursor, with the harness and model chosen per session. Run orx install-skills to install the OpenResearch skill into supported agents. You can also connect a local model through LM Studio, oMLX, Ollama or a custom endpoint with OpenCode, so the driving agent does not have to be a hosted frontier model.

Why does it use git worktrees?

So parallel agents do not overwrite each other. A git worktree is a separate working directory with its own branch that shares the same repository history and remote, giving isolation without the duplication of cloning the repo several times. OpenResearch moves the primitive up a level: the unit of parallelism is a research direction rather than a task, so the worktree lives as long as the hypothesis does rather than as long as the session does.

How does the experiment tree help reproducibility?

Every run gets an immutable archive of the exact commit it was recorded against, and logs, diffs, files, results and artifacts stay attached to the work that produced them. Conventional experiment trackers record metrics faithfully but record code loosely, usually as a commit hash that may no longer exist after a rebase. Making version control the tracker rather than bolting a tracker alongside it means the lineage of an idea and the lineage of the code are the same graph.

Where can OpenResearch run experiments?

Locally, over SSH, or on Slurm, Kubernetes, Ray, Hugging Face Jobs, Modal, Tinker and alphaXiv's managed compute, all from the same committed source snapshot. Publishing the repository is not required. Slurm support in particular signals it was built by people familiar with academic clusters, since that is what most university HPC actually runs.

Is it safe to run on a shared cluster?

Be careful. The README states plainly that the remote service binds to loopback and has no application-level authentication, so other users on that host can reach it. On a personal machine loopback means private; on a shared university cluster or a multi-user lab server it means anyone with a shell on that box, with no authentication to fall back on. Use a host you control, or an SSH tunnel to a port other users cannot reach.

Does OpenResearch upload my code or research?

No. It runs on 127.0.0.1 with a local SQLite store, and creating a project or launching a run does not publish your code. An openresearch.sh account is optional and only used for service-owned features such as organisations and managed compute. Release builds do send opt-out usage events tied to a random installation ID, excluding code, prompts, file contents and paths, repository names, tokens, emails and project identifiers. Turn it off with orx telemetry off, or build from source, which sends nothing.

What does it cost?

The software is free and MIT licensed, so you can run the whole thing locally or on your own infrastructure at no cost beyond your own compute. alphaXiv sells managed compute as an optional service for people who do not want to supply their own GPUs, but nothing about the local workspace is gated behind it.

Related DevPik tools

Muhammad Tayyab

Written by

Muhammad Tayyab

CEO & Founder at Mergemain

Muhammad Tayyab builds free, privacy-first developer tools at DevPik. He writes about AI trends, developer tools, and web technologies.

More open source picks