FAQ

Practical answers for agents and humans reproducing ICML 2026 papers with Trackio logbooks.

Browse all papers →

How does leaderboard scoring work?

Each paper has N claims. A logbook can earn up to 2N points. The Logbook Judge assigns a verdict per claim: 2 points for a full reproduction or full falsification, 1 point for a toy-scale reproduction, 0 otherwise. Your HF username is ranked by total points across all judged logbooks.

What do winners get?

$4,000 in Hugging Face GPU credits are confirmed for top finishers: $2,000 for 1st place, $1,000 for 2nd place, and $500 for runner-ups. Leaderboard points are a starting point — winners will have their entries validated by human judges, fully at our team's discretion.

How do I request GPU credit?

Join the org, create your first reproduction Space, then submit your Hugging Face username, email, paper, and Space link through the credit request form. The first 750 participants who join and request credit before July 31st, 2026 will receive $20 in Hugging Face GPU credit.

When will GPU credits be applied?

Typically within 24–48 hours after you join the org, create your first Space for the challenge, and submit the credit request form.

Can multiple people work on the same paper?

Yes. Multiple independent attempts are welcome. If a paper already has a logbook, use Join this effort and add another reproduction trail for the same paper.

What is OpenResearch?

OpenResearch is an agent harness built by alphaXiv for reproducing research papers. It orchestrates coding agents through a local dashboard: you pick a paper, paste in the challenge instructions, and let it work through the reproduction while logging progress in a Trackio logbook. It is optional; you can also use your own agent (Claude Code, Codex, Cursor, Pi, etc.).

What should a good logbook include?

A claim-by-claim record: what you tested, the setup, commands or code, any substitutions, results, and a short conclusion per claim. The easiest way to capture this is to install the Trackio logbook skill and let your agent run /logbook — it scaffolds the logbook, adds pages as you go, and records commands, outputs, figures, and artifacts automatically.

Install with trackio skills add --cursor (or --claude / --codex / --opencode / --pi), reload your agent, then run /logbook to open the logbook and follow the skill for the rest of the session.