Advanced Coding Expert
About this opportunity
We are hiring experienced software engineers to design long-horizon coding tasks: realistic, investigation-heavy engineering problems drawn from substantial codebases, used to evaluate and train advanced AI coding agents. The ideal candidate is comfortable working across large multi-file codebases, reasoning about how a bug or feature ripples across modules, and building reproducible, containerized environments backed by rigorous tests. You should be able to author a problem a strong engineer would recognize as real work, and a verifier that grades a solution the way a careful reviewer would. Experience with Docker and containerized workflows is required. You will also need a substantial private codebase of your own to author tasks against.
Responsibilities:
• Author long-horizon coding tasks grounded in real engineering problems (bugs, features, regressions, migrations) from substantial codebases.
• Write clear prompts that require genuine investigation and complicated execution by a coding agent.
• Build deterministic test suites and review criteria that verify correctness, catch regressions, and resist shortcut or "reward-hacking" solutions.
• Containerize tasks with Docker so they build and run reproducibly.
Required qualifications:
• Proficiency in at least one modern programming language (e.g., JavaScript/TypeScript, Java, Go, Rust, C/C++, Python, etc.).
• Experience architecting or maintaining substantial, complex, and large software projects.
• Hands-on experience with Docker or similar containerization tools.
• Strong technical writing, documentation, and testing skills.
• Access to a large, complex, and private codebase you own and can author tasks against.
• Experience using coding agents for long tasks.
Preferred qualifications:
• Background in code review, test authoring, or maintaining production and/or open-source systems.
• Exposure to AI evaluation, rubric/data grading, or prompt writing.
Check the original posting for the complete requirements and current terms before applying. Applications are handled by AFTERQUERY.