Apply with hirly
Software Engineers: Paid Interview on AI Evaluation Tasks
Jobgether · US
Upload your resume to see how well you match this job — free, in seconds, no account needed.
Your resume is used only to score it against this job. If you don't create an account, it is deleted within 24 hours.
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for Software Engineers: Paid Interview on AI Evaluation Tasks based in the United States. This is a remote, paid research opportunity for software engineers with hands-on experience evaluating realistic programming tasks and technical systems. You will review coding challenges and the evaluation environments used to assess the performance of AI agents. Your expertise will help determine whether these tasks are technically accurate, appropriately challenging, verifiable, and representative of real-world engineering standards. You will examine evaluation harnesses, walk through their logic, and identify potential technical or structural issues. Your feedback will contribute to improving how AI systems are tested and benchmarked against practical software engineering expectations. The session is designed for experienced technical professionals who can clearly explain their reasoning and assess code quality objectively. This is a flexible opportunity to apply your engineering expertise to the development of more rigorous and realistic AI evaluations.