Responsibilities

• Participate in the entire software development lifecycle, encompassing all stages from requirements analysis to test planning, execution, defect tracking, through to product release and maintenance.
• Go to person in relation to A.I Agents evaluation and continuously monitoring.
• Create comprehensive and effective test strategies and hands-on testing to ensure the accuracy, reliability, and performance of AI and data applications.
• Root cause analysis of test failures and product issues in an effective manner, and drive optimization for future enhancements.
• Design and develop internal tools leveraging AI technology to improve engineering and testing work efficiency.

Requirements

• Bachelors or Masters degree in Computer Science, Artificial Intelligence, Data Science, or a related field.
• Strong understanding of Large Language Models (LLMs), autonomous AI agents, and their system architectures.
• Experience with AI evaluation methodologies, including offline benchmarking, online monitoring, and hybrid human-AI evaluation approaches.
• Familiarity with software engineering best practices such as Test-Driven Development (TDD), Behavior-Driven Development (BDD), and their limitations in AI contexts.
• Proficiency in designing adaptive, lifecycle-spanning evaluation frameworks that incorporate both quantitative and qualitative metrics.
• Experience with evaluation tools and frameworks (e.g., Opik,LangSmith) is a plus.
• Ability to analyze complex system-level behaviors, including reasoning pipelines, tool integrations, and emergent agent actions.
• Strong analytical skills with experience in data-driven diagnostics and root cause analysis.
• Excellent communication skills to document evaluation plans, results, and recommendations clearly.
• Experience working in cross-functional teams and managing feedback loops between evaluation and development.
• Experience collaborating with infrastructure or platform teams to improve AI tooling and automation platforms.