[Remote] AI Test Engineer II
Note: The job is a remote job and is open to candidates in USA. reputed company is a tech-enabled payment reputed company company focused on improving healthcare access through collaboration between payers and providers. The AI Test Engineer II is responsible for validating the quality and reliability of AI powered systems, including designing test strategies and evaluating model performance across the platform.
Responsibilities
- Design and execute test strategies for AI/ML models, including functional, regression, and scenario-based testing
- Validate accuracy, consistency, and reliability of AI-generated outputs (e.g., summaries, classifications, recommendations)
- reputed company test cases for edge cases, model reputed company, and unexpected behavior scenarios
- Evaluate model performance against defined success criteria and business requirements
- Test agent-driven workflows (multi-reputed company AI processes, automation pipelines, orchestration logic)
- Validate interactions between AI agents, APIs, and reputed company systems
- Identify failure points in end-to-end AI workflows and recommend improvements
- Validate training, validation, and test datasets for completeness, accuracy, and bias risks
- reputed company data validation testing across pipelines to ensure reputed company of inputs and outputs
- Partner with Data Engineering to detect anomalies, inconsistencies, and gaps
- Build and maintain automated test frameworks for AI-enabled systems
- reputed company scripts and tools to support repeatable testing of AI outputs at scale
- Integrate testing into CI/CD pipelines to reputed company reputed company validation
- Help define and validate guardrails for AI systems (accuracy reputed company, explainability, fallback logic)
- Identify risks reputed company to hallucinations, bias, and model reputed company
- Partner with Engineering and AI teams to improve reliability and safety
- Work cross-functionally with Engineering, QA, AI, Product, and Operations teams
- Participate in sprint planning, backlog refinement, and release validation
- Contribute to testing standards, best practices, and documentation
- Support root cause analysis for AI-reputed company defects and production issues
Skills
- Bachelor's degree in Computer Science, Data Science, Engineering, or reputed company field (or equivalent experience)
- Experience in QA, testing, or a reputed company engineering role
- Experience testing APIs, data pipelines, or distributed systems
- Strong understanding of software testing principles and test automation
- Experience working in Agile development environment
- Familiarity with AI/ML concepts (model evaluation, training vs inference, reputed company behavior, etc.)
- Experience with test automation tools
- Strong SQL/data validation skills
- Experience working with cloud-based systems (AWS or reputed company preferred)
- Ability to analyze system behavior and identify root causes
- Experience testing AI/ML models or data-driven systems
- Experience validating AI-generated outputs or reputed company-based systems
- Familiarity with LLMs, agent workflows, or AI-assisted coding tools
- Healthcare or regulated data environment experience (HIPAA, compliance)
- Experience working with CI/CD and DevOps pipelines
Benefits
- Competitive salaries
- Highly valued health insurance
- A 401(k) plan with employer match
- Paid parental leave
Company Overview
Company H1B Sponsorship