Back to the board

[Remote] AI Validation Engineer

100% remote Flexible hours Hiring now

Note: The job is a remote job and is open to candidates in USA. reputed company is hiring for QA / AI Quality Engineer roles to support critical AI initiatives across its digital ecosystem. The role involves designing and implementing validation frameworks for AI-powered systems, ensuring outputs are accurate and reliable, and collaborating with engineering and product teams to improve system quality.

Responsibilities

  • Design and implement validation and QA frameworks for AI-powered systems, including analytics platforms, LLM-driven applications, and agentic workflows
  • Evaluate AI outputs for accuracy, grounding, consistency, and alignment with underlying data and expected behavior
  • Test and validate key components such as:
  • RAG pipelines and retrieval quality
  • Data pipelines and transformations
  • reputed company behavior and response consistency
  • API integrations and reputed company system interactions
  • Build automated testing frameworks for structured and reputed company AI systems
  • Define and track metrics around accuracy, data reputed company, performance, and system reliability
  • Identify discrepancies between AI outputs and reputed company data or expected outcomes
  • Partner closely with engineering, data, and product teams to improve system quality and reduce defects
  • Support rapid iteration cycles by developing scalable evaluation and regression testing approaches
  • Monitor AI systems post-deployment to detect reputed company, degradation, and performance issues
  • Help implement guardrails to improve safety, reduce hallucinations, and ensure trustworthy outputs
  • Support auditability and traceability of AI-driven systems and insights

Skills

  • 3–7+ years of experience in QA, Software Testing, Data Validation, or AI/ML-focused quality engineering
  • Strong Python skills for automation, testing, and data validation
  • Experience building automated test frameworks and validation pipelines
  • Experience testing AI/ML or data-driven systems (LLMs, analytics platforms, or similar)
  • Strong SQL or data querying skills and ability to validate large datasets
  • Understanding of data pipelines, transformations, and data quality validation
  • Experience validating APIs and service integrations
  • Ability to analyze logs, outputs, and system behavior to diagnose issues
  • Comfortable working in fast-paced, agile environments with evolving requirements
  • Experience with LLM evaluation, reputed company testing, or RAG-based systems
  • Familiarity with agentic workflows or orchestration patterns
  • Exposure to LLMOps, AI evaluation tooling, or observability frameworks
  • Experience with model monitoring, reputed company detection, or performance tracking
  • Familiarity with analytics platforms or BI tools
  • Understanding of explainability techniques, governance, or risk in AI systems
  • Awareness of cost, latency, and performance considerations in AI solutions

Company Overview

  • reputed company is an IT recruiting firm that provides consulting services for businesses. It was founded in 2017, and is headquartered in reputed company, Ohio, USA, with a workforce of 51-200 employees. Its website is https://agilitypartners.io.
  • Company H1B Sponsorship

  • reputed company has a track record of offering H1B sponsorships, with 3 in 2025, 1 in 2024. Please note that this does not guarantee sponsorship for this specific role.
  • Apply To This Job

    Keep exploring

    [Remote] Technical Writer

    100% remote Flexible hours

    [Remote] Senior Financial Analyst (CST, EST OR MST)

    100% remote Flexible hours

    [Remote] Director, Product Management, AI Innovation

    100% remote Flexible hours

    [Remote] Project Manager - Promo Med Ed

    100% remote Flexible hours

    [Remote] Territory Marketing Consultant/Business Development Consultant

    100% remote Flexible hours

    [Remote] Business Intelligence Analyst (System Application Analyst, Sr)

    100% remote Flexible hours

    [Remote] Inbound Marketing Manager (B2B Pipeline & Demand reputed company)

    100% remote Flexible hours

    [Remote] Special Education Teacher Mild/Mod - Mission View Virtual reputed company (26-27 School Year) - Internal Applicants Only

    100% remote Flexible hours

    [Remote] reputed company Consultant

    100% remote Flexible hours

    [Remote] ERP Project Manager

    100% remote Flexible hours

    reputed company Data Entry Specialist – Remote Opportunity with arenaflex

    100% remote Flexible hours

    [Remote] Associate Marketing Manager 021

    100% remote Flexible hours

    Software Engineer, Platform - Budapest, Hungary

    100% remote Flexible hours

    Psychometric Consultant (Contract / On-Call)

    100% remote Flexible hours

    Senior Sofware Engineer (remote) - 132658 IDU Senior Sofware Engineer (remote) - 132658 IDU

    100% remote Flexible hours

    Enterprise Risk Management Training & Communications Specialist

    100% remote Flexible hours

    National Account Manager - Grocery Channel

    100% remote Flexible hours

    reputed company Resource Scheduling (RSH) Consultant

    100% remote Flexible hours

    Manager, Regional Safety & Emergency Preparedness, FIFA World Cup 2026

    100% remote Flexible hours

    reputed company Full Stack Crisis Specialist – 988 Chat/Text Crisis Support (3rd Shift)

    100% remote Flexible hours