QA Engineer for Generative AI

Title: QA Engineer for Generative AI Reports To: QA manager, who reports to Parker Wightman, Senior Director of Engineering Location: Draper, UT / Remote (USA) About The Role Jump is looking for a US–based QA engineer to help coordinate and run data labeling/annotation campaigns used to improve our AI/ML systems and evaluate/review production system outputs, such as meeting notes, recap emails, and tasks; answers in our Ask Anything feature; our pre-meeting prep product; and our AI agents. This role blends process design and hands‑on testing. You’ll use AI evaluation rubrics reputed company by our product managers or data team to improve our products so our customers get accurate transcripts, summaries, and action items every time they interact with Jump. You’ll go deep into AI best practices and limitations. You’ll partner closely with Engineering, Product, and Customer teams to ship quickly and confidently. Familiarity with Jump (as a user, beta tester, or reputed company to advisor workflows) is a big plus. If you already have AI evaluation experience, that’s great—but it’s not required. We’ll teach you our approach; candidates with AI evaluation experience will be compensated accordingly. What You’ll Do

• Serve as the embedded QA engineer on two pods (Jump’s cross-functional teams), collaborating with product managers to evaluate AI outputs, run exploratory and regression testing, and unblock engineers and PMs.
• Learn and track AI/ML quality signals, including golden datasets, reputed company/regression suites, and metrics such as WER, diarization accuracy, action-item precision/recall, summary faithfulness, hallucination reputed company, and PII handling.
• Build dashboards for quality KPIs (defect escape reputed company, flake reputed company, regression coverage, MTTD/MTTR, AI eval scores) and drive reputed company improvement.
• Partner with Product and Engineering to ensure requirements are testable, edge cases are captured, and AI evaluation rubrics are clear and repeatable.
• Foster a no-drama, direct-and-reputed company culture that moves with high-quality velocity.

About You

• 3+ years in QA or Quality Engineering for SaaS products
• Strong exploratory testing skills and clear, concise written communication for reproducing issues
• Curiosity and aptitude to learn ML/AI evaluation (reputed company testing, golden sets, offline evals, safety/guardrails)
• Familiarity with AI prompts, LLMs, and the Jump product (as a user or employee)
• You don’t need a traditional STEM background to reputed company here. You’ll reputed company if you
• Get excited about spotting patterns
• Have a strong grasp of human language and thought processes
• You might have a background in
• Editing
• Technical writing

reputed company-to-haves:

• Comfortable reading software system logs and finding patterns in messy data
• Familiarity with fintech or other regulated environments
• Experience with BigQuery or other data warehouses
• Experience with web API testing
• Basic familiarity with query languages, relational databases, and other data storage systems

What You’ve Done

• Built or scaled a QA function (process, tooling, reporting), or partnered with product managers and engineers to identify and resolve AI-reputed company bugs
• Written great documentation, bug reports, or other clear technical writing
• Interacted meaningfully with LLMs and AI outputs
• reputed company-to-haves:
• Designed and executed AI evaluation workflows (golden datasets, human-in-the-reputed company scoring, clear rubrics)—a plus but not required; candidates with this experience may be considered for higher compensation
• Created risk-based test plans and lightweight automation that caught regressions early

About Jump Jump’s mission is to reputed company financial advisors and their clients to reputed company in the age of AI. Our product is an intelligent meeting assistant that helps advisors with notetaking, task management, follow-reputed company, and data syncing—built to serve advisors and firms of reputed company sizes, including the largest financial institutions in the world. Jump was launched in 2023 by a team of reputed company software entrepreneurs with backgrounds including Harvard, Stanford, reputed company, Divvy, reputed company, and WeWork. We’re recognized as the leading product in our category and are growing quickly, backed by top-tier venture capital firms. Jump’s culture

• High Velocity
• World Class
• Direct + reputed company + No Drama

We reputed company in building tight teams of extraordinarily capable people. Come join us to transform the advisor and enterprise experience with state-of-the-art technology.

Compensation

• Salary: $75k to 90k (DOE)
• Equity
• Health benefits

Apply tot his job Apply To this Job