The ERDM Sr. AI Operations Analyst is responsible for ensuring that Sales & Finance AI solutions operate with the highest levels of trust, accuracy, control, and compliance.
We are seeking a detail-obsessed, data driven expert to serve as the guardian of accuracy for our Sales & Finance AI chat assistants. You will design validation frameworks, partner with data engineering teams, and ensure every metric the AI surfaces - especially Bookings and Revenue - is accurate, consistent, and trustworthy. You will lead incident response on any data accuracy or hallucination even
The role also serves as the bridge between Finance users and technical development teams. This role is responsible for translating business requirements into AI-enabled solutions and ensuring that Finance AI capabilities align with business goals and leadership needs.
Requirements & Solution Definition
- Gather and document business requirements from Finance users and stakeholders through interviews, workshops, observation, and feedback analysis
- Write clear, detailed user stories with well-defined acceptance criteria that IT can act on without ambiguity
- Ensure requirements reflect metric integrity standards and executive-readiness expectations
AI Quality Assurance
- Develop and maintain AI-specific testing frameworks - define test scenarios, expected outputs, edge cases, and pass/fail criteria tailored to conversational AI behavior
- Establish and document quality standards for AI-generated responses - accuracy, completeness, formatting, tone, and contextual relevance
- Validate AI-generated responses against Snowflake source data using SQL queries to confirm metric accuracy and consistency
- Conduct thorough UAT, regression testing, and release testing for every new feature, enhancement, prompt update, data refresh, or model change before go-live
- Confirm metric integrity - verify that Bookings and Revenue figures surfaced by the AI match official Finance reporting
- Coordinate business sign-off activities - manage the approval workflow with Finance stakeholders before releases are promoted to production
Hallucination Prevention
- Design and execute validation test strategies specifically targeting hallucination risks - test for fabricated data, incorrect metric associations, misleading trends, and unsupported narratives
- Identify and catalog high-risk Finance use cases where hallucination potential is elevated (e.g., multi-step calculations, year-over-year comparisons, segment-level breakdowns, forecast vs. actuals)
- Monitor response quality on an ongoing basis - conduct periodic spot-checks and audits of AI outputs across different query types and user scenarios
- Escalate and resolve quality concerns immediately - flag suspected hallucinations to the Data Accuracy Specialist (IC4) and team lead; coordinate with IT/Engineering to implement fixes and guardrails
Executive Use Cases
- Support in designing and testing executive-focused use cases - ensure AI responses meet the precision, tone, and contextual depth expected by C-suite and SVP users
- Test and validate executive prompt frameworks - verify that curated queries, guided flows, and personalized dashboards produce accurate, polished outputs
- Contribute to improving usability and decision-support capabilities - identify opportunities to make the executive experience more intuitive, actionable, and insightful
Adoption & Change Management
- Develop and execute adoption strategies - create structured plans to drive Finance user engagement, from initial awareness through active daily use
- Lead user onboarding - design and deliver onboarding experiences for new users, including guided walkthroughs, sandbox environments, and office hours
- Deliver training programs - conduct live demos, workshops, and drop-in sessions tailored to different user groups (analysts, managers, directors)
- Manage communications - draft and distribute release notes, feature announcements, tips & tricks, known issue updates, and adoption progress reports to Finance stakeholders
- Collect and synthesize user feedback through surveys, interviews, and usage analytics to continuously improve the experience
- Bachelor's degree Required
- 7+ years in Business Analysis / Finance Analytics / FP&A Operations
- Experience with AI-powered tools, chatbots, or conversational AI interfaces
- Understanding of Finance metrics and reporting (Bookings, Revenue, or related areas)
- Working knowledge of SQL and Snowflake
- Understanding of AI/LLM fundamentals - how large language models retrieve, process, and present information; awareness of common failure modes (hallucination, data staleness, context window limitations)
- Prompt Engineering Certificate or related experience
- Familiarity with Abacus or similar platforms
- Hands-on experience with prompt engineering — designing, testing, and refining prompts to improve AI response quality and accuracy
- Experience creating training materials, user guides, or knowledge base content
- Experience with AI testing frameworks or quality assurance for data-driven applications
- Familiarity with AI governance or responsible AI principles
At NetApp, we embrace a hybrid working environment designed to strengthen connection, collaboration, and culture for all employees. This means that most roles will have some level of in-office and/or in-person expectations, which will be shared during the recruitment process.
Equal Opportunity Employer:
NetApp is firmly committed to Equal Employment Opportunity (EEO) and to compliance with all federal, state and local laws that prohibit employment discrimination based on age, race, color, gender, sexual orientation, gender identity, national origin, religion, disability or genetic information, pregnancy, protected veteran status, and any other protected classification.
Why You'll Thrive at NetApp
At NetApp, you won't wait for the perfect moment—you'll make it. The early planning, the extra thought, the bold idea that turns good into great: That's how our people operate and how we continue to push the boundaries of data infrastructure.
NetApp is the trusted partner for organizations transforming data into opportunity. As the only enterprise-grade storage service natively embedded in Google Cloud, AWS, and Microsoft Azure, we empower customers to run everything from traditional workloads to enterprise AI with unmatched performance, resilience, and security.
Our culture
We celebrate mold breakers, bold thinkers, and problem solvers. We reward initiative, impact, and ownership. We provide flexibility so you can balance professional ambition with your personal life. Here, differences are not just welcomed—they drive everything we do.
If you're ready to innovate, rise to the challenge, and own every moment - make your next move your best one. Apply now.
Submitting an Application
To ensure a streamlined and fair hiring process for all candidates, our team only reviews applications submitted through our company website. This practice allows us to track, assess, and respond to applicants efficiently. Emailing our employees, recruiters, or Human Resources personnel directly will not influence your application.
AI Disclosure
For select roles, some stages of our hiring process may use artificial intelligence tools to help evaluate applications and candidate selection. These tools support—rather than replace—human decision-making.
