- The Sr
- Responsible AI Analyst Responsible AI Analyst will be responsible for supporting the design development evaluation and implementation of Responsible AI solutions across model lifecycle stages
- The role involves conducting technical assessments building evaluation datasets defining metrics supporting model testing contributing to guardrail design assisting in deploying Responsible AI workflows and collaborating with engineering product governance ISG and DPO teams to ensure that AI systems adhere to ethical safety fairness transparency and regulatory expectations
- Execute Responsible AI Evaluations
- Conduct structured assessments on fairness safety robustness explainability and hallucination risks for models under development or deployment
- Design Evaluation Datasets and Metrics
- Create curated adversarial and edge case datasets and define quantitative metrics for evaluating fairness toxicity reliability and ethical alignment
- Support Guardrail and Control Implementation
- Contribute to implementing technical guardrails safety filters explainability modules and automated checks within AI pipelines
- Analyze Ethical and Technical Risks
- Review datasets model outputs and system behavior to identify fairness gaps robustness issues transparency deficiencies and other Responsible AI risks
- Participate in Red Teaming and Stress Testing
- Assist with scenario based adversarial evaluations prompt safety checks robustness tests and model vulnerability analysis
- Support Deployment of Responsible AI Workflows
- Assist in implementing lifecycle governance workflows templates and processes such as via IBM OpenPages or equivalent governance tooling
- Prepare Transparency and Governance Documentation
- Develop model cards system cards evaluation reports risk logs and supporting documentation required for governance reviews and audit readiness
- Assist in Continuous Monitoring
- Support creation of dashboards metrics and monitoring signals to track fairness drift hallucination patterns model instability and safety deviations
- Collaborate Across Engineering and Governance Functions
- Work closely with AI engineers data scientists product teams legal ISG DPO and governance bodies to ensure Responsible AI requirements are consistently applied
- Assist in Training and Knowledge Enablement
- Help develop training content guides and resources to educate internal teams on Responsible AI evaluation methods guardrails and governance expectations
- Proficiency in Python and ML DL frameworks PyTorch TensorFlow for evaluation and experimentation
- Understanding of fairness libraries Fairlearn AIF360 and ability to compute ethics related evaluation metrics
- Familiarity with explainability tools SHAP LIME Captum Integrated Gradients
- Exposure to red teaming concepts prompt safety evaluation and data integrity checks
- Experience with MLOps basics including evaluation pipelines experiment tracking and CI workflows
- Understanding of ML algorithms generative models and supervised unsupervised learning techniques
- Familiarity with NLP vision speech and structured data domains
- Knowledge of datasets benchmark suites and third party model ecosystems
Technology->AI-Generative AI->Generative AI - Basic->retrieval augmented generation (rag),Technology->AI-Responsible AI->Responsible AI
