Senior Specialist - AI Model Validation

## Your Opportunity At Schwab, innovative thinking meets practical problem solving. As part of Model Risk Oversight, you will work at the intersection of AI /ML engineering, quantitative analysis, and responsible model governance. You will help evaluate AI and machine learning systems before and after they reach production, with a particular focus on generative AI and applications that use large language models. The AI Model Validation Team designed this role for an early-career engineer or quantitative researcher who wants hands-on exposure to real AI systems, rigorous experimentation, and high-impact technical communication. You will write code, design tests, analyze model behavior, investigate failure modes, and present evidence-based conclusions to technical teams and senior leaders. ## What you will do - Validate AI and machine learning models by independently assessing intended use, design choices, data, implementation, controls, performance, and limitations. - Build reproducible testing frameworks, analyses, and prototypes to evaluate model quality, robustness, safety, and reliability. - Evaluate generative AI and retrieval-augmented generation applications, including prompt behavior, retrieval quality, groundedness, hallucination risk, response consistency, and guardrail effectiveness. - The candidate will apply statistical and machine learning methods to benchmark models, analyze errors, compare alternatives, and assess results. - Review source code, notebooks, technical documentation, model inventories, performance monitoring, and production-control evidence. - Partner with model developers, product teams, data scientists, engineers, risk partners, and external consultants to understand systems and challenge assumptions constructively. - Document findings in clear validation reports, technical appendices, issue statements, and executive-ready presentations. - Evaluate ongoing performance monitoring and participate in periodic reviews of models after implementation. - Contribute to reusable validation frameworks, testing utilities, and effective-challenge methods as AI capabilities and industry practices evolve. ## What you have ### Required Qualifications - Master's degree, or bachelor's degree with strong relevant project or research experience, in computer science, data science, statistics, applied mathematics, engineering, economics, quantitative finance, or a related field. - 0-2 years of professional experience. Relevant internships, research, capstone projects, open-source contributions, or substantial independent projects are welcome. - Proficiency in Python and experience using common data science or machine learning libraries. - Working knowledge of core machine learning concepts, experimental design, model evaluation, and statistical reasoning. - Ability to read technical documentation and code, investigate unfamiliar systems, and communicate conclusions clearly. - Curiosity, sound judgment, attention to detail, and a willingness to challenge assumptions with evidence. ### Preferred Qualifications - Hands-on experience with large language models, generative AI, natural language processing, embeddings, vector search, or retrieval-augmented generation. - Experience with evaluation frameworks, prompt testing, red teaming, model monitoring, or responsible AI techniques. - Familiarity with SQL, Git, containers, cloud platforms, or modern ML development workflows. - Understanding software testing, data quality, reproducibility, or model deployment concepts. - Experience presenting technical work through research papers, project reports, demos, or presentations. - Interest in financial services, model risk management, AI governance, or building trustworthy AI systems.