As an Adversarial AI Evaluation Scientist, you will review and validate adversarial agent tasks: indirect prompt-injection (IPI), data-exfiltration, tool-misuse, and related attack scenarios to ensure each one is realistic, fair, deterministic, reproducible, and correctly graded. You will decide whether a scenario genuinely demonstrates a model vulnerability or merely an artifact of how the task was built, and you will combine adversarial-security judgment with statistical analysis and LLM-assisted tooling to find quality issues, analyze attack-success patterns, and recommend or automate improvements.
Responsibilities
Review adversarial tasks for realism and threat-model soundness: is the attackers capability plausible, can the attacker actually reach the poisoned content, and is the target system a realistic one?
Assess attack fairness and attribution: confirm the harm is driven by the adversary (not pre-baked into the system prompt, vouched for by a trusted layer, or otherwise rigged), so a careful agent could plausibly pass.
Validate grading correctness: verify that success/failure criteria, reward logic, and deterministic checks detect the intended harm, with no reward hacks, token-collapse, or evadable gates.
Measure and interpret attack success rate (ASR@K) and its variance; verify deterministic, reproducible behavior across repeated executions.
Judge attack quality - novelty vs. known patterns, sophistication, stealth, and coverage across harm types and target surfaces.
Identify anomalies and systemic issues across large sets of tasks (e.g. scenarios that succeed ~100% of the time against every model - a red flag for an unfair task, not a win).
Define validation guidelines, acceptance criteria, quality thresholds, and release gates for adversarial tasks.
Use LLMs and AI models to analyze attack failures, identify root causes, and assist with task or grader fixes.
Build automated validation, regression, and quality-monitoring pipelines for adversarial task sets.
Work closely with red-teamers, researchers, and task authors to improve attack design, fairness, and reliability.
Requirements: Must Have
4+ years in an analytical, ML, security-research, or applied-research role, including hands-on work with adversarial ML, LLM security, red-teaming, or AI safety evaluation.
Bachelors or Masters in Computer Science, Machine Learning, Statistics, a related quantitative field, or equivalent practical experience.
Working knowledge of prompt injection / indirect prompt injection, jailbreaks, data exfiltration, or tool-misuse attacks against LLM-based systems or agents.
Strong Python and data-analysis skills.
Solid grounding in statistics and experiment design: distributions, variance analysis, hypothesis testing, and anomaly detection: applied to evaluation and attack-success measurement.
Experience building or auditing data-driven validation, evaluation, or quality frameworks.
Experience using LLMs for analysis, evaluation, code generation, or automated remediation.
Strong understanding of reproducibility, deterministic systems, and automated testing.
Sharp adversarial thinking and the ability to spot subtle ways a test can be unfair, gameable, or mis-graded.
Excellent analytical, problem-solving, and communication skills.
Preferred Skills That Set You Apart
Certifications in offensive cybersecurity (e.g., OSWA, OSWE, OSCE3, SEC542, SEC522)
Experience designing or evaluating AI agents, LLM-powered applications, or agentic workflows (e.g. email/doc assistants, tool-using agents).
Familiarity with RL Gym-style task formats, Gymnasium/OpenAI Gym, or agent-evaluation harnesses.
Experience red-teaming production AI systems or building adversarial benchmark datasets.
Experience with automated evaluation pipelines, regression testing, and quality gates.
Familiarity with safety/security taxonomies (harm types, attack patterns) and threat modeling.
This position is open to all candidates.