Senior Applied AI Scientist : Workflow Agents & Prompt Ownership | Maternity Leave Replacement | 50% Position | Ramat Gan
A leading high-tech company is developing Workflow Agents, an AI platform for building agents with data from calls, emails, and deals. These agents run multi-step workflows, making prompt design and evaluation essential to quality.
We are seeking a Senior AI Practitioner to join as the team's data Scientist, owning LLM prompts, evaluation, and quality while shaping product capabilities.
What You'll Do:
Own LLM prompts, evaluations, and quality.
Partner with Product and Engineering to define capabilities and measure agent performance offline and in production.
Guide engineers in prompt and skill ownership, including prompt writing, evals, and trace analysis.
Establish best practices and tools for experimentation, tracing, and evaluation.
Requirements: What you bring
Hands-on experience developing LLM applications and agent workflows using agent frameworks.
Strong experience designing evals for LLM systems, including LLM-as-judge and regression TEST sets.
Strong Python skills and practical experience building evaluations, analyzing traces, and improving prompts based on evidence.
Experience with tracing and observability tools for LLM apps, such as LangSmith.
Ability to work independently, make practical tradeoffs, and translate product needs into technical recommendations.
Nice to Have
Experience mentoring or managing engineers or data scientists, or leading technical work across teams.
Ability to influence product and engineering decisions without formal authority.
The Details
Contract for 6-12 months, about 50% time. Freelancers are welcome.
We are also hiring for a Senior data Scientist / ML Engineer role.
This position is open to all candidates.