Required Al Builder - Performance
About the role (overview)
As part of our Business Technologies & AI organization, youll build and ship AI agents and workflows that drive measurable impact on our internal core business processesand then own their performance over time.
Using our AI Gateway, Custom GPTs, and low-code/no-code tooling, youll orchestrate prebuilt actions, design robust prompts, and wire data flows across systems like Salesforce, Snowflake, Gong, ERP, and internal toolswithout writing low-level API code.
You will also treat AI agents like a real workforce: defining how they are onboarded, measured, evaluated, and improved. Youll build the operating model, metrics, and evaluation solutions that keep our AI ecosystem powerful, safe, and cost-effective.
1) AI Solution Design & Development
Rapidly build, test, and deploy AI agents using the AI Gateway, Custom GPTs, and low/no-code tools.
Design scalable, secure, and reliable workflows with strong prompt patterns, evals, and canary tests.
Define new actions with Platform/Integrations when needed; ensure every agent has clear I/O, purpose, and a KPI.
2) Orchestration, Data, & Operations
Configure and chain actions; handle mapping, preprocessing, errors, retries, and safe context assembly.
Use managed auth/secrets with least-privilege access and data minimization.
Write SQL for retrieval context, routing rules, golden sets, and performance queries.
Integrate logs, BI, and admin tools into a single operational view.
3) Performance, Quality & Evaluation
Define and track core metrics: usage, adoption, accuracy, cost, time saved, revenue impact.
Build dashboards, scorecards, leaderboards, and automated alerts (health, anomalies, regressions).
Maintain evaluation frameworks and pipelines (golden sets, scenarios, automated checks).
Requirements: 2+ years in roles such as, Business Application Developer, Business Analyst, Product Operations, RevOps, AI/ML Ops, or Digital Transformation in a global SaaS / data-heavy company.
Proven experience designing and delivering AI-driven workflows or agents in a SaaS environment, partnering closely with business stakeholders.
Low-code/no-code build skills and strong prompt engineering fundamentals (tool/function calling, structured/JSON outputs, context windows, grounding).
Hands-on experience with AI Gatewaystyle action catalogs (configure inputs/outputs, field mapping, chaining, retries, error handling), and familiarity with platforms like Salesforce, Snowflake, Gong, ERP, ticketing, HRIS, etc.
Solid SQL skills for analysis, shaping data for agents, defining evaluation sets, and measuring workflow performance (no need to be a full data engineer).
Familiarity with Context Engineering using APIs, MCPs, retrieval, and data contracts.
Understanding of hallucination-mitigation strategies (retrieval grounding, instruction hierarchy, eval sets, guardrails, refusals).
Systems thinker who can map complex ecosystems (agents, tools, data sources, owners) and make them understandable.
High data literacy: comfortable with usage data exploration, defining metrics, and building dashboards with Data/BI teams.
Strong stakeholder management: able to work with executives on value, Security/IT on controls, Data/Engineering on telemetry/models, and business teams on workflows.
Outcome-driven: bias to measurable impact (time saved, error reduction, revenue influence) over vanity metrics.
Clear, concise communication and documentation skills.
Nice to have
Experience launching organization-wide AI/automation programs or agent governance.
Background in observability/telemetry for LLM systems (cost tracking, evals, quality signals).
Prior work with risk/compliance programs (data classification, RBAC, policy reviews).
Built enablement programs (docs, workshops, champions communities) that changed behavior.
Familiarity with offline/online experimentation, prompt/agent evaluation, and KPI design for AI workflows.
This position is open to all candidates.