Required Data Infrastructure Engineer
What Youll Do:
Design, implement, and enhance robust and scalable infrastructure that enables efficient deployment, monitoring, and management of machine learning models in production. In this role, you will bridge the gap between research and production environments, streamline data and feature pipelines, optimize model serving, and ensure governance and reproducibility across our ML lifecycle.
Responsibilities:
Decouple data prep from model training to accelerate experimentation and deployment
Build efficient data workflows with versioning, lineage, and optimized resource use (e.g., Snowflake, Dask, Airflow)
Develop reproducible training pipelines with MLflow, supporting GPU and distributed training
Automate and standardize model deployment with pre-deployment testing (E2E, dark mode)
Maintain a model repository with traceability, governance, and consistent metadata
Monitor model performance, detect drift, and trigger alerts across the ML lifecycle
Enable model comparison with A/B testing and continuous validation
Support infrastructure for deploying LLMs, embeddings, and advanced ML use cases
Manage a unified feature store with history, drift detection, and centralized feature/label tracking
Establish a single source of truth for features across research and production across research and production.
Requirements: 3+ years of experience as an MLOps, ML Infrastructure, or Software Engineer in ML-driven environments, preferably with PyTorch.
Strong proficiency in Python, SQL (leveraging platforms like Snowflake and RDS), and distributed computing frameworks (e.g., Dask, Spark) for processing large-scale data in formats like Parquet.
Hands-on experience with feature stores, key-value stores like Redis, MLflow (or similar tools), Kubernetes, Docker, cloud infrastructure (AWS, specifically S3 and EC2), and orchestration tools (Airflow).
Proven ability to build and maintain scalable and version-controlled data pipelines, including real-time streaming with tools like Kafka.
Experience in designing and deploying robust ML serving infrastructures with CI/CD automation.
Familiarity with monitoring tools and practices for ML systems, including drift detection and model performance evaluation.
Nice to Have
Experience with GPU optimization frameworks and distributed training.
Familiarity with advanced ML deployments, including NLP and embedding models.
Knowledge of data versioning tools (e.g., DVC) and infrastructure-as-code practices.
Prior experience implementing structured A/B testing or dark mode deployments for ML models.
This position is open to all candidates.