We are looking for a Senior Big Data Engineer to join our growing ML Platform team. This is a great opportunity for an experienced Data Engineer who wants to apply their skills to the world of Machine Learning. Together, well provide the high-throughput tools and distributed systems needed to process billions of data points, ensuring that our models have the feature-rich, reliable data they need to perform in real-time.
ML is central to our work. It enables us to process billions of dollars worth of e-commerce transactions and identify fraud rings. Precision at scale is crucial-our models depend on engineered features derived from massive datasets. As the volume of data and the complexity of these models grow, we need senior engineers who can solve the "Big Data" challenges inherent in modern MLOps.
Why should you join us?
Youll be part of a highly proficient engineering team that is the focal point for all ML engineering activity. This role is perfect for a Big Data specialist who wants to own the data-intensive side of ML-building the pipelines and infrastructure that transform raw global traffic into actionable intelligence.
What you will be doing:
Designing and building the data-heavy components of our ML infrastructure, allowing our models to make billions of real-time decisions every year.
Building distributed data processing pipelines that power model training, feature engineering, and offline research at scale.
Optimizing the "Feature Store" concept-ensuring that the data used for training matches the data used in production for real-time serving.
Acting as a technical consultant to data scientists and researchers, helping them navigate massive datasets and providing the Spark-based tooling they need to iterate faster.
Expanding our ML data infrastructure to be scalable and efficient, focusing on the performance of data ingestion and transformation for diverse model types.
Improving MLOps standards by ensuring data reproducibility, lineage, and quality across the entire model lifecycle.
Requirements: 5+ years of experience with large-scale data processing, with deep expertise in Apache Spark.
5+ years developing complex software with at least one general-purpose language (preferably Python or Scala).
Backend and server-side development experience of complex, highly scalable systems.
Strong understanding of Data Engineering patterns (Partitioning, Sharding, Schema evolution) and how they apply to model performance.
Interest in Machine Learning concepts-you don't need to be a researcher, but you should understand how data flows into a model.
Experience working with public clouds (AWS / GCP / Azure).
Fluent in written and spoken English.
Itd be really cool if you also:
Are familiar with Databricks or Airflow for orchestrating complex ML workflows.
Are comfortable in a containerized environment (Kubernetes/Docker).
Have experience with low-latency, real-time data serving or streaming (Kafka).
This position is open to all candidates.