דרושים » דאטה » data Engineering Lead

you will be part of our company rem department. our road experience management (rem) is an end-to-end mapping and localization engine. this process leverages advanced algorithms, massive parallelization, and Big Data technologies, creating a highly complex system that demands both a deep understanding of the map creation workflow and strong technical expertise. were looking for a senior data engineer to lead the architecture and development of large-scale, production-grade data pipelines supporting ml inference systems. what will your job look like:
architect and own end-to-end data pipelines for large-scale model inference
design high-throughput, scalable data streaming to the cloud
integrate data conversion into data collection and inference pipelines
drive performance, scalability, and reliability across distributed systems
partner with ml, platform, and infrastructure teams

We are looking for a DataOps Engineer to own the infrastructure that powers our large-scale data processing platform. This is a platform-facing role sitting at the intersection of data engineering and infrastructure - you'll be the person who makes Spark run reliably and efficiently on Kubernetes, so that data engineers can build with confidence.
You understand data workloads deeply enough to make smart infrastructure decisions, and you have the production instincts to keep complex systems healthy at scale. If you get excited about shaving minutes off Spark job runtimes, right-sizing cluster autoscalers, and building the internal tooling that makes a data platform feel effortless, this role is for you.
RESPONSIBILITIES:
Design, deploy, and operate the Kubernetes-based infrastructure that runs Apache Spark and large-scale data processing workloads
Own the reliability, performance, and cost-efficiency of the data platform - including SLAs, autoscaling, resource quotas, and workload isolation
Manage Spark-on-K8s configurations, Airflow infrastructure, and Databricks integration; tune for throughput, latency, and cost
Build and maintain CI/CD pipelines and infrastructure-as-code for data platform components
Develop observability tooling - metrics, logging, alerting, and data quality dashboards - to proactively surface issues across the pipeline stack
Collaborate closely with Data Engineers to understand workload patterns and translate them into infrastructure decisions
Manage cloud storage (GCS/S3), Delta Lake, and Unity Catalog infrastructure
Drive platform improvements end-to-end: from design through deployment and ongoing ownership.