דרושים » דאטה » data Engineer

Required Senior Data Engineer
Description
Join our core platform engineering team, developing our AI-powered automotive data management platform.
We are developing the next generation data-driven products for the Automotive industry, focusing on cybersecurity (XDR) and vehicle quality. Our products monitor and secure millions of vehicles worldwide and help automakers leverage connected vehicle data to deliver cyber resilience, safety, customer satisfaction and increase brand loyalty.
Our Data Engineering & Data Science Group leads the development of our Iceberg-based data platform, including data lake, query engine, and ML-Ops tools, serving as a solid AI-ready foundation for all our products.
At the core of our Engineering Team, you will build and operate scalable, production-grade customer-facing data and ML platform components, focusing on reliability and performance.
Technological background and focus: Iceberg, Trino, Prefect, GitHub Actions, Kubernetes, JupyterHub, MLflow, dbt
This role is full-time and is Herzliya, Israel based.
Responsibilities
Design, build, and maintain scalable data pipelines to ingest and transform batch data on our data lake, enabling analytics and ML with strong data quality, governance, observability, and CI/CD.
Build and expand our foundational data infrastructure, including our data lake, analytics engine, and batch processing frameworks.
Create robust infrastructure to enable automated pipelines that will ingest and process data into our analytical platforms, leveraging open-source, cloud-agnostic frameworks and toolsets.
Develop and maintain our data lake layouts and architectures for efficient data access and advanced analytics.
Build our ML platform and automate the ML lifecycle.
Drive business-wide ML projects from an engineering perspective.
Develop and manage orchestration tools, governance tools, data discovery tools, and more.
Work with other team members of the engineering group, including data architects, data analysts, and data scientists, to provide solutions using a use case-based approach that drives the construction of technical data flows.

We are seeking an experienced Data Engineer with a strong background in BI and Data Engineering, excellent problem-solving skills, and the ability to work independently and learn quickly.
Role Overview
You will join a dynamic team responsible for designing and implementing end-to-end data solutions on the Microsoft Fabric platform. This role involves:
Building and maintaining robust data processes.
Driving adoption of new technology within the organization.
Ensuring scalability, performance, and reliability of data solutions.

Responsibilities:
Collaborate with leaders, stakeholders, and product managers to understand data requirements and deliver effective BI solutions.
Design, develop and maintain robust data pipelines for efficient data ingestion, processing, and storage.
Optimize data storage and retrieval processes for optimal performance.
Monitor data pipelines and systems consistently to identify performance issues, ensuring timely maintenance and troubleshooting.
Create and maintain documentation of data processes and systems.

As a Data Engineer , you will own the architecture and optimization of large-scale ETL processes that transform raw heavy-duty vehicle telemetry into production-grade intelligence. You will operate at the intersection of Big Data and AI, building scalable pipelines, enforcing data quality standards, and managing cost-efficiency for a system processing billions of time-series records. You will be a technical owner, collaborating directly with Data Scientists to ensure our fleet intelligence models run reliably in production.
What Youll Do
Architect and build robust ETLs and scalable data pipelines on Databricks and AWS.
Optimize high-throughput ingestion workflows for billions of time-series records, ensuring low latency and data integrity.
Engineer data validation frameworks and automated monitoring to proactively detect anomalies before they impact models.
Drive cost-efficiency by tuning Spark jobs and managing compute resources in a high-volume environment.
Transform raw IoT/telemetry signals into structured, enriched Feature Stores ready for Machine Learning production.
Define best practices for data engineering, CI/CD for data, and lakehouse architecture across the organization.