דרושים » תוכנה » Senior data Engineer

we are a dynamic technology firm dedicated to building exceptional products using modern technologies for the world's most admired companies. We pride ourselves on innovation, collaboration, and delivering excellence in every project.

Job Description
Responsibilities:

Design and maintain scalable data pipelines with Spark Structured Streaming.
Implement Lakehouse architecture with Apache Iceberg.
Develop ETL processes for data transformation.
Ensure data integrity, quality, and governance.
Collaborate with stakeholders and IT teams for seamless solution integration.
Optimize data processing workflows and performance.

We're looking for an experienced Software Engineer to join our Data Platform Group. In this key role, you will build the company data platform: cloud-based microservices and data pipelines that process on the order of 1M records/sec at low latency. Because the platform is the foundation other groups build on, your work has a direct impact on our customers and enables engineering, product, and research teams across the organization.
The Lakehouse team owns the data itself - how it is stored, organized, retained, and served. We own our analytical storage layer, the batch processing built on top of it, and the APIs through which customers and the rest of the company consume data. If you enjoy the problems that only appear at petabyte scale - physical data layout, query performance, storage cost, and retention - this is the role.

Responsibilities:
End-to-end ownership of our large-scale analytical storage layer: data modeling, schema and table design, partitioning, retention, and query performance.
Design and develop the batch processing layer over our data lake using Spark on EMR (Java and PySpark).
Build and evolve Java/Spring Boot services that expose our data through well-defined APIs to customers and to consumers across the company.
Own performance and cost: query optimization, file layout and compaction, cluster sizing, and storage efficiency at scale.
Research new technologies in the lakehouse and analytical-storage space and adapt them for use in our product.
Work closely with product, DevOps, and security teams.

We are seeking an experienced Solutions Data Engineer who possess both technical depth and strong interpersonal skills to partner with internal and external teams to develop scalable, flexible, and cutting-edge solutions. Solutions Engineers collaborate with operations and business development to help craft solutions to meet customer business problems.
A Solutions Engineer works to balance various aspects of the project, from safety to design. Additionally, a Solutions Engineer researches advanced technology regarding best practices in the field and seek to find cost-effective solutions.
Job Description:
Were looking for a Solutions Engineer with deep experience in Big Data technologies, real-time data pipelines, and scalable infrastructure-someone whos been delivering critical systems under pressure, and knows what it takes to bring complex data architectures to life. This isnt just about checking boxes on tech stacks-its about solving real-world data problems, collaborating with smart people, and building robust, future-proof solutions.
In this role, youll partner closely with engineering, product, and customers to design and deliver high-impact systems that move, transform, and serve data at scale. Youll help customers architect pipelines that are not only performant and cost-efficient but also easy to operate and evolve.
We want someone whos comfortable switching hats between low-level debugging, high-level architecture, and communicating clearly with stakeholders of all technical levels.
Key Responsibilities:
Build distributed data pipelines using technologies like Kafka, Spark (batch & streaming), Python, Trino, Airflow, and S3-compatible data lakes-designed for scale, modularity, and seamless integration across real-time and batch workloads.
Design, deploy, and troubleshoot hybrid cloud/on-prem environments using Terraform, Docker, Kubernetes, and CI/CD automation tools.
Implement event-driven and serverless workflows with precise control over latency, throughput, and fault tolerance trade-offs.
Create technical guides, architecture docs, and demo pipelines to support onboarding, evangelize best practices, and accelerate adoption across engineering, product, and customer-facing teams.
Integrate data validation, observability tools, and governance directly into the pipeline lifecycle.
Own end-to-end platform lifecycle: ingestion → transformation → storage (Parquet/ORC on S3) → compute layer (Trino/Spark).
Benchmark and tune storage backends (S3/NFS/SMB) and compute layers for throughput, latency, and scalability using production datasets.
Work cross-functionally with R&D to push performance limits across interactive, streaming, and ML-ready analytics workloads.
Operate and debug object store-backed data lake infrastructure, enabling schema-on-read access, high-throughput ingestion, advanced searching strategies, and performance tuning for large-scale workloads.

we are looking for a Big Data Engineer.
As a Senior Big Data Engineer, working within Mobility Group, you will play a pivotal role in designing, developing, and maintaining the data infrastructure that powers our location analytics platform.
RESPONSIBILITIES:
Data Pipeline Architecture and Development: Design, build, and optimize robust and scalable data pipelines to process, transform, and integrate large volumes of data from various sources into our analytics platform.
Data Quality Assurance: Implement data validation, cleansing, and enrichment techniques to ensure high-quality and consistent data across the platform.
Performance Optimization: Identify performance bottlenecks and optimize data processing and storage mechanisms to enhance overall system performance and reduce latency.
Cloud Infrastructure: Work extensively with cloud-based technologies (GCP and AWS), to design and manage scalable data infrastructure.
Collaboration: Collaborate with cross-functional teams including Data Analysts, Data Scientists, Product Managers, and Software Engineers to understand requirements and deliver solutions that meet business needs.
Data Governance: Implement and enforce data governance practices, ensuring compliance with relevant regulations and best practices related to data privacy and security.
Monitoring and Maintenance: Monitor the health and performance of data pipelines, troubleshoot issues, and ensure high availability of data infrastructure.
Mentorship: Provide technical guidance and mentorship to junior data engineers, fostering a culture of learning and growth within the team.

We are looking for a Software Developer to join our Big Data and Analytics department.
Our Big Data Platform delivers the foundational, high-performance data infrastructure that powers dashboards, APIs and contextual insights for both customer facing and internal products.
As a developer on the Big Data Platform, you will be a part of the outstanding group that builds our intelligent data infrastructure, directly powering context-aware AI models and autonomous workflow automation. We are looking for people who look at engineering differently-people who are passionate about scaling AI-ready data layers, optimizing real-time context streaming, and leading the smart platform solutions they envision. If you are passionate about developing a cutting-edge data foundation that drives conversational AI execution across enterprise applications, making a massive impact, and striving to work with the industrys best talent-this is where you want to be!
What You'll Own:
End to end development of features and systems that are used daily by key stakeholders, both internally and externally
A technical focal point for software development best practices and design of main components, services and systems.
Explore, implement, and introduce cutting-edge technologies and innovative ideas.

As a Senior Software Engineer your role will be to:

Play a key role in the design and architecture of cutting-edge, innovative data systems.
Develop and maintain a multi-tenant environment for our advanced measurement solutions.
Build and manage cloud-based, distributed computing environments to support scalable, high-performance operations.
Design and implement robust ETL workflows, integrating multiple data sources for cleaning, transformation, aggregation, and querying.
Troubleshoot and resolve complex issues related to data integration, storage, and retrieval.
Architect and build big data solutions capable of managing and automating the processing of tens of terabytes.
Optimize database performance and streamline query execution in large-scale environments.
Collaborate closely with Data Scientists to align technical solutions with data-driven objectives.

Were looking for a Big Data Engineer to develop and integrate systems that retrieve, process and analyze data from around the digital world, generating customer-facing data. This role will report to our Team Manager, R&D.
Why is this role so important?
A data-focused company, and data is the heart of our business.
As a big data engineer developer, you will work at the very core of the company, designing and implementing complex high scale systems to retrieve and analyze data from millions of digital users.
Your role as a big data engineer will give you the opportunity to use the most cutting-edge technologies and best practices to solve complex technical problems while demonstrating technical leadership.
So, what will you be doing all day?
Your role as part of the R&D team means your daily responsibilities may include:
Design and implement complex high scale systems using a large variety of technologies.
You will work in a data research team alongside other data engineers, data scientists and data analysts. Together you will tackle complex data challenges and bring new solutions and algorithms to production.
Contribute and improve the existing infrastructure of code and data pipelines, constantly exploring new technologies and eliminating bottlenecks.
You will experiment with various technologies in the domain of Machine Learning and big data processing.
You will work on a monitoring infrastructure for our data pipelines to ensure smooth and reliable data ingestion and calculation.