לוח משרות דרושים איש ביג דאטה Big Data, לחיפוש זה נמצאו 21 משרות

We are seeking an experienced Solutions Data Engineer who possess both technical depth and strong interpersonal skills to partner with internal and external teams to develop scalable, flexible, and cutting-edge solutions. Solutions Engineers collaborate with operations and business development to help craft solutions to meet customer business problems.
A Solutions Engineer works to balance various aspects of the project, from safety to design. Additionally, a Solutions Engineer researches advanced technology regarding best practices in the field and seek to find cost-effective solutions.
Job Description:
Were looking for a Solutions Engineer with deep experience in Big Data technologies, real-time data pipelines, and scalable infrastructure-someone whos been delivering critical systems under pressure, and knows what it takes to bring complex data architectures to life. This isnt just about checking boxes on tech stacks-its about solving real-world data problems, collaborating with smart people, and building robust, future-proof solutions.
In this role, youll partner closely with engineering, product, and customers to design and deliver high-impact systems that move, transform, and serve data at scale. Youll help customers architect pipelines that are not only performant and cost-efficient but also easy to operate and evolve.
We want someone whos comfortable switching hats between low-level debugging, high-level architecture, and communicating clearly with stakeholders of all technical levels.
Key Responsibilities:
Build distributed data pipelines using technologies like Kafka, Spark (batch & streaming), Python, Trino, Airflow, and S3-compatible data lakes-designed for scale, modularity, and seamless integration across real-time and batch workloads.
Design, deploy, and troubleshoot hybrid cloud/on-prem environments using Terraform, Docker, Kubernetes, and CI/CD automation tools.
Implement event-driven and serverless workflows with precise control over latency, throughput, and fault tolerance trade-offs.
Create technical guides, architecture docs, and demo pipelines to support onboarding, evangelize best practices, and accelerate adoption across engineering, product, and customer-facing teams.
Integrate data validation, observability tools, and governance directly into the pipeline lifecycle.
Own end-to-end platform lifecycle: ingestion → transformation → storage (Parquet/ORC on S3) → compute layer (Trino/Spark).
Benchmark and tune storage backends (S3/NFS/SMB) and compute layers for throughput, latency, and scalability using production datasets.
Work cross-functionally with R&D to push performance limits across interactive, streaming, and ML-ready analytics workloads.
Operate and debug object store-backed data lake infrastructure, enabling schema-on-read access, high-throughput ingestion, advanced searching strategies, and performance tuning for large-scale workloads.

As a Senior Software Engineer your role will be to:

Play a key role in the design and architecture of cutting-edge, innovative data systems.
Develop and maintain a multi-tenant environment for our advanced measurement solutions.
Build and manage cloud-based, distributed computing environments to support scalable, high-performance operations.
Design and implement robust ETL workflows, integrating multiple data sources for cleaning, transformation, aggregation, and querying.
Troubleshoot and resolve complex issues related to data integration, storage, and retrieval.
Architect and build big data solutions capable of managing and automating the processing of tens of terabytes.
Optimize database performance and streamline query execution in large-scale environments.
Collaborate closely with Data Scientists to align technical solutions with data-driven objectives.

we are a dynamic technology firm dedicated to building exceptional products using modern technologies for the world's most admired companies. We pride ourselves on innovation, collaboration, and delivering excellence in every project.

Job Description
Responsibilities:

Design and maintain scalable data pipelines with Spark Structured Streaming.
Implement Lakehouse architecture with Apache Iceberg.
Develop ETL processes for data transformation.
Ensure data integrity, quality, and governance.
Collaborate with stakeholders and IT teams for seamless solution integration.
Optimize data processing workflows and performance.

In this role, you will be responsible for building data infrastructure for performance tuning, and analysis of a new analytics product. As a member of our Data Performance team, you will collaborate closely with other SW & HW Engineers to improve the performance of our new cutting-edge APUs. You will collaborate with our business leaders and participate in the product design and implementation. You will evaluate, implement, and integrate a diverse set of tools and technologies such as Spark, Comet, Velox, Dagster, Hardware emulators and software simulators.





Join a multi-disciplinary team of experts operating at the foundation of data infrastructure stack

Build a robust data benchmarking infrastructure

Develop in-house measurements and ETL tools

Conduct detailed performance analysis and characterization

Research state-of-the-art benchmarks and techniques for SQL and data optimization

We are looking for a Senior Data Engineer with strong Python expertise and extensive experience building large-scale, distributed data systems.

You will play a key role in developing our next-generation data platform, building integrations with external data sources, and designing the services and pipelines that collect, process, and serve large volumes of data across our product.

You will own systems throughout their full lifecycle - from system design and hands-on development to deployment, scalability, and production monitoring - while contributing to technical decisions and working closely with the team on building reliable, high-performance data solutions.

Our Office is located in Kiryat Ono.This is a hybrid position, including two work-from-home days per week, with flexibility .

Were looking for a Big Data Engineer to develop and integrate systems that retrieve, process and analyze data from around the digital world, generating customer-facing data. This role will report to our Team Manager, R&D.
Why is this role so important?
A data-focused company, and data is the heart of our business.
As a big data engineer developer, you will work at the very core of the company, designing and implementing complex high scale systems to retrieve and analyze data from millions of digital users.
Your role as a big data engineer will give you the opportunity to use the most cutting-edge technologies and best practices to solve complex technical problems while demonstrating technical leadership.
So, what will you be doing all day?
Your role as part of the R&D team means your daily responsibilities may include:
Design and implement complex high scale systems using a large variety of technologies.
You will work in a data research team alongside other data engineers, data scientists and data analysts. Together you will tackle complex data challenges and bring new solutions and algorithms to production.
Contribute and improve the existing infrastructure of code and data pipelines, constantly exploring new technologies and eliminating bottlenecks.
You will experiment with various technologies in the domain of Machine Learning and big data processing.
You will work on a monitoring infrastructure for our data pipelines to ensure smooth and reliable data ingestion and calculation.

We are looking for a talented and dedicated ML & Big Data Analyst who will be a part of the Core Data group.
As part of this role, you will help us process and streamline rich and complicated datasets. Will explore and validate new data sources. Be part of the data science and data engineering force creating new features for our solution.
What does the day-to-day of an ML & Big Data Analyst look like:
Apply your expertise in quantitative analysis and data mining to turn data into insights.
Conduct research and develop tools that will ensure the quality of our data and algorithms.
Utilize your deep understanding of our data in order to evaluate and improve novel algorithms produced by the data scientists in your team.
Partner with Big Data Engineers to establish an infrastructure to scale solutions.

We are seeking a highly skilled and versatile Senior Data Infrastructure Engineer to join our team and participate in the design, implementation and maintainance of cutting-edge Data Platforms and Big Data solutions. This position allows you to have the unique opportunity to significantly impact intelligence processes, contribute to counter-terrorism efforts by leveraging advanced Big Data technologies.
This role combines expertise in Big Data Infrastructure, Big Data Engineering, architecture and support of critical operations and intelligence processes.
עוד על התפקיד
Design, install, maintain and upgrade Big Data technologies.
Monitor and optimize system performance, identifying areas for improvement and implementing solutions.
Provide technical support to internal teams and end-users, ensuring seamless operations.
Conduct POCs and recommend adoption of new technologies and tools in the Big Data domain.
Develop automation solutions for infrastructure management, empowering teams to operate efficiently.

we are looking for a Big Data Engineer.
As a Senior Big Data Engineer, working within Mobility Group, you will play a pivotal role in designing, developing, and maintaining the data infrastructure that powers our location analytics platform.
RESPONSIBILITIES:
Data Pipeline Architecture and Development: Design, build, and optimize robust and scalable data pipelines to process, transform, and integrate large volumes of data from various sources into our analytics platform.
Data Quality Assurance: Implement data validation, cleansing, and enrichment techniques to ensure high-quality and consistent data across the platform.
Performance Optimization: Identify performance bottlenecks and optimize data processing and storage mechanisms to enhance overall system performance and reduce latency.
Cloud Infrastructure: Work extensively with cloud-based technologies (GCP and AWS), to design and manage scalable data infrastructure.
Collaboration: Collaborate with cross-functional teams including Data Analysts, Data Scientists, Product Managers, and Software Engineers to understand requirements and deliver solutions that meet business needs.
Data Governance: Implement and enforce data governance practices, ensuring compliance with relevant regulations and best practices related to data privacy and security.
Monitoring and Maintenance: Monitor the health and performance of data pipelines, troubleshoot issues, and ensure high availability of data infrastructure.
Mentorship: Provide technical guidance and mentorship to junior data engineers, fostering a culture of learning and growth within the team.

Architectural leadership of the enterprise DWH project in an AWS cloud environment. The role focuses on designing robust data infrastructure, establishing the technological roadmap, and enabling seamless large-scale data processing to support BI and AI initiatives.

Key Responsibilities:

Lead the architectural design and implementation of the new enterprise DWH project.
Design, develop, and optimize complex cloud-based data solutions for processing massive datasets.
Select and implement the optimal AWS services (Redshift, Glue, Athena, EMR) and data formats (Parquet, Iceberg) for the core data stack.
Provide technical leadership, mentorship, and professional guidance to the Data Engineering team.
Collaborate closely with development teams, Data Scientists, and end-users to define requirements and deliver innovative Data and AI solutions.

Integration into the development team for a core DWH project and various AI projects. Responsibilities include the ingestion of large volumes of new data, followed by deep understanding and analysis of data in close collaboration with Data Scientists. Design, development, and optimization of critical, diverse, and large-scale data pipelines - in both cloud and on-premise environments.



Key Responsibilities:

Take responsibility for the ingestion of large volumes of new data.
Conduct deep understanding and examination of data in close collaboration with Data Scientists.
Design and develop critical, diverse, and large-scale data pipelines and processes.
Deploy and optimize data solutions across both cloud and on-premise environments.

Perform Client Due Diligence to evaluate the customer from a compliance perspective and ensure that all established procedures are followed while providing a great customer experience

Continuously improve streamlined processes to manage the end-to-end onboarding process from a financial crimes risk perspective

Work with the Compliance Team to ensure adherence to local and global compliance policies and procedures

Stay up-to-date on AML-related news and emerging regulatory developments Perform period reviews and support audits

Were looking for an experienced Query Infra Team Lead to build and lead this team - someone who comes from deep data infrastructure, analytical databases, search infrastructure, and large-scale querying, and who wants to shape one of the most important technical foundations of platform.
What youll do
Create, build and lead Data Infra team from the ground up.

Own the infrastructure of investigation data layer for the Platform.

Turn investigation infrastructure into a dedicated, well-defined platform capability.

Design and evolve the data models, schemas, indexing strategies, and query patterns that power investigation databases.

Lead the architecture, operation, and scaling of the databases and search infrastructure that power investigation, analytics, and query use cases across the platform.

Partner closely with Product, Backend, DevOps, Security Research, and customer-facing engineering teams to define clear boundaries, contracts, and data-layer APIs.

Make the data layer reliable, observable, cost-efficient, and performant across many tenants and large volumes of security data.

Set engineering standards for schema design, data modeling, query optimization, migrations, backfills, and data-quality monitoring.

Mentor engineers, review technical designs, and raise the bar for how builds and operates data infrastructure.

Stay hands-on: this is a leadership role for someone who can still go deep into architecture, production issues, performance bottlenecks, and complex queries.

We're looking for an experienced Software Engineer to join our Data Platform Group. In this key role, you will build the company data platform: cloud-based microservices and data pipelines that process on the order of 1M records/sec at low latency. Because the platform is the foundation other groups build on, your work has a direct impact on our customers and enables engineering, product, and research teams across the organization.
The Lakehouse team owns the data itself - how it is stored, organized, retained, and served. We own our analytical storage layer, the batch processing built on top of it, and the APIs through which customers and the rest of the company consume data. If you enjoy the problems that only appear at petabyte scale - physical data layout, query performance, storage cost, and retention - this is the role.

Responsibilities:
End-to-end ownership of our large-scale analytical storage layer: data modeling, schema and table design, partitioning, retention, and query performance.
Design and develop the batch processing layer over our data lake using Spark on EMR (Java and PySpark).
Build and evolve Java/Spring Boot services that expose our data through well-defined APIs to customers and to consumers across the company.
Own performance and cost: query optimization, file layout and compaction, cluster sizing, and storage efficiency at scale.
Research new technologies in the lakehouse and analytical-storage space and adapt them for use in our product.
Work closely with product, DevOps, and security teams.