דרושים » דאטה » Core Data - Big Data Engineer

Were looking for a Senior Data Engineer in the Data Collection Ingest team to contribute to the design and coding of our data ingestion services & pipelines. This role involves working as part of a team that handles millions of requests per minute across multiple servers, and is responsible for a wide-range of data pipelines, processing billions of events each day.
Why is this role so important?
As the most trusted platform for measuring online behavior, millions of people rely on our insights daily as the ground truth for their knowledge of the digital world. Producing these insights requires large scale raw data to be ingested reliably in high scale to provide stable signals for analysis. As a Data Engineer you will have the opportunity to perform hands-on work and own our raw data ingestion pipeline end-to-end. Your work will have a direct impact on the quality and reliability of our data and the insights that our products are delivering to our customers.
So, what will you be doing all day?
Your role as Data Engineer of the Ingest Data Collection team means your daily responsibilities may include:
Design, code and manage end-to-end our data ingestion pipelines, both online and offline.
Take charge of developing & maintaining modern data infrastructure, while implementing best practices for building data pipelines.
Be responsible for high-scale ingestion services, solving challenges of availability, reliability, and scalability.
Run the production environment by monitoring availability and taking a holistic view of system health and data quality.
Own data infrastructure features from design to production using industry best practices with focus on quality and delivery.
Lead design & decision-making processes of the team.
Solve diverse complex problems of scale, performance and business logic.
Collaborate with product managers and other team leaders to plan, nurture, and implement an efficient and effective development process.
Continuously learn and evaluate new technologies in the everlasting effort to perfect our products
Perform code reviews, evaluate implementations, and provide feedback about potential improvements.
Improve your skills, learn from and mentor top-notch engineers and enrich other team members.
Have lots of fun!

Were looking for a Data Engineer to join our Data Labs (DL) department, which specializes in professional services for our super-premium customers. This role will report to our DL (Datalabs) Data Science Team Manager in the R&D.
Why is this role so important?
We are a data-focused company, and our unique AI and machine learning capabilities are the center of our business.
As part of this role, you will create and support a complex data model pipeline that helps analyze the petabytes of data we receive from various sources, and research and develop new features and capabilities for our product solutions
As a data engineer in the Datalabs team, you will work on the very core of the company. Part of your role will be to create processes that help turn raw data into usable metrics and leverage AI models and statistical algorithms to support out-of-the-box requests from customers who want custom data labs. The Datalabs departments business-oriented nature also means you will be supporting a team of analysts and data scientists who interact directly with customers. Together with them, you will translate the voice of these customers into best-in-class data labs.
So, what will you be doing all day?
Building and maintaining our big-data pipelines
Take a major part in designing and implementing complex high-scale systems using a large variety of technologies
Be part of a team with smart and motivated engineers, and data scientists, to collaborate on the planning, development, and maintenance of our products
Implement solutions in the AWS cloud environment, and work in Databricks with PySpark

Grip Security is looking for a Senior Data Platform Engineer to join our community!
We are a fast-growing startup in the software-as-a-service and AI Security Industry. We provide innovative solutions to securing the whole organization-to-SaaS surface. (More details: https://grip.security)
Using the newest technologies, we're working on solving a huge problem all enterprises face today - to govern the accessibility of all their employees to all 3rd party vendors (GitHub, SendGrid, Atlassian, and thousands more!), and ensure there is no leftover/unwanted access to any of the organization's SaaS and AI assets. The SaaS and AI security field is complex and challenging; therefore, we're looking for super-talented people, who are not afraid of technical challenges and breaking down barriers to achieve good solutions.
The job
As a Senior Data Platform Engineer, you will play a key role in building and evolving Grips modern data platform - the infrastructure that powers product features and analytics across the company.
You will focus on designing and operating scalable, reliable data systems and platform tooling that support our Data Lakehouse, enabling engineers, analysts and research teams to work with data efficiently and with minimal friction.
Responsibilities:
Design, build and operate a cloud-native modern data platform.
Develop and optimize data processing frameworks and pipelines across batch and streaming workloads.
Improve developer experience and platform usability through tooling and automation.
Lead and support large-scale data migrations and architectural improvements.
Drive best practices around infrastructure, CI/CD, testing, and system design.
Collaborate with developers, analysts, data scientists and other stakeholders to develop new products and features.
Contribute to a strong engineering culture of ownership, learning, and knowledge sharing.

As a Senior Data Engineer, youll collaborate with top-notch engineers and data scientists to elevate our platform to the next level and deliver exceptional user experiences. Your primary focus will be on the data engineering aspects-ensuring the seamless flow of high-quality, relevant data to train and optimize content models, including GenAI foundation models, supervised fine-tuning, and more.

Youll work closely with teams across the company to ensure the availability of high-quality data from ML platforms, powering decisions across all departments. With access to petabytes of data through MySQL, Snowflake, Cassandra, S3, and other platforms, your challenge will be to ensure that this data is applied even more effectively to support business decisions, train and monitor ML models and improve our products.


Key Job Responsibilities and Duties:

Rapidly developing next-generation scalable, flexible, and high-performance data pipelines.

Dealing with massive textual sources to train GenAI foundation models.

Solving issues with data and data pipelines, prioritizing based on customer impact.

End-to-end ownership of data quality in our core datasets and data pipelines.

Experimenting with new tools and technologies to meet business requirements regarding performance, scaling, and data quality.

Providing tools that improve Data Quality company-wide, specifically for ML scientists.

Providing self-organizing tools that help the analytics community discover data, assess quality, explore usage, and find peers with relevant expertise.

Acting as an intermediary for problems, with both technical and non-technical audiences.

Promote and drive impactful and innovative engineering solutions.

Technical, behavioral and interpersonal competence advancement via on-the-job opportunities, experimental projects, hackathons, conferences, and active community participation.

Collaborate with multidisciplinary teams: Collaborate with product managers, data scientists, and analysts to understand business requirements and translate them into machine learning solutions. Provide technical guidance and mentorship to junior team members.

As a Data Engineer, youll collaborate with top-notch engineers and data scientists to elevate our platform to the next level and deliver exceptional user experiences. Your primary focus will be on the data engineering aspects-ensuring the seamless flow of high-quality, relevant data to train and optimize content models, including GenAI foundation models, supervised fine-tuning, and more.

Youll work closely with teams across the company to ensure the availability of high-quality data from ML platforms, powering decisions across all departments. With access to petabytes of data through MySQL, Snowflake, Cassandra, S3, and other platforms, your challenge will be to ensure that this data is applied even more effectively to support business decisions, train and monitor ML models and improve our products.


Key Job Responsibilities and Duties:

Rapidly developing next-generation scalable, flexible, and high-performance data pipelines.

Dealing with massive textual sources to train GenAI foundation models.

Solving issues with data and data pipelines, prioritizing based on customer impact.

End-to-end ownership of data quality in our core datasets and data pipelines.

Experimenting with new tools and technologies to meet business requirements regarding performance, scaling, and data quality.

Providing tools that improve Data Quality company-wide, specifically for ML scientists.

Providing self-organizing tools that help the analytics community discover data, assess quality, explore usage, and find peers with relevant expertise.

Acting as an intermediary for problems, with both technical and non-technical audiences.

Promote and drive impactful and innovative engineering solutions.

Technical, behavioral and interpersonal competence advancement via on-the-job opportunities, experimental projects, hackathons, conferences, and active community participation.

Collaborate with multidisciplinary teams: Collaborate with product managers, data scientists, and analysts to understand business requirements and translate them into machine learning solutions. Provide technical guidance and mentorship to junior team members.

Required Software & Data Engineer
Tel Aviv
We are on a mission to create public transportation systems that provide far greater access to jobs, healthcare, and education. Our platform serves as the technology backbone for modern transit networks, transforming antiquated and siloed public transportation systems into smart, data-driven, and efficient digital networks. With hundreds of agency partners around the world, we are recognized as the leading transportation technology and service provider globally.
As a Software & Data Engineer, you will play a central role in building and evolving the data infrastructure that powers one of the world's leading transportation platforms. You'll work with rich, complex transportation data - from ride bookings and payments to real-time operational signals - designing the pipelines and systems that turn raw data into clean, reliable, and actionable insight across the business. This is a high-ownership role on a talented and deeply motivated engineering team in Tel Aviv, where your work will directly shape how we understand and improve our service for cities and riders around the world.
About the Role:
Design and build highly scalable, reliable data pipelines that serve clean, structured data across our engineering, product, and business teams - ensuring the entire organisation can trust the data it works with
Own the architecture of complex data models that translate messy, real-world transportation data - bookings, payments, driver activity, and more - into systems that are fast, efficient, and built to scale
Lead end-to-end development across the full data lifecycle: from architecture and design through to deployment, monitoring, and continuous improvement
Proactively monitor data quality and reliability, identifying and resolving discrepancies before they reach stakeholders - building the kind of data infrastructure people can depend on
Collaborate with a broad forum of engineers, analysts, and business stakeholders to identify data needs, design POCs, and ship scalable solutions that make a real difference to how we operate
Contribute to the adoption and evolution of our modern data stack, including DBT, Airflow, Iceberg storage, and AWS big data tooling such as Glue, EMR, and Athena.

We are seeking a Data Platform Team Lead to play a pivotal role in building and maintaining our robust data platform. You will lead a talented team of data engineers ensuring a collaborative and high-performing environment within and outside the team.

The ideal candidate must have excellent business understanding, attention to detail, technical and analytical abilities, and a deep Data Engineeiring understanding. The candidate will be self-driven, independent, able to coordinate with different teams and groups and handle ambiguity well within the organization.

How will you contribute?
Provide technical leadership and direction for the data platform, including data pipelines, data lake, data governance, and data security.
Partner with data analysts and BI analysts to understand their needs and ensure the platform effectively supports their work.
Design, develop, and implement solutions to optimize data ingestion, storage, processing, and analysis.
Oversee data platform projects
Build self-serve accessibility tools and platforms for the entire company to use
Foster a culture of collaboration and continuous improvement within the data teams.
Mentor and develop the technical and analytical skills of the team members.
Stay up-to-date on the latest trends and technologies in data platforms and analytics.
Communicate effectively with stakeholders across technical and non-technical departments.

we are looking for a Data Engineer to join our growing team!
This is a great opportunity to be part of one of the fastest-growing infrastructure companies in history, an organization that is in the center of the hurricane being created by the revolution in artificial intelligence.
"our company's data management vision is the future of the market."- Forbes
we are the data platform company for the AI era. We are building the enterprise software infrastructure to capture, catalog, refine, enrich, and protect massive datasets and make them available for real-time data analysis and AI training and inference. Designed from the ground up to make AI simple to deploy and manage, our company takes the cost and complexity out of deploying enterprise and AI infrastructure across data center, edge, and cloud.
Our success has been built through intense innovation, a customer-first mentality and a team of fearless workers who leverage their skills & experiences to make real market impact. This is an opportunity to be a key contributor at a pivotal time in our companys growth and at a pivotal point in computing history.
In this role, you will be responsible for:
Designing, building, and maintaining scalable data pipeline architectures
Developing ETL processes to integrate data from multiple sources
Creating and optimizing data models for efficient storage and retrieval
Implementing data quality controls and monitoring systems
Collaborating with data scientists and analysts to deliver data solutions
Building and maintaining data warehouses and data lakes
Performing in-depth data analysis and providing insights to stakeholders
Taking full ownership of data quality, documentation, and governance processes
Building and maintaining comprehensive reports and dashboards
Ensuring data security and regulatory compliance.

we are looking for a Senior Data Engineer.
As our Senior Data Engineer you will handle Big Data in real time, and become the owner and gatekeeper of all data and data flows within the company. You will bring data ingenuity and technological excellence while gaining a deep business understanding.
This is an amazing opportunity to join a multi-disciplinary A-team while working in a fast-paced, modern cloud, data-oriented environment.
Implement robust, reliable and scalable data pipelines and data architecture.
Own and develop DWH (Petabytes of data!).
Own the entire data development process, including business knowledge, methodology, quality assurance, and monitoring.
Collaborate with cross-functional teams to define, design, and ship new features.
Continuously discover, evaluate, and implement new technologies to maximize development efficiency.
Develop tailormade solutions as part of our data pipelines.
Lead complex Big Data projects and build data platforms from scratch.
Work on a high-scale, real-time, real-world business-critical data stores and data endpoints.
Implement data profiling to identify anomalies and maintain data integrity.
Work in a results-driven, high-paced, rewarding environment.