דרושים » הנדסה » Experienced Big Data Engineer

Were looking for a Senior Data Engineer in the Data Collection Ingest team to contribute to the design and coding of data ingestion services & pipelines. This role involves working as part of a team that handles millions of requests per minute across multiple servers, and is responsible for a wide-range of data pipelines, processing billions of events each day.
Why is this role so important?
As the most trusted platform for measuring online behavior, millions of people rely on insights daily as the ground truth for their knowledge of the digital world. Producing these insights requires large scale raw data to be ingested reliably in high scale to provide stable signals for analysis. As a Data Engineer you will have the opportunity to perform hands-on work and own raw data ingestion pipeline end-to-end. Your work will have a direct impact on the quality and reliability of our data and the insights that our products are delivering to our customers.

Your Career We are looking for a highly talented technical individual to join the Cortex XDR infrastructure team. The team is responsible for developing automation infrastructure and various cloud based tools and platforms that are used across the research, development and QA departments to ensure the functionality, stability and quality of the XDR product, alongside the efficiency of the infrastructure and process used to build, test and deploy on various clouds and distributions. We believe that the platforms & infrastructure that the team provides are a critical & crucial part of our department's progress to the modern future and one of our key growth factors. As a Platform engineer you will play a pivotal role in enhancing our development and automation experience by pushing forward modern automation approaches, eliminating manual efforts and introducing new development operations for continuous integration, scale & durability using advanced cloud services. Your expertise will be used in areas such as infrastructure development, cloud based automation, serverless infrastructure, automation as a service, providing technical guidance, and pushing infrastructure\configuration as a code and GitOps approach across the development departments. To succeed in this role, you should have a strong foundation in modern cloud based automation methodologies and a comprehensive understanding of industry best practices, especially in redundancy and scalability of large systems and the ability to control them via SCM based declarative configs. You should be familiar with modern public clouds approaches and serverless based architectures, including virtualization containers and container based orchestration including multiple Kubernetes based deployments. You should be comfortable engaging in complex technical discussions and advocating for optimal solutions in a fast-paced growing environment as part of our quest for continuous improvement. Your Impact Utilize modern technologies including serverless cloud services, Kubernetes, Terraform, among others, and use them all in an infrastructure/configuration as a code GItOps approach to manage everything via source code and continuous integration processes Design and implement (hands on) the next generation of platforms, automation frameworks, SDKs, and tools to be used across our entire R&D group, and be part of our infrastructure transition to the cloud Develop and maintain a cloud based test execution system, that supports parallel executions on multiple operating systems and multiple cloud providers and at a very large scale, and by so helping reduce the amount of effort required to perform automatic testing and manual testing, and reduce time to market Provide tools, systems and simulators for scaling up all lifecycle phases of our products and services including cross company and third party integrations and frameworks to be used in high scale Introduce progress and help revolutionize our operations and lay the foundation for innovation and growth.

As a Data Engineer at Meta, you will shape the future of people-facing and business-facing products we build across our entire family of applications (Facebook, Instagram, Messenger, WhatsApp, Reality Labs, Threads). Your technical skills and analytical mindset will be utilized designing and building some of the world's most extensive data sets, helping to craft experiences for billions of people and hundreds of millions of businesses worldwide.
In this role, you will collaborate with software engineering, data science, and product management teams to design/build scalable data solutions across Meta to optimize growth, strategy, and user experience for our 3 billion plus users, as well as our internal employee community.
You will be at the forefront of identifying and solving some of the most interesting data challenges at a scale few companies can match. By joining Meta, you will become part of a world-class data engineering community dedicated to skill development and career growth in data engineering and beyond.
Data Engineering: You will guide teams by building optimal data artifacts (including datasets and visualizations) to address key questions. You will refine our systems, design logging solutions, and create scalable data models. Ensuring data security and quality, and with a focus on efficiency, you will suggest architecture and development approaches and data management standards to address complex analytical problems.
Product leadership: You will use data to shape product development, identify new opportunities, and tackle upcoming challenges. You'll ensure our products add value for users and businesses, by prioritizing projects, and driving innovative solutions to respond to challenges or opportunities.
Communication and influence: You won't simply present data, but tell data-driven stories. You will convince and influence your partners using clear insights and recommendations. You will build credibility through structure and clarity, and be a trusted strategic partner.
Data Engineer, Product Analytics Responsibilities
Conceptualize and own the data architecture for multiple large-scale projects, while evaluating design and operational cost-benefit tradeoffs within systems
Create and contribute to frameworks that improve the efficacy of logging data, while working with data infrastructure to triage issues and resolve
Collaborate with engineers, product managers, and data scientists to understand data needs, representing key data insights visually in a meaningful way
Define and manage Service Level Agreements for all data sets in allocated areas of ownership
Determine and implement the security model based on privacy requirements, confirm safeguards are followed, address data quality issues, and evolve governance processes within allocated areas of ownership
Design, build, and launch collections of sophisticated data models and visualizations that support multiple use cases across different products or domains
Solve our most challenging data integration problems, utilizing optimal Extract, Transform, Load (ETL) patterns, frameworks, query techniques, sourcing from structured and unstructured data sources
Assist in owning existing processes running in production, optimizing complex code through advanced algorithmic concepts
Optimize pipelines, dashboards, frameworks, and systems to facilitate easier development of data artifacts.

As a Data Engineer at Meta, you will shape the future of people-facing and business-facing products we build across our entire family of applications. Your technical skills and analytical mindset will be utilized designing and building some of the world's most extensive data sets, helping to craft experiences for billions of people and hundreds of millions of businesses worldwide.
In this role, you will collaborate with software engineering, data science, and product management teams to design/build scalable data solutions across Meta to optimize growth, strategy, and user experience for our 3 billion plus users, as well as our internal employee community.
You will be at the forefront of identifying and solving some of the most interesting data challenges at a scale few companies can match. By joining Meta, you will become part of a world-class data engineering community dedicated to skill development and career growth in data engineering and beyond.
Data Engineering: You will guide teams by building optimal data artifacts (including datasets and visualizations) to address key questions. You will refine our systems, design logging solutions, and create scalable data models. Ensuring data security and quality, and with a focus on efficiency, you will suggest architecture and development approaches and data management standards to address complex analytical problems.
Product leadership: You will use data to shape product development, identify new opportunities, and tackle upcoming challenges. You'll ensure our products add value for users and businesses, by prioritizing projects, and driving innovative solutions to respond to challenges or opportunities.
Communication and influence: You won't simply present data, but tell data-driven stories. You will convince and influence your partners using clear insights and recommendations. You will build credibility through structure and clarity, and be a trusted strategic partner.
Data Engineer, Product Analytics Responsibilities
Conceptualize and own the data architecture for multiple large-scale projects, while evaluating design and operational cost-benefit tradeoffs within systems
Create and contribute to frameworks that improve the efficacy of logging data, while working with data infrastructure to triage issues and resolve
Collaborate with engineers, product managers, and data scientists to understand data needs, representing key data insights in a meaningful way
Define and manage Service Level Agreements for all data sets in allocated areas of ownership
Determine and implement the security model based on privacy requirements, confirm safeguards are followed, address data quality issues, and evolve governance processes within allocated areas of ownership
Design, build, and launch collections of sophisticated data models and visualizations that support multiple use cases across different products or domains
Solve our most challenging data integration problems, utilizing optimal Extract, Transform, Load (ETL) patterns, frameworks, query techniques, sourcing from structured and unstructured data sources
Assist in owning existing processes running in production, optimizing complex code through advanced algorithmic concepts
Optimize pipelines, dashboards, frameworks, and systems to facilitate easier development of data artifacts.

Required Senior Data Engineer
About us:
A pioneering health-tech startup on a mission to revolutionize weight loss and well-being. Our innovative metabolic measurement device provides users with a comprehensive understanding of their metabolism, empowering them with personalized, data-driven insights to make informed lifestyle choices.
Data is at the core of everything we do. We collect and analyze vast amounts of user data from our device and app to provide personalized recommendations, enhance our product, and drive advancements in metabolic health research. As we continue to scale, our data infrastructure is crucial to our success and our ability to empower our users.
About the Role:
As a Senior Data Engineer, youll be more than just a coder - youll be the architect of our data ecosystem. Were looking for someone who can design scalable, future-proof data pipelines and connect the dots between DevOps, backend engineers, data scientists, and analysts.
Youll lead the design, build, and optimization of our data infrastructure, from real-time ingestion to supporting machine learning operations. Every choice you make will be data-driven and cost-conscious, ensuring efficiency and impact across the company.
Beyond engineering, youll be a strategic partner and problem-solver, sometimes diving into advanced analysis or data science tasks. Your work will directly shape how we deliver innovative solutions and support our growth at scale.
Responsibilities:
Design and Build Data Pipelines: Architect, build, and maintain our end-to-end data pipeline infrastructure to ensure it is scalable, reliable, and efficient.
Optimize Data Infrastructure: Manage and improve the performance and cost-effectiveness of our data systems, with a specific focus on optimizing pipelines and usage within our Snowflake data warehouse. This includes implementing FinOps best practices to monitor, analyze, and control our data-related cloud costs.
Enable Machine Learning Operations (MLOps): Develop the foundational infrastructure to streamline the deployment, management, and monitoring of our machine learning models.
Support Data Quality: Optimize ETL processes to handle large volumes of data while ensuring data quality and integrity across all our data sources.
Collaborate and Support: Work closely with data analysts and data scientists to support complex analysis, build robust data models, and contribute to the development of data governance policies.

As a Data Engineer at Meta, you will shape the future of people-facing and business-facing products we build across our entire family of applications. Your technical skills and analytical mindset will be utilized designing and building some of the world's most extensive data sets, helping to craft experiences for billions of people and hundreds of millions of businesses worldwide.
In this role, you will collaborate with software engineering, data science, and product management teams to design/build scalable data solutions across Meta to optimize growth, strategy, and user experience for our 3 billion plus users, as well as our internal employee community.
You will be at the forefront of identifying and solving some of the most interesting data challenges at a scale few companies can match. By joining Meta, you will become part of a world-class data engineering community dedicated to skill development and career growth in data engineering and beyond.
Data Engineering: You will guide teams by building optimal data artifacts (including datasets and visualizations) to address key questions. You will refine our systems, design logging solutions, and create scalable data models. Ensuring data security and quality, and with a focus on efficiency, you will suggest architecture and development approaches and data management standards to address complex analytical problems.
Product leadership: You will use data to shape product development, identify new opportunities, and tackle upcoming challenges. You'll ensure our products add value for users and businesses, by prioritizing projects, and driving innovative solutions to respond to challenges or opportunities.
Communication and influence: You won't simply present data, but tell data-driven stories. You will convince and influence your partners using clear insights and recommendations. You will build credibility through structure and clarity, and be a trusted strategic partner.
Data Engineer, Product Analytics Responsibilities
Collaborate with engineers, product managers, and data scientists to understand data needs, representing key data insights in a meaningful way
Design, build, and launch collections of sophisticated data models and visualizations that support multiple use cases across different products or domains
Define and manage Service Level Agreements for all data sets in allocated areas of ownership
Solve challenging data integration problems, utilizing optimal Extract, Transform, Load (ETL) patterns, frameworks, query techniques, sourcing from structured and unstructured data sources
Improve logging
Assist in owning existing processes running in production, optimizing complex code through advanced algorithmic concepts
Optimize pipelines, dashboards, frameworks, and systems to facilitate easier development of data artifacts
Influence product and cross-functional teams to identify data opportunities to drive impact.

Our data engineering team is looking for an experienced professional with expertise in SQL, Python, and strong data modeling skills. In this role, you will be at the heart of our data ecosystem, designing and maintaining cross-engineering initiatives and projects, as well as developing high-quality data pipelines and models that drive decision-making across the organization.

You will play a key role in ensuring data quality, building scalable systems, and supporting cross-functional teams with clean, accurate, and actionable data.



What you will do:

Design, develop, and optimize data services and solutions required to support various company products (like FeatureStore or synthetic data management); Work closely with data analysts, data scientists, engineers, and cross-functional teams to understand data requirements and deliver high-quality solutions.
Design and integrate LLM- and agent-based capabilities into data platforms and services, enabling smarter data operations and AI-driven data products.
Design, develop, and optimize scalable data pipelines to ensure data is clean, accurate, and ready for analysis.
Build and maintain robust data models that support clinical, business intelligence, and operational needs.
Implement and enforce data quality standards, monitoring, and best practices across systems and pipelines.
Manage and optimize large-scale data storage and processing systems to ensure reliability and performance.

Required Big Data Engineer Team Lead
We are an innovative technology company led by data scientists and engineers devoted to mobile app growth. Our proprietary ad platform is powered by machine learning and is constantly improving its distribution algorithms.
We deliver valuable results and insights for a fast-growing clientele of major app developers using elite programmatic user acquisition and retargeting technologies.
Our state-of-the-art machine learning technology analyzes 50TB of raw data per day to produce millions of ad recommendations in real-time. This data is used to power our machine learning predictions, business-critical metrics, and analytics to power our decision-making.
As a Big Data Team Lead, you will lead a team focused on building scalable, algorithm-heavy data products rather than owning the underlying data infrastructure.
Your team will be responsible for implementing complex business and optimization logic-often incorporating multiple algorithms using Peta-Byte scale data processing (primarily PySpark). A core part of the role is ensuring these data products are explainable, observable, and trusted by Data Science, BI, Product, and business stakeholders.
You will work closely with the Data Infrastructure TL, Data Science, BI, and Product to turn advanced logic into production-grade, transparent, and measurable systems.
Responsibilities:
Lead development of scalable, algorithm-heavy data products using PySpark and Implement and maintain complex, multi-stage data flows with multiple interacting algorithms.
Translate business and algorithmic requirements into clear, testable data logic. Ensuring explainability of outputs through intermediate artifacts, metadata, and documentation & Defining and monitoring; observability signals for data quality, algorithm health, and business impact.
Mentor engineers on reasoning about complex logic, edge cases, and failure modes.