דרושים » הנדסה » Senior Big Data Infrastructure Engineer 2572

Were looking for a Senior Data Engineer in the Data Collection Ingest team to contribute to the design and coding of data ingestion services & pipelines. This role involves working as part of a team that handles millions of requests per minute across multiple servers, and is responsible for a wide-range of data pipelines, processing billions of events each day.

So, what will you be doing all day?

Your role as Data Engineer of the Ingest Data Collection team means your daily responsibilities may include:

Design, code and manage end-to-end data ingestion pipelines, both online and offline.
Take charge of developing & maintaining modern data infrastructure, while implementing best practices for building data pipelines.
Be responsible for high-scale ingestion services, solving challenges of availability, reliability, and scalability.
Run the production environment by monitoring availability and taking a holistic view of system health and data quality.
Own data infrastructure features from design to production using industry best practices with focus on quality and delivery.
Lead design & decision-making processes of the team.
Solve diverse complex problems of scale, performance and business logic.
Collaborate with product managers and other team leaders to plan, nurture, and implement an efficient and effective development process.
Continuously learn and evaluate new technologies in the everlasting effort to perfect our products
Perform code reviews, evaluate implementations, and provide feedback about potential improvements.
Improve your skills, learn from and mentor top-notch engineers and enrich other team members.
Have lots of fun!

As a Senior Big Data Engineer, working within Mobility Group, you will play a pivotal role in designing, developing, and maintaining the data infrastructure that powers our location analytics platform.
RESPONSIBILITIES:

Data Pipeline Architecture and Development: Design, build, and optimize robust and scalable data pipelines to process, transform, and integrate large volumes of data from various sources into our analytics platform.
Data Quality Assurance: Implement data validation, cleansing, and enrichment techniques to ensure high-quality and consistent data across the platform.
Performance Optimization: Identify performance bottlenecks and optimize data processing and storage mechanisms to enhance overall system performance and reduce latency.
Cloud Infrastructure: Work extensively with cloud-based technologies (GCP and AWS), to design and manage scalable data infrastructure.
Collaboration: Collaborate with cross-functional teams including Data Analysts, Data Scientists, Product Managers, and Software Engineers to understand requirements and deliver solutions that meet business needs.
Data Governance: Implement and enforce data governance practices, ensuring compliance with relevant regulations and best practices related to data privacy and security.
Monitoring and Maintenance: Monitor the health and performance of data pipelines, troubleshoot issues, and ensure high availability of data infrastructure.
Mentorship: Provide technical guidance and mentorship to junior data engineers, fostering a culture of learning and growth within the team.

As a Senior Platform Engineer you will help deliver a smooth and consistent developer experience to our product/AI teams. This enables them to iterate quickly and deliver products faster to the market. The team aims to streamline development processes, reduce friction and provide developers with the right tools and infrastructure to build high-quality software efficiently which effectively aligns technical execution with our business objectives. Our techstack includes AWS, Python, RDS, EKS, ECS, RMQ and Redis.

You will:
You will have a direct impact on devs' daily smile :) , by focusing on their Devex and velocity through implementing and providing easy and intuitive self serve tools

Lead the design and implementation of infrastructure of complex, time-critical microservices systems, collaborating with AI, data, and product teams to ensure high-quality code using latest technologies while maintaining high developer velocity.

Drive improvements to backend infrastructure, focusing on scalability, reliability, and performance to enhance the daily experience and productivity of developers, including implementing intuitive self-serve tools.

Contribute to defining and realizing the technology vision of the company, evaluating and integrating new technologies to enhance platform capabilities including Gen AI, and troubleshooting complex issues to ensure platform performance and availability. Proactively establish and maintain CI/CD practices and processes including hybrid envs.

Enhance system scalability, performance, and reliability.
What success looks like:

Leading the design and implementation of infrastructure for complex microservices systems, ensuring high-quality code, and maintaining a high pace of development while collaborating effectively with other teams.

Driving improvements to the backend infrastructure, focusing on scalability, reliability, and performance to enhance the daily experience and productivity of developers.

Contributing to the company's technology vision by evaluating and integrating new technologies, including Gen AI, and troubleshooting complex issues to ensure platform performance and availability.

Working closely with development teams to support architecture best practices and enhance system scalability, performance, and reliability.

Mentoring and advising other team members in code review, performance, scalability, and more.

Demonstrating excellent communication and collaboration skills, and being a team player with a highly motivated, can-do approach.

Were looking for a Big Data Engineer to develop and integrate systems that retrieve, process and analyze data from around the digital world, generating customer-facing data. This role will report to our Team Manager, R&D.

So, what will you be doing all day?

Your role as part of the R&D team means your daily responsibilities may include:

Design and implement complex high scale systems using a large variety of technologies.
You will work in a data research team alongside other data engineers, data scientists and data analysts. Together you will tackle complex data challenges and bring new solutions and algorithms to production.
Contribute and improve the existing infrastructure of code and data pipelines, constantly exploring new technologies and eliminating bottlenecks.
You will experiment with various technologies in the domain of Machine Learning and big data processing.
You will work on a monitoring infrastructure for our data pipelines to ensure smooth and reliable data ingestion and calculation.

we are searching for the best Senior Software Engineer out there!
We are a 4 years old startup founded by successful serial entrepreneurs and a strong team with the vision to evolve digital marketing from the measurement of traffic to measuring VALUE.
Advertising for the past 100+ years has faced a strange dilemma: Half the budget I spend on Advertising is wasted. The problem is - I dont know which half (John Wanamaker, 1838-1922)
We are a deep machine learning and AI company, using the latest technologies, to solve a really simple question - which half is wasted?
our company launched in summer of 2020, backed by strategic investors from around the world who believe in what were doing.
We have customers and a constant flow of interest from the press, investors and more large companies who want to join us.
our company's innovative model, which has sparked interest within the academic community, has been acknowledged by the Israel Innovation Authority with a grant-a rare achievement in the marketing field.
Role Description:
As a Senior Software Engineer your role will be to:
Play a key role in the design and architecture of cutting-edge, innovative data systems.
Develop and maintain a multi-tenant environment for our advanced measurement solutions.
Build and manage cloud-based, distributed computing environments to support scalable, high-performance operations.
Design and implement robust ETL workflows, integrating multiple data sources for cleaning, transformation, aggregation, and querying.
Troubleshoot and resolve complex issues related to data integration, storage, and retrieval.
Architect and build big data solutions capable of managing and automating the processing of tens of terabytes.
Optimize database performance and streamline query execution in large-scale environments.
Collaborate closely with Data Scientists to align technical solutions with data-driven objectives.
You will also participate in important decisions around the product, our strategy as well as have the opportunity to choose the areas you work on based on what you want or like to do the most. We believe in helping each other get most of ourselves rather than do what were not interested in doing.

Were looking for a Senior Data Engineer to join our Research Engineering Team and play a pivotal role in shaping the data infrastructure that powers innovative solutions. In this role, youll design, build, and scale data pipelines and data warehouses from the ground up, creating the backbone for the worlds leading data classification engine. Youll collaborate closely with researchers and machine learning experts to enable the development of advanced models, ensuring a seamless flow of high-quality, reliable, and well-structured data. Your contributions will directly impact the scalability, performance, and precision of our platform, advancing our mission to protect critical data.

What Youll Do

Design and build scalable batch and streaming data pipelines that process complex datasets from diverse sources, enabling reliable and high-performance model training and inference.

Collaborate closely with researchers and data scientists to deliver high-quality, structured datasets that accelerate experimentation and model iteration.

Lead large-scale historical backfills and migration initiatives to ensure data consistency and integrity across evolving storage and compute platforms.

Optimize data workflows through advanced query tuning, indexing, partitioning, and cost optimization strategies to support efficient large-scale analytics.

Architect and maintain high-performance cloud-based data platforms using modern data stack components across AWS, GCP, or Azure.

Operate distributed data processing engines to handle massive volumes of structured and unstructured data.

Design and implement event-driven architectures using large-scale queue systems such as Kafka or SQS to ensure reliable and efficient data movement.

Develop automated monitoring and validation systems that guarantee uptime, schema compatibility, and pipeline reliability.

Deploy and manage data infrastructure in containerized, Kubernetes-based environments to support scalable and resilient services.

We're a leading force in the ad tech industry, revolutionizing how brands connect with their audiences. Our platform processes billions of ad impressions daily, generating massive datasets that drive our core business.

We thrive on innovation and seek a Senior Data Infra Engineer to help us build and scale the data infrastructure that powers our insights and analytics.
This is a unique opportunity to work with cutting-edge technologies and make a direct impact on our products.

What will you do?
As a Senior Data Infra Engineer, you'll be a key part of our data platform team, responsible for designing, building, and maintaining robust and scalable data pipelines. You'll work closely with data scientists, analysts, and server side engineers to ensure our data is reliable, accessible, and ready for analysis. Your expertise will be crucial in expanding our data warehouse and data lake capabilities, enabling us to deliver next-generation ad tech solutions.

Your mission will be to:

Develop and Optimize Data Pipelines: Design, build, and maintain ETL/ELT pipelines using Apache Spark to ingest, process, and transform large-scale datasets from various sources.
Manage Cloud Infrastructure: Architect and manage our data infrastructure primarily on Google Cloud Platform (GCP) or Amazon Web Services (AWS). This includes services like BigQuery, S3, GCS, EMR, and AirFlow.
Enhance Data Storage: Improve and manage our data warehouse and data lake solutions, ensuring data quality, consistency, and accessibility for business intelligence and machine learning applications.
Collaborate and Innovate: Partner with cross-functional teams to understand data needs and implement solutions that support new product features and business initiatives.
Ensure Data Integrity: Implement monitoring, alerting, and logging systems to maintain data pipeline health and ensure data accuracy.

Engineering - Our engineering team is at the core of our products and connected directly to the mission of preventing cyberattacks. We are constantly innovating - challenging the way we, and the industry, think about cybersecurity. Our engineers dont shy away from building products to solve problems no one has pursued before. We define the industry instead of waiting for directions. We need individuals who feel comfortable in ambiguity, excited by the prospect of a challenge, and empowered by the unknown risks facing our everyday lives that are only enabled by a secure digital environment.
Job Summary
In this role, you will be dedicated to safeguarding our clients data within the dynamic cloud landscape. As an expert in Data Security Posture Management (DSPM) for SaaS and DBaaS, you will develop cutting-edge solutions for data discovery, classification, and protection across platforms like Snowflake, Databricks and MongoDB Atlas. Your work will focus on proactively identifying and eliminating risks, ensuring our clients' sensitive information remains secure, compliant, and accessible only to authorized users, empowering them to grow their business securely.
Key Responsibilities
Drive innovation by designing and implementing impactful solutions that address client needs, contributing to the full feature development lifecycle from design to deployment.
Take ownership of specific feature segments, ensuring high-quality code and robust functionality through meticulous attention to detail and a focus on execution.
Proactively collaborate and exchange information with cross-functional teams, including product and infrastructure, to ensure seamless integration and alignment on shared objectives.
Challenge the status quo by generating innovative ideas and actively participating in brainstorming sessions to foster product and architectural improvements.
Actively engage in technical discussions, openly sharing knowledge and learning from others to solve complex problems and elevate team expertise.
Design and build highly scalable, resilient, and secure cloud-based applications and microservices.
Contribute to an agile and dynamic engineering culture, demonstrating a strong drive and outstanding communication skills to deliver results efficiently.

We seek an experienced and highly skilled Senior Engineer to join our Data Platform Team. This is an exciting opportunity to work on cutting-edge technologies, drive innovation, and play a key role in designing and implementing the companys data infrastructure and architecture.
What you'll do:
Database Infrastructure Development - Design, develop, and maintain scalable and reliable self-serve database infrastructure, including PostgreSQL, MongoDB, SingleStore, and AWS Aurora.
Big Data Infrastructure Development - Design, develop, and maintain scalable and reliable self-serve big data infrastructure, including cloud-agnostic data lakehouse, distributed databases, and platform-powered data pipelines.
End-to-End Ownership - Take responsibility for the entire lifecycle of data infrastructure, from DevOps (Terraform, Kubernetes) to application-level components (infrastructure libraries, core components in data pipelines).
Architecture Leadership: Collaborate with the Architecture group to define and maintain the companys core architectural vision.
Partner with development teams, providing technical guidance, best practices, and support from the design phase to production deployment.
Innovation and Exploration - Work on diverse and impactful projects across different layers of the tech stack, exploring new technologies and approaches to improve reliability, efficiency, and scalability.

In this role, you will be dedicated to safeguarding our clients data within the dynamic cloud landscape. As an expert in Data Security Posture Management (DSPM) for SaaS and DBaaS, you will develop cutting-edge solutions for data discovery, classification, and protection across platforms like Snowflake, Databricks and MongoDB Atlas. Your work will focus on proactively identifying and eliminating risks, ensuring our clients' sensitive information remains secure, compliant, and accessible only to authorized users, empowering them to grow their business securely.
Key Responsibilities
Drive innovation by designing and implementing impactful solutions that address client needs, contributing to the full feature development lifecycle from design to deployment.
Take ownership of specific feature segments, ensuring high-quality code and robust functionality through meticulous attention to detail and a focus on execution.
Proactively collaborate and exchange information with cross-functional teams, including product and infrastructure, to ensure seamless integration and alignment on shared objectives.
Challenge the status quo by generating innovative ideas and actively participating in brainstorming sessions to foster product and architectural improvements.
Actively engage in technical discussions, openly sharing knowledge and learning from others to solve complex problems and elevate team expertise.
Design and build highly scalable, resilient, and secure cloud-based applications and microservices.
Contribute to an agile and dynamic engineering culture, demonstrating a strong drive and outstanding communication skills to deliver results efficiently.