דרושים » תוכנה » Data Engineer

We are looking for an experienced Data Engineer to own and evolve data infrastructure and analytics platform.

You will design and build scalable data pipelines and data models, working with large and diverse datasets generated across Trigos global retail deployments. You will own the data lifecycle from ingestion and transformation through orchestration, reliability, and analytics-ready datasets.

As a key technical owner of data platform, you will also work closely with analysts, Product, R&D, Operations, and Customer Success to build trusted data products, evolve our BI semantic layer, and turn complex data into scalable solutions

A day in the life
Design, build, and maintain scalable ETL/ELT pipelines processing large and diverse datasets.
Own and evolve data architecture, from ingestion and transformation through analytics-ready data models and consumption.
Build reliable data workflows and orchestration processes, with a focus on scalability, performance, and maintainability.
Design and maintain curated data models that power analytics, operational workflows, and data-driven products.
Ensure data quality, observability, monitoring, and reliability across datasets and pipelines.
Develop and evolve the BI semantic layer, enabling trusted metrics and self-service analytics across the company.
Work closely with R&D and Product on data integrations and new data-driven capabilities, including use cases that bridge offline analytics and production systems.
Partner with analysts and business stakeholders to translate complex requirements into scalable data solutions.

We are seeking an experienced Solutions Data Engineer who possess both technical depth and strong interpersonal skills to partner with internal and external teams to develop scalable, flexible, and cutting-edge solutions. Solutions Engineers collaborate with operations and business development to help craft solutions to meet customer business problems.
A Solutions Engineer works to balance various aspects of the project, from safety to design. Additionally, a Solutions Engineer researches advanced technology regarding best practices in the field and seek to find cost-effective solutions.
Job Description:
Were looking for a Solutions Engineer with deep experience in Big Data technologies, real-time data pipelines, and scalable infrastructure-someone whos been delivering critical systems under pressure, and knows what it takes to bring complex data architectures to life. This isnt just about checking boxes on tech stacks-its about solving real-world data problems, collaborating with smart people, and building robust, future-proof solutions.
In this role, youll partner closely with engineering, product, and customers to design and deliver high-impact systems that move, transform, and serve data at scale. Youll help customers architect pipelines that are not only performant and cost-efficient but also easy to operate and evolve.
We want someone whos comfortable switching hats between low-level debugging, high-level architecture, and communicating clearly with stakeholders of all technical levels.
Key Responsibilities:
Build distributed data pipelines using technologies like Kafka, Spark (batch & streaming), Python, Trino, Airflow, and S3-compatible data lakes-designed for scale, modularity, and seamless integration across real-time and batch workloads.
Design, deploy, and troubleshoot hybrid cloud/on-prem environments using Terraform, Docker, Kubernetes, and CI/CD automation tools.
Implement event-driven and serverless workflows with precise control over latency, throughput, and fault tolerance trade-offs.
Create technical guides, architecture docs, and demo pipelines to support onboarding, evangelize best practices, and accelerate adoption across engineering, product, and customer-facing teams.
Integrate data validation, observability tools, and governance directly into the pipeline lifecycle.
Own end-to-end platform lifecycle: ingestion → transformation → storage (Parquet/ORC on S3) → compute layer (Trino/Spark).
Benchmark and tune storage backends (S3/NFS/SMB) and compute layers for throughput, latency, and scalability using production datasets.
Work cross-functionally with R&D to push performance limits across interactive, streaming, and ML-ready analytics workloads.
Operate and debug object store-backed data lake infrastructure, enabling schema-on-read access, high-throughput ingestion, advanced searching strategies, and performance tuning for large-scale workloads.

We are looking for a Backend Engineer with deep data engineering expertise to build and evolve the systems that power product, analytics, experimentation, and AI development.
You will work across production backend services and data infrastructure, taking technical ownership of the full data lifecycle - from event ingestion and real-time processing to orchestration, modeling, quality, and reliable data access. This role combines hands-on backend development with ownership of shared data platform, working closely with backend, frontend, AI, DevOps, analytics, and product teams.
What you'll do:
Build and maintain production-grade Python and Java backend services, APIs, and reactive data-processing pipelines with clear interfaces, resilient failure handling, and comprehensive observability.
Own and evolve batch and streaming data pipelines supporting product analytics, learner insights, experimentation, and AI model training.
Design and maintain trusted data models, shared metrics, and semantic-layer business logic across the analytical data platform.
Define and enforce standards for data contracts, testing, lineage, freshness, and observability.
Develop reliable approaches to schema evolution, backfills, replay, workload distribution, and failure recovery.
Collaborate with DevOps, backend, frontend, AI, analytics, and product teams to deliver reusable data capabilities and maintain clear system boundaries.

We are looking for a Data Engineer to join our team and play a key role in designing, building, and maintaining scalable, cloud-based data pipelines. You will work with AWS services , Airflow and databricks to integrate, process, and analyze large datasets, ensuring data reliability and efficiency.
Your work will directly impact business intelligence, analytics, and data-driven decision-making across the company.
What Youll Do:
ETL & Data Processing: Develop and maintain ETL processes, integrating data from various sources (APIs, databases, external platforms) using Python, SQL, and cloud technologies.
Utilize LangGraph and other frameworks to create state of the art AI agents that integrate various tools and data sources.
Implement integrations that allow agents to take automated actions on behalf of users, streamlining workflows and reducing overhead.
Data Modeling: Design and maintain logical and physical data models to support business needs.
Optimization & Scalability: Improve process efficiency and optimize runtime performance to handle large-scale data workloads.
Collaboration: Work closely with BI analysts and business stakeholders to define data requirements and functional specifications.
Monitoring & Troubleshooting: Ensure data integrity and reliability by proactively monitoring pipelines and resolving issues.

As a Senior Software Engineer your role will be to:

Play a key role in the design and architecture of cutting-edge, innovative data systems.
Develop and maintain a multi-tenant environment for our advanced measurement solutions.
Build and manage cloud-based, distributed computing environments to support scalable, high-performance operations.
Design and implement robust ETL workflows, integrating multiple data sources for cleaning, transformation, aggregation, and querying.
Troubleshoot and resolve complex issues related to data integration, storage, and retrieval.
Architect and build big data solutions capable of managing and automating the processing of tens of terabytes.
Optimize database performance and streamline query execution in large-scale environments.
Collaborate closely with Data Scientists to align technical solutions with data-driven objectives.

We're looking for an experienced Software Engineer to join our Data Platform Group. In this key role, you will build the company data platform: cloud-based microservices and data pipelines that process on the order of 1M records/sec at low latency. Because the platform is the foundation other groups build on, your work has a direct impact on our customers and enables engineering, product, and research teams across the organization.
The Lakehouse team owns the data itself - how it is stored, organized, retained, and served. We own our analytical storage layer, the batch processing built on top of it, and the APIs through which customers and the rest of the company consume data. If you enjoy the problems that only appear at petabyte scale - physical data layout, query performance, storage cost, and retention - this is the role.

Responsibilities:
End-to-end ownership of our large-scale analytical storage layer: data modeling, schema and table design, partitioning, retention, and query performance.
Design and develop the batch processing layer over our data lake using Spark on EMR (Java and PySpark).
Build and evolve Java/Spring Boot services that expose our data through well-defined APIs to customers and to consumers across the company.
Own performance and cost: query optimization, file layout and compaction, cluster sizing, and storage efficiency at scale.
Research new technologies in the lakehouse and analytical-storage space and adapt them for use in our product.
Work closely with product, DevOps, and security teams.

We're seeking an experienced Software Engineer with a strong background to become an integral member of our Data-Core team, tasked with the mission of processing, structuring, and analyzing hundreds of millions of data sources. Your role will be pivotal in creating a unified, up-to-date, and accurate utilities map, services, and applications for accelerating our mapping operations. Your contributions will directly impact our core product's success.





Responsibilities



Collaborate with cross-functional teams to design, build, and maintain data processing pipelines while contributing to our common codebase.
Contribute to designing and implementing data architecture, ensuring effective data storage and retrieval.
Develop and optimize complex Python-based applications and services to allow more efficient data processing and orchestration, enhancing the quality and accuracy of our datasets.
Drive the scalability and performance optimization of data systems, addressing infrastructure challenges as data volume and complexity grow.
Create and manage data infrastructure components, including ETL workflows, databases, supporting seamless data flow and accessibility.
Design and implement CI/CD processes for data processing, model training, releasing, testing and monitoring, ensuring robustness and consistency.
Implement geospatial data processing techniques and contribute to the creation of our unified utilities map, enhancing the product's geospatial features.

We are seeking talented and passionate Senior Data Engineer to join our Data team. In this pivotal role, you will be instrumental in designing, building, and optimizing the critical data infrastructure that underpins innovative creative intelligence platform. You will tackle complex data challenges, ensuring our systems are robust, scalable, and capable of delivering high-quality data to power our advanced AI models, customer-facing analytics, and internal business intelligence. This is an opportunity to make a significant impact on our product, contribute to a data-driven culture, and help solve fascinating problems at the intersection of data, AI, and marketing technology.

Key Responsibilities
Architect & Develop Data Pipelines: Design, implement, and maintain sophisticated, end-to-end data pipelines for ingesting, processing, validating, and transforming large-scale, diverse datasets.
Manage Data Orchestration: Implement and manage robust workflow orchestration for complex, multi-step data processes, ensuring reliability and visibility.
Advanced Data Transformation & Modeling: Develop and optimize complex data transformations using advanced SQL and other data manipulation techniques. Contribute to the design and implementation of effective data models for analytical and operational use.
Ensure Data Quality & Platform Reliability: Establish and improve processes for data quality assurance, monitoring, alerting, and performance optimization across the data platform. Proactively identify and resolve data integrity and pipeline issues.
Cross-Functional Collaboration: Partner closely with AI engineers, product managers, developers, customer success and other stakeholders to understand data needs, integrate data solutions, and deliver features that provide exceptional value.
Drive Data Platform Excellence: Contribute to the evolution of our data architecture, champion best practices in data engineering (e.g., DataOps principles), and evaluate emerging technologies to enhance platform capabilities, stability, and cost-effectiveness.
Foster a Culture of Learning & Impact: Actively share knowledge, contribute to team growth, and maintain a strong focus on how data engineering efforts translate into tangible product and business outcomes.

We are looking for a Sr. Staff Backend Engineer to join our team. This is a Hybrid (Ramat Gan) role, reporting to the Manager, Software Engineering in the Exposure Management & Security Operations department. Avalor, now part of Zscaler, powers the data foundation behind Security Operations platform, ingesting, unifying, and contextualizing data from hundreds of security and business tools to power security modules that deliver actionable insights and measurable risk reduction to our customers. In this role, you'll own complex backend systems that sit at the heart of our data platform, designing and building services that ingest, process, and move massive amounts of data reliably and at scale for enterprise customers worldwide.

What youll do (Role Expectations)

Developing and maintaining high-quality, scalable backend services within a microservices architecture using Java/Spring Boot and Python
Designing and building data pipelines and ETL workflows that ingest, transform, and unify data from diverse sources at enterprise scale
Leading solutions from scratch to production; creating technical designs, driving architectural decisions and taking full ownership of complex projects - from design and implementation through deployment, monitoring performance, reliability and cost, and proactively driving improvements
Collaborating closely with cross-functional partners - product managers, frontend/backend engineers and UX designers
Serving as a technical anchor for the team - leading design discussions, setting engineering standards, and driving best practices across the codebase
Who You Are (Success Profile)

You thrive in ambiguity. You're comfortable building the path as you walk it. You thrive in a dynamic environment, seeing ambiguity not as a hindrance, but as the raw material to build something meaningful.
You act like an owner. Your passion for the mission fuels your bias for action. You operate with integrity because you genuinely care about the outcome. True ownership involves leveraging dynamic range: the ability to navigate seamlessly between high-level strategy and hands-on execution.
You are a problem-solver. You love running towards the challenges because you are laser-focused on finding the solution, knowing that solving the hard problems delivers the biggest impact.
You are a high-trust collaborator. You are ambitious for the team, not just yourself. You embrace our challenge culture by giving and receiving ongoing feedback-knowing that candor delivered with clarity and respect is the truest form of teamwork and the fastest way to earn trust.
You are a learner. You have a true growth mindset and are obsessed with your own development, actively seeking feedback to become a better partner and a stronger teammate. You love what you do and you do it with purpose.

As a Senior Software Engineer, youll play a key role in building and scaling the data infrastructure that powers our platform. Youll work on complex, large-scale data systems that collect, process, and transform massive volumes of cloud and SaaS usage data into reliable, actionable insights for our customers.

Youll take ownership of core data pipelines and backend services, tackle challenging problems in distributed systems, and drive improvements in performance, reliability, and scalability. Working closely with engineering teams and other business units, youll contribute to architectural decisions and help shape the next generation of our data platform.

This is a high-impact role for an engineer who enjoys solving complex data engineering challenges, working with modern cloud and big data technologies, and building resilient systems that operate at scale.

Responsibilities:

Own, enhance, and improve our core platform.

Build and maintain systems for collecting and processing metrics from client environments.

Focus on system efficiency and resilient infrastructure.

Integrate well-known SaaS platforms into our big data repository (e.g., major cloud providers, Datadog, Snowflake).

Optimize Spark and Airflow processes.

Contribute to data design and platform architecture while working closely with other business units and engineering teams.

Face the challenges of testing and monitoring large-scale data pipelines.