דרושים » הנדסה » Senior Data Engineer

We are looking for a Data Engineer to help build and scale the data platform behind our search quality, ML pipelines, product analytics, and business operations.
In this role, you will contribute across the full data lifecycle: ingesting data from production systems, designing and evolving our data warehouse, building batch and streaming pipelines, and making high-quality datasets available to researchers, engineers, analysts, and product teams across the company.
The platform spans tens of terabytes and ingests data from tens of proprietary and third-party sources - including our search engine and its components, CRM, billing, identity, and product analytics across multi-region production environments. Around 100 internal users rely on it daily.
You will work closely with engineers and stakeholders across the company, contribute to architectural and modeling decisions, and help improve the reliability, usability, and scalability of the data platform as it grows.

In this position, your responsibility will be to:
Contribute to the design, development, and operation of Tavily's data platform - from real-time ingestion through data warehouse medallion layers to consumer-facing datasets and dashboards.
Build and maintain reliable batch and streaming pipelines that ingest data from production services and external systems.
Design and evolve scalable, analytics-ready data models in the data warehouse.
Work closely with engineers across the company to ensure data produced by production systems is reliable, well-structured, and usable downstream.
Improve observability across the data platform, including data quality checks, freshness monitoring, lineage, schema evolution, and cost controls.
Partner with researchers, engineers, analysts, finance, and product managers to deliver trustworthy datasets for product, search quality, ML, and GTM analytics.
Contribute to defining the objects, entities, and relationships that represent Tavily's search domain - including agent inputs, URLs, chunks, agent sessions, crawls, and the connections between them - and translate them into clean, queryable data models.
Improve engineering practices around testing, documentation, deployment, and incident response.
Investigate and resolve production data issues, including broken pipelines, corrupted datasets, schema changes, and large-scale backfills.
Contribute to technical standards and best practices for data engineering across the company.
Help maintain high standards of data quality, integrity, security, and governance across environments.

We are looking for a Backend Engineer with deep data engineering expertise to build and evolve the systems that power product, analytics, experimentation, and AI development.
You will work across production backend services and data infrastructure, taking technical ownership of the full data lifecycle - from event ingestion and real-time processing to orchestration, modeling, quality, and reliable data access. This role combines hands-on backend development with ownership of shared data platform, working closely with backend, frontend, AI, DevOps, analytics, and product teams.
What you'll do:
Build and maintain production-grade Python and Java backend services, APIs, and reactive data-processing pipelines with clear interfaces, resilient failure handling, and comprehensive observability.
Own and evolve batch and streaming data pipelines supporting product analytics, learner insights, experimentation, and AI model training.
Design and maintain trusted data models, shared metrics, and semantic-layer business logic across the analytical data platform.
Define and enforce standards for data contracts, testing, lineage, freshness, and observability.
Develop reliable approaches to schema evolution, backfills, replay, workload distribution, and failure recovery.
Collaborate with DevOps, backend, frontend, AI, analytics, and product teams to deliver reusable data capabilities and maintain clear system boundaries.

We're looking for an experienced Software Engineer to join our Data Platform Group. In this key role, you will build the company data platform: cloud-based microservices and data pipelines that process on the order of 1M records/sec at low latency. Because the platform is the foundation other groups build on, your work has a direct impact on our customers and enables engineering, product, and research teams across the organization.
The Lakehouse team owns the data itself - how it is stored, organized, retained, and served. We own our analytical storage layer, the batch processing built on top of it, and the APIs through which customers and the rest of the company consume data. If you enjoy the problems that only appear at petabyte scale - physical data layout, query performance, storage cost, and retention - this is the role.

Responsibilities:
End-to-end ownership of our large-scale analytical storage layer: data modeling, schema and table design, partitioning, retention, and query performance.
Design and develop the batch processing layer over our data lake using Spark on EMR (Java and PySpark).
Build and evolve Java/Spring Boot services that expose our data through well-defined APIs to customers and to consumers across the company.
Own performance and cost: query optimization, file layout and compaction, cluster sizing, and storage efficiency at scale.
Research new technologies in the lakehouse and analytical-storage space and adapt them for use in our product.
Work closely with product, DevOps, and security teams.

Required Senior Data Engineer
Description
We are making the future of Mobility come to life starting today.
We support the worlds largest vehicle fleet operators and transportation providers to optimize existing operations and seamlessly launch new, dynamic business models - driving efficient operations and maximizing utilization.
At the heart of our platform lies the data infrastructure, driving advanced machine learning models and optimization algorithms. As the owner of data pipelines, you'll tackle diverse challenges spanning optimization, prediction, modeling, inference, transportation, and mapping.
As a Senior Data Engineer, you will play a key role in owning and scaling the backend data infrastructure that powers our platform-supporting real-time optimization, advanced analytics, and machine learning applications.
What You'll Do:
Design, implement, and maintain robust, scalable data pipelines for batch and real-time processing using Spark, and other modern tools.
Own the backend data infrastructure, including ingestion, transformation, validation, and orchestration of large-scale datasets.
Leverage Google Cloud Platform (GCP) services to architect and operate scalable, secure, and cost-effective data solutions across the pipeline lifecycle.
Develop and optimize ETL/ELT workflows across multiple environments to support internal applications, analytics, and machine learning workflows.
Build and maintain data marts and data models with a focus on performance, data quality, and long-term maintainability.
Collaborate with cross-functional teams including development teams, product managers, and external stakeholders to understand and translate data requirements into scalable solutions.
Help drive architectural decisions around distributed data processing, pipeline reliability, and scalability.

We are looking for a highly skilled Senior Data Platform Engineer to join our Data Platform team and play a key role in designing, building, and evolving the companys data infrastructure.

This is a hands-on engineering position focused on developing scalable data platforms, high-volume ingestion pipelines, and the foundational data systems that power analytics, product applications, AI workloads, and large-scale model execution. You will help architect our data lake, build reliable ingestion capabilities at scale, and shape the core infrastructure that supports the companys future growth.

This is an exciting opportunity to work with cutting-edge technologies and have a defining impact on the architecture and evolution of our data platform.


What Youll Do
Build complex, high-volume batch and near-real-time ingestion services while helping evolve the platforms streaming capabilities.
Design and implement data solutions for all application requirements in a distributed microservices environment
Develop and optimize large-scale data pipelines for batch and streaming use cases.
Ensure data quality, compliance, and governance.
Manage and optimize Kubernetes infrastructure, utilizing KEDA to dynamically scale resources for large-scale data processing workloads.
Own the full lifecycle of our data stack using Terraform and Kubernetes, from provisioning to application-level frameworks.
Provide technical guidance for the team, drive best practices around infrastructure, CI/CD, testing, and system design.
Deep-dive into memory management and performance tuning to ensure our large-scale distributed systems are both cost-effective and lightning-fast.

Were looking for a Core Full-Stack Engineer who combines strong product instincts, deep engineering skills, and the ability to move fast without compromising on execution quality.
Youll be a primary technical driver, taking significant product modules from concept through design, architecture, implementation, and production. You wont just implement finished specifications, youll partner with product and design to shape the solution, make technical decisions, and own delivery across the entire stack.
Your work will turn complex, large-scale data into fast, intuitive products for Fortune 500 marketing teams. Youll build across the entire data and application stack-from data ingestion and ETL pipelines to scalable backend services, polished interfaces, and responsive visualizations. This role combines full-stack development with hands-on data engineering, owning how data is collected, transformed, stored, and served to make exploring datasets with billions of records feel effortless.
This is a hands-on role for someone who has driven meaningful product development in a fast-growing startup and is comfortable taking ownership before every requirement has been defined.
Responsibilities
Own complete product modules from concept to production.
Build and own production data pipelines.
Design scalable data models and APIs.
Build fast, polished, data-intensive interfaces.
Drive performance across the entire stack.
Ensure data quality and reliability.
Create reusable product foundations.
Own production reliability and safe delivery.
Lead through hands-on execution.

We're hiring a Data Engineer to architect and own the systems that turn raw signal from merchants, customers, and experiments into something the whole company can build on.
Data is at the heart of everything we do - it powers our AI models, drives merchant-facing analytics, and informs how we make product decisions every day.
This is a high-ownership role for someone who wants to shape data architecture from the ground up- not inherit someone else's roadmap. You'll be the person who makes sure our data is fast, trustworthy, and easy for every team to use.
What You'll Do:
Architect and build scalable data pipelines processing millions of events daily from merchant stores, product usage, experiments, and third-party integrations
Own our data warehouse strategy end-to-end- reliability, performance, and scalability as we grow
Define our event architecture and set the standards for how the company captures and models product and business data
Build data quality practices- monitoring, validation, testing, and alerting- that the rest of the team can trust
Partner directly with Product and Engineering to unlock the data behind our AI features, merchant analytics, and experimentation
Build self-service tooling so other teams can query and analyze data without depending on you as a bottleneck
Set the bar for data modeling, orchestration, and documentation across the org

We are seeking an experienced Solutions Data Engineer who possess both technical depth and strong interpersonal skills to partner with internal and external teams to develop scalable, flexible, and cutting-edge solutions. Solutions Engineers collaborate with operations and business development to help craft solutions to meet customer business problems.
A Solutions Engineer works to balance various aspects of the project, from safety to design. Additionally, a Solutions Engineer researches advanced technology regarding best practices in the field and seek to find cost-effective solutions.
Job Description:
Were looking for a Solutions Engineer with deep experience in Big Data technologies, real-time data pipelines, and scalable infrastructure-someone whos been delivering critical systems under pressure, and knows what it takes to bring complex data architectures to life. This isnt just about checking boxes on tech stacks-its about solving real-world data problems, collaborating with smart people, and building robust, future-proof solutions.
In this role, youll partner closely with engineering, product, and customers to design and deliver high-impact systems that move, transform, and serve data at scale. Youll help customers architect pipelines that are not only performant and cost-efficient but also easy to operate and evolve.
We want someone whos comfortable switching hats between low-level debugging, high-level architecture, and communicating clearly with stakeholders of all technical levels.
Key Responsibilities:
Build distributed data pipelines using technologies like Kafka, Spark (batch & streaming), Python, Trino, Airflow, and S3-compatible data lakes-designed for scale, modularity, and seamless integration across real-time and batch workloads.
Design, deploy, and troubleshoot hybrid cloud/on-prem environments using Terraform, Docker, Kubernetes, and CI/CD automation tools.
Implement event-driven and serverless workflows with precise control over latency, throughput, and fault tolerance trade-offs.
Create technical guides, architecture docs, and demo pipelines to support onboarding, evangelize best practices, and accelerate adoption across engineering, product, and customer-facing teams.
Integrate data validation, observability tools, and governance directly into the pipeline lifecycle.
Own end-to-end platform lifecycle: ingestion → transformation → storage (Parquet/ORC on S3) → compute layer (Trino/Spark).
Benchmark and tune storage backends (S3/NFS/SMB) and compute layers for throughput, latency, and scalability using production datasets.
Work cross-functionally with R&D to push performance limits across interactive, streaming, and ML-ready analytics workloads.
Operate and debug object store-backed data lake infrastructure, enabling schema-on-read access, high-throughput ingestion, advanced searching strategies, and performance tuning for large-scale workloads.

Were looking for a senior engineer who is ready to move beyond traditional software development and take full ownership of product delivery.

This is not a classic backend engineering role. You will work directly with Product to turn ideas and requirements into production-ready solutions, make the technical decisions, and use AI coding agents as a core part of your engineering workflow.

You will own the outcome, not just the code. AI-generated code is your code, and you are accountable for its quality, security, performance, and maintainability.


Own product features end-to-end, from requirements through production and monitoring.
Work with Product to translate business problems into clear technical solutions and execution plans.
Design scalable and maintainable solutions, including architecture, APIs, data models, and integrations.
Use AI coding agents to accelerate implementation, testing, debugging, and automation.
Review and improve AI-generated code, ensuring high standards for correctness, security, performance, and maintainability.
Make independent technical decisions and communicate architectural trade-offs clearly.
Contribute across backend, frontend, QA, CI/CD, and infrastructure when needed to deliver a feature.
Investigate and solve complex production and performance issues.
Continuously improve how we build, test, and ship software using AI-native development practices.

We are looking for a Senior II Software Engineer to join the CX Platform team, the foundational engineering team powering our entire Customer Experience group. This is a high-impact, hands-on role at the intersection of backend engineering, AI infrastructure, and customer-facing product.

You'll work across the full platform stack (backend services, data pipelines, security, cost, and scale) with a meaningful and growing focus on AI infrastructure. We own the agentic platform for the entire Product Offering group: from building the LLM infrastructure and agentic workflows to ensuring they're reliable, observable, and safe in production.

What you'll be doing:
Own AI infrastructure for the Product Offering group. Design, build, and evolve the shared AI platform (agentic workflows, LLM integrations, observability, and guardrails) that CX product teams build on.
Ship agentic features end to end. Lead development of AI-driven capabilities using LangChain, LangFuse, and AWS Bedrock, from architecture through production deployment and monitoring.
Drive platform architecture. Set the technical direction for the CX backend (services, data pipelines, API patterns) with an eye for scalability, reliability, and developer experience.
Own core data foundations. Design resilient data-access patterns across Snowflake, Elasticsearch, Kafka, Redis, and MySQL; keep pipelines fast, fresh, and reliable.
Mentor and elevate. Help engineers across the CX group grow in backend craft, AI engineering, and system design thinking.
Collaborate cross-functionally. Work with product, design, and customer-facing teams to turn ambiguous problems into well-scoped, high-quality solutions.