דרושים » דאטה » Senior Data Engineer

We are looking for an experienced and visionary Data Engineer to help design, build, and scale our BI platform.
In this role, you will be responsible for developing our data analytics platform - enabling scalable data pipelines and robust data modeling to support real-time and batch analytics to provide insights that serve both business intelligence and product needs.
You will be part of the R&D team, collaborating closely with engineers, analysts, and product managers to deliver a modern data architecture that supports internal dashboards and future-facing operational analytics.
If you enjoy turning raw data into powerful insights and owning the full data lifecycle, this role is for you!
Responsibilities:
Take full ownership of the design and implementation of a scalable and efficient BI data infrastructure, ensuring high performance, reliability, and security.
Design and architect data products, from ingestion to transformation, modeling, storage, and access.
Build and maintain ETL/ELT pipelines, batch and real-time, to support analytics, reporting, and product integrations.
Establish and enforce best practices for data quality, lineage, observability, and governance to ensure accuracy and consistency.
Integrate modern tools and frameworks such as Airflow, Databricks, Power BI, and streaming platforms.
Collaborate cross-functional with product, engineering, and analytics teams to translate business needs into data infrastructure.
Promote a data-driven culture - be an advocate for data-driven decision-making across the company by empowering stakeholders with reliable and self-service data access.

We are seeking talented and passionate Senior Data Engineer to join our Data team. In this pivotal role, you will be instrumental in designing, building, and optimizing the critical data infrastructure that underpins innovative creative intelligence platform. You will tackle complex data challenges, ensuring our systems are robust, scalable, and capable of delivering high-quality data to power our advanced AI models, customer-facing analytics, and internal business intelligence. This is an opportunity to make a significant impact on our product, contribute to a data-driven culture, and help solve fascinating problems at the intersection of data, AI, and marketing technology.

Key Responsibilities
Architect & Develop Data Pipelines: Design, implement, and maintain sophisticated, end-to-end data pipelines for ingesting, processing, validating, and transforming large-scale, diverse datasets.
Manage Data Orchestration: Implement and manage robust workflow orchestration for complex, multi-step data processes, ensuring reliability and visibility.
Advanced Data Transformation & Modeling: Develop and optimize complex data transformations using advanced SQL and other data manipulation techniques. Contribute to the design and implementation of effective data models for analytical and operational use.
Ensure Data Quality & Platform Reliability: Establish and improve processes for data quality assurance, monitoring, alerting, and performance optimization across the data platform. Proactively identify and resolve data integrity and pipeline issues.
Cross-Functional Collaboration: Partner closely with AI engineers, product managers, developers, customer success and other stakeholders to understand data needs, integrate data solutions, and deliver features that provide exceptional value.
Drive Data Platform Excellence: Contribute to the evolution of our data architecture, champion best practices in data engineering (e.g., DataOps principles), and evaluate emerging technologies to enhance platform capabilities, stability, and cost-effectiveness.
Foster a Culture of Learning & Impact: Actively share knowledge, contribute to team growth, and maintain a strong focus on how data engineering efforts translate into tangible product and business outcomes.

Required Senior Data Engineer
Description
We are making the future of Mobility come to life starting today.
We support the worlds largest vehicle fleet operators and transportation providers to optimize existing operations and seamlessly launch new, dynamic business models - driving efficient operations and maximizing utilization.
At the heart of our platform lies the data infrastructure, driving advanced machine learning models and optimization algorithms. As the owner of data pipelines, you'll tackle diverse challenges spanning optimization, prediction, modeling, inference, transportation, and mapping.
As a Senior Data Engineer, you will play a key role in owning and scaling the backend data infrastructure that powers our platform-supporting real-time optimization, advanced analytics, and machine learning applications.
What You'll Do:
Design, implement, and maintain robust, scalable data pipelines for batch and real-time processing using Spark, and other modern tools.
Own the backend data infrastructure, including ingestion, transformation, validation, and orchestration of large-scale datasets.
Leverage Google Cloud Platform (GCP) services to architect and operate scalable, secure, and cost-effective data solutions across the pipeline lifecycle.
Develop and optimize ETL/ELT workflows across multiple environments to support internal applications, analytics, and machine learning workflows.
Build and maintain data marts and data models with a focus on performance, data quality, and long-term maintainability.
Collaborate with cross-functional teams including development teams, product managers, and external stakeholders to understand and translate data requirements into scalable solutions.
Help drive architectural decisions around distributed data processing, pipeline reliability, and scalability.

As our Founding Data Engineer, you will be the first dedicated Data Engineer at the company, with the opportunity to build our data engineering function from the ground up. You'll own the architecture and foundations of our internal data platform - the central warehouse, pipelines, and models every team relies on to understand how our product is performing, and provide tools for day to day operations. You'll define how data from across the company is collected, modeled, transformed, and governed, and establish the standards and best practices that will scale with us as we grow. Working closely with Engineering, Data Science, Delivery, Product, and Analytics teams, you'll turn data into reliable, structured assets that power product and business decisions.

What Youll Do
Build our Data Engineering Function: Be the first dedicated Data Engineer and will be required to be opinionated about the warehouse architecture, tooling, standards, and best practices for how we work with data across the company.
Develop Data Pipelines: Design, build, and maintain data pipelines that transform company data into structured, queryable assets.
Enable Data-Driven Decisions: Partner with Engineering, Data Science, Product and Analytics teams to deliver trusted data foundations and actionable insights.
Create BI Dashboards: Build and maintain dashboards and reporting layers that support product, and operational decision-making.
Ensure Data Quality and Accessibility: Implement best practices for data modeling, governance, monitoring, and reliability.

We are looking for a Data Engineer to help build and scale the data platform behind our search quality, ML pipelines, product analytics, and business operations.
In this role, you will contribute across the full data lifecycle: ingesting data from production systems, designing and evolving our data warehouse, building batch and streaming pipelines, and making high-quality datasets available to researchers, engineers, analysts, and product teams across the company.
The platform spans tens of terabytes and ingests data from tens of proprietary and third-party sources - including our search engine and its components, CRM, billing, identity, and product analytics across multi-region production environments. Around 100 internal users rely on it daily.
You will work closely with engineers and stakeholders across the company, contribute to architectural and modeling decisions, and help improve the reliability, usability, and scalability of the data platform as it grows.

In this position, your responsibility will be to:
Contribute to the design, development, and operation of Tavily's data platform - from real-time ingestion through data warehouse medallion layers to consumer-facing datasets and dashboards.
Build and maintain reliable batch and streaming pipelines that ingest data from production services and external systems.
Design and evolve scalable, analytics-ready data models in the data warehouse.
Work closely with engineers across the company to ensure data produced by production systems is reliable, well-structured, and usable downstream.
Improve observability across the data platform, including data quality checks, freshness monitoring, lineage, schema evolution, and cost controls.
Partner with researchers, engineers, analysts, finance, and product managers to deliver trustworthy datasets for product, search quality, ML, and GTM analytics.
Contribute to defining the objects, entities, and relationships that represent Tavily's search domain - including agent inputs, URLs, chunks, agent sessions, crawls, and the connections between them - and translate them into clean, queryable data models.
Improve engineering practices around testing, documentation, deployment, and incident response.
Investigate and resolve production data issues, including broken pipelines, corrupted datasets, schema changes, and large-scale backfills.
Contribute to technical standards and best practices for data engineering across the company.
Help maintain high standards of data quality, integrity, security, and governance across environments.

a central part of the R&D department is seeking a versatile Senior Data Engineer to take part in leading data vision using a lake-house architecture, and building high-scale solutions for handling and consuming the different data assets of the company.
In this role youll go beyond traditional boundaries, taking on full product responsibilities - from conceptualization and architecture to design, maintenance, and continuous development. You will collaborate closely with stakeholders across the company to ensure our solutions effectively meet their needs.
The ideal candidate will have strong data engineering capabilities, along with software development skills including understanding of designing and managing AWS cloud infrastructure and strong knowledge of different data architectures, methods and tools.
Responsibilities:
Design and development of scalable, reliable, and secure data infrastructure and build ETL pipelines that handle diverse clinical data for research. Write production SQL, Spark jobs and craft schemas that evolve gracefully as research and production questions change.
Automate releases with CI/CD and Infrastructure as Code.
Optimise throughput, latency and cloud cost to meet research timelines at large scale.
Develop and maintain high-quality, scalable, and efficient code while fostering a culture of continuous improvement through regular retrospectives and knowledge sharing.
Take ownership of the full development lifecycle, including requirement gathering, design, implementation, testing, deployment, and ongoing maintenance.
Collaborate with cross-functional teams, including data scientists, data analysts, product managers, regulatory teams, and other developers, to drive the development of new tools and features that support mission.
Explore and adopt new technologies and frameworks that can enhance the capabilities of the Data-Nexus team and the overall data projects .

Design, build, and maintain robust pipelines feeding both DWH and production processes
Own pipeline orchestration in Apache Airflow end to end, including DAG design, monitoring, alerting, and production reliability
Design the data models behind our core domains and the contracts that downstream consumers depend on
Partner with business analysts, PMs, and business stakeholders to translate product and business questions into stable, qualified, and well-modeled data
Work with a variety of data sources including product events, application data, APIs, operational databases, logs, and third-party systems
Debug and solve data quality, pipeline, performance, and reliability issues
Use and create AI agents and LLM-based tooling to improve development, testing, documentation, and root cause analysis
Be part of the team and adopt effective ways of using AI tools in the engineering workflow.

We are a well-funded, early-stage startup looking for a talented and motivated Backend Engineer to join our founding team. The focus of this role is to design, develop, and deploy autonomous AI agents that can automate and optimize complex enterprise workflows. You will work on building intelligent systems capable of decision-making, data extraction, document processing, and other tasks typically requiring human intervention. This is an opportunity to influence the architecture and strategy of AI-driven automation solutions for large-scale enterprise environments.

Your Impact
AI Agent Development

Design and develop autonomous AI agents using best of breed large language models to automate tasks such as document processing, data extraction, and workflow management.

Implement reinforcement learning techniques to enhance decision-making capabilities of AI agents.

AI Integration

Build and integrate APIs that connect AI agents with external systems and enterprise software (ERP, CRM).

Use frameworks like TensorFlow, PyTorch, or Hugging Face to deploy and optimize AI models for real-time processing.

Data Processing and Pipelines

Design and manage data pipelines to process and analyze large volumes of documents and unstructured data efficiently.

Optimize data handling to improve speed and accuracy of AI agents.

Security and Compliance

Implement authentication and authorization mechanisms to secure AI-driven systems.

Ensure compliance with data privacy standards (e.g., GDPR, HIPAA) and adopt best practices for secure data handling.

Cloud Infrastructure and Scalability

Deploy AI agents on cloud platforms (AWS, GCP, or Azure) ensuring scalability and reliability.

Leverage containerization (Docker, Kubernetes) for efficient deployment and management.

Testing and Optimization

Develop and execute unit, integration, and performance tests for AI-driven systems.

Continuously monitor and optimize system performance for speed, accuracy, and cost-efficiency.

Collaboration

Work closely with AI researchers, front end engineers, and product teams to align AI agent capabilities with business requirements.

Participate in code reviews, design discussions, and architecture planning to drive innovation.

As a Senior Data Engineer, youll collaborate with top-notch engineers and data scientists to elevate our platform to the next level and deliver exceptional user experiences. Your primary focus will be on the data engineering aspects-ensuring the seamless flow of high-quality, relevant data to train and optimize content models, including GenAI foundation models, supervised fine-tuning, and more.

Youll work closely with teams across the company to ensure the availability of high-quality data from ML platforms, powering decisions across all departments. With access to petabytes of data through MySQL, Snowflake, Cassandra, S3, and other platforms, your challenge will be to ensure that this data is applied even more effectively to support business decisions, train and monitor ML models and improve our products.



Key Job Responsibilities and Duties:

Rapidly developing next-generation scalable, flexible, and high-performance data pipelines.

Dealing with massive textual sources to train GenAI foundation models.

Solving issues with data and data pipelines, prioritizing based on customer impact.

End-to-end ownership of data quality in our core datasets and data pipelines.

Experimenting with new tools and technologies to meet business requirements regarding performance, scaling, and data quality.

Providing tools that improve Data Quality company-wide, specifically for ML scientists.

Providing self-organizing tools that help the analytics community discover data, assess quality, explore usage, and find peers with relevant expertise.

Acting as an intermediary for problems, with both technical and non-technical audiences.

Promote and drive impactful and innovative engineering solutions

Technical, behavioral and interpersonal competence advancement via on-the-job opportunities, experimental projects, hackathons, conferences, and active community participation

Collaborate with multidisciplinary teams: Collaborate with product managers, data scientists, and analysts to understand business requirements and translate them into machine learning solutions. Provide technical guidance and mentorship to junior team members.

We are seeking an experienced Solutions Data Engineer who possess both technical depth and strong interpersonal skills to partner with internal and external teams to develop scalable, flexible, and cutting-edge solutions. Solutions Engineers collaborate with operations and business development to help craft solutions to meet customer business problems.
A Solutions Engineer works to balance various aspects of the project, from safety to design. Additionally, a Solutions Engineer researches advanced technology regarding best practices in the field and seek to find cost-effective solutions.
Were looking for a Solutions Engineer with deep experience in Big Data technologies, real-time data pipelines, and scalable infrastructure-someone whos been delivering critical systems under pressure, and knows what it takes to bring complex data architectures to life. This isnt just about checking boxes on tech stacks-its about solving real-world data problems, collaborating with smart people, and building robust, future-proof solutions.
In this role, youll partner closely with engineering, product, and customers to design and deliver high-impact systems that move, transform, and serve data at scale. Youll help customers architect pipelines that are not only performant and cost-efficient but also easy to operate and evolve.
We want someone whos comfortable switching hats between low-level debugging, high-level architecture, and communicating clearly with stakeholders of all technical levels.
Key Responsibilities:
Build distributed data pipelines using technologies like Kafka, Spark (batch & streaming), Python, Trino, Airflow, and S3-compatible data lakes-designed for scale, modularity, and seamless integration across real-time and batch workloads.
Design, deploy, and troubleshoot hybrid cloud/on-prem environments using Terraform, Docker, Kubernetes, and CI/CD automation tools.
Implement event-driven and serverless workflows with precise control over latency, throughput, and fault tolerance trade-offs.
Create technical guides, architecture docs, and demo pipelines to support onboarding, evangelize best practices, and accelerate adoption across engineering, product, and customer-facing teams.
Integrate data validation, observability tools, and governance directly into the pipeline lifecycle.
Own end-to-end platform lifecycle: ingestion → transformation → storage (Parquet/ORC on S3) → compute layer (Trino/Spark).
Benchmark and tune storage backends (S3/NFS/SMB) and compute layers for throughput, latency, and scalability using production datasets.
Work cross-functionally with R&D to push performance limits across interactive, streaming, and ML-ready analytics workloads.
Operate and debug object store-backed data lake infrastructure, enabling schema-on-read access, high-throughput ingestion, advanced searching strategies, and performance tuning for large-scale workloads.