דרושים » ניהול ביניים » Principal Data Engineer (Cortex)

Required Principal Data Engineer (Cortex)
Job Summary
As a Principal Data Engineer, you will be integral to the security research group, designing, building, and maintaining the data pipelines and data frameworks that are the backbone of our cyber attack detection capabilities. You will work closely with top security and AI researchers, enabling their work and seeing the direct impact of your contributions on our services and the day-to-day effectiveness of the research team.
Key Responsibilities:
Lead the design and implementation of end-to-end, complex data pipelines and data platforms in cloud environment.
Collect, analyze, and translate complex data requirements from various stakeholders including data analysts, developers, and Cyber security researchers.
Develop robust, scalable Cloud based data solutions using Python and distributed processing frameworks such as Apache Beam (Dataflow) , GKE , Spark , etc.
Utilize AI as part of the development cycle, including generating, reviewing, and testing code via prompts.
Model and manage data within cloud MPP databases and data lakes such as BigQuery, Redshift, or Snowflake.

We are looking for a Data Engineer to help build and scale the data platform behind our search quality, ML pipelines, product analytics, and business operations.
In this role, you will contribute across the full data lifecycle: ingesting data from production systems, designing and evolving our data warehouse, building batch and streaming pipelines, and making high-quality datasets available to researchers, engineers, analysts, and product teams across the company.
The platform spans tens of terabytes and ingests data from tens of proprietary and third-party sources - including our search engine and its components, CRM, billing, identity, and product analytics across multi-region production environments. Around 100 internal users rely on it daily.
You will work closely with engineers and stakeholders across the company, contribute to architectural and modeling decisions, and help improve the reliability, usability, and scalability of the data platform as it grows.

In this position, your responsibility will be to:
Contribute to the design, development, and operation of Tavily's data platform - from real-time ingestion through data warehouse medallion layers to consumer-facing datasets and dashboards.
Build and maintain reliable batch and streaming pipelines that ingest data from production services and external systems.
Design and evolve scalable, analytics-ready data models in the data warehouse.
Work closely with engineers across the company to ensure data produced by production systems is reliable, well-structured, and usable downstream.
Improve observability across the data platform, including data quality checks, freshness monitoring, lineage, schema evolution, and cost controls.
Partner with researchers, engineers, analysts, finance, and product managers to deliver trustworthy datasets for product, search quality, ML, and GTM analytics.
Contribute to defining the objects, entities, and relationships that represent Tavily's search domain - including agent inputs, URLs, chunks, agent sessions, crawls, and the connections between them - and translate them into clean, queryable data models.
Improve engineering practices around testing, documentation, deployment, and incident response.
Investigate and resolve production data issues, including broken pipelines, corrupted datasets, schema changes, and large-scale backfills.
Contribute to technical standards and best practices for data engineering across the company.
Help maintain high standards of data quality, integrity, security, and governance across environments.

We are looking for a Principal MLOps Engineer with a deep focus on ML Platforms and Infrastructure to join our Data & AI group at Cortex Research. Our team is responsible for designing, building, and scaling the foundational MLOps and LLMOps platforms that power both our Data Scientists and Security Researchers. You will architect the high-performance core infrastructure that enables these roles to build, train, and deploy advanced AI systems-ranging from optimized Small Language Models (SLMs) to complex agentic workflows and RAG systems. If you are passionate about building scalable compute platforms and automating the full ML lifecycle to solve complex data and security challenges, we want to hear from you.
Key Responsibilities
Scale Distributed Training: Design and optimize infrastructure for training and fine-tuning LLMs and SLMs, leveraging distributed GPU workloads, efficient clustering, and compute optimization.
Automate the ML Lifecycle: Architect robust, automated pipelines for continuous training (CT) and deployment (CD) of models, ensuring a seamless flow from raw data collection to production environments.
Build Model Infrastructure: Own the serving architecture for LLMs/SLMs, balancing latency, throughput, and GPU utilization under production traffic.
Implement Advanced Monitoring: Establish comprehensive observability systems to monitor live model performance, data drift, and computational metrics, feeding insights back into the automated training loops for continuous improvement.
Collaborative Architecture: Partner closely with data scientists and security researchers to productize complex model architectures and streamline their workflows, while collaborating with our DevOps team to integrate with core cloud infrastructure.

Were looking for a talented Data Engineer to join our AI team and build the data foundations that power our AI systems at scale. As a key member of the AI team, youll be responsible for designing, building, and operating the data layer that enables our AI models and agents to perform in production. This includes data ingestion, storage, transformation, monitoring, and defining the data strategy that ensures our AI applications receive high-quality, reliable, and timely data.

Were seeking an ambitious Data Engineer who is passionate about building robust data infrastructure, working with large-scale and complex datasets, and partnering closely with AI engineers and researchers to translate model needs into production-grade data systems.


WHAT YOU WILL DO
Design, build, and operate scalable data pipelines (batch + streaming) that power our AI systems in production.
Own the end-to-end data lifecycle: ingestion, storage, transformation, serving, and continuous improvement.
Build and maintain the data layer from raw data to semantically accessible data that enables AI models and agents to perform reliably at scale.
Ensure high data quality and high-standard operation through monitoring, alerting, and validation checks.
Work with modern storage systems (data lakes, warehouses, relational, graph, and vector databases) to support diverse AI workloads.
Partner closely with AI engineers and researchers to translate model requirements into production-grade data infrastructure.
Define and evolve the data strategy to for AI applications.

We are looking for a Data Engineer to join our team and play a key role in designing, building, and maintaining scalable, cloud-based data pipelines. You will work with AWS services , Airflow and databricks to integrate, process, and analyze large datasets, ensuring data reliability and efficiency.
Your work will directly impact business intelligence, analytics, and data-driven decision-making across the company.
What Youll Do:
ETL & Data Processing: Develop and maintain ETL processes, integrating data from various sources (APIs, databases, external platforms) using Python, SQL, and cloud technologies.
Utilize LangGraph and other frameworks to create state of the art AI agents that integrate various tools and data sources.
Implement integrations that allow agents to take automated actions on behalf of users, streamlining workflows and reducing overhead.
Data Modeling: Design and maintain logical and physical data models to support business needs.
Optimization & Scalability: Improve process efficiency and optimize runtime performance to handle large-scale data workloads.
Collaboration: Work closely with BI analysts and business stakeholders to define data requirements and functional specifications.
Monitoring & Troubleshooting: Ensure data integrity and reliability by proactively monitoring pipelines and resolving issues.

We are looking for a Principal MLOps Engineer with a deep focus on ML Platforms and Infrastructure to join our Data & AI group at Cortex Research. Our team is responsible for designing, building, and scaling the foundational MLOps and LLMOps platforms that power both our Data Scientists and Security Researchers. You will architect the high-performance core infrastructure that enables these roles to build, train, and deploy advanced AI systems-ranging from optimized Small Language Models (SLMs) to complex agentic workflows and RAG systems. If you are passionate about building scalable compute platforms and automating the full ML lifecycle to solve complex data and security challenges, we want to hear from you.
Key Responsibilities
Scale Distributed Training: Design and optimize infrastructure for training and fine-tuning LLMs and SLMs, leveraging distributed GPU workloads, efficient clustering, and compute optimization.
Automate the ML Lifecycle: Architect robust, automated pipelines for continuous training (CT) and deployment (CD) of models, ensuring a seamless flow from raw data collection to production environments.
Build Model Infrastructure: Own the serving architecture for LLMs/SLMs, balancing latency, throughput, and GPU utilization under production traffic.
Implement Advanced Monitoring: Establish comprehensive observability systems to monitor live model performance, data drift, and computational metrics, feeding insights back into the automated training loops for continuous improvement.
Collaborative Architecture: Partner closely with data scientists and security researchers to productize complex model architectures and streamline their workflows, while collaborating with our DevOps team to integrate with core cloud infrastructure.

We are looking for a Principal MLOps Engineer with a deep focus on ML Platforms and Infrastructure to join our Data & AI group at Cortex Research. Our team is responsible for designing, building, and scaling the foundational MLOps and LLMOps platforms that power both our Data Scientists and Security Researchers. You will architect the high-performance core infrastructure that enables these roles to build, train, and deploy advanced AI systems-ranging from optimized Small Language Models (SLMs) to complex agentic workflows and RAG systems. If you are passionate about building scalable compute platforms and automating the full ML lifecycle to solve complex data and security challenges, we want to hear from you.
Key Responsibilities
Scale Distributed Training: Design and optimize infrastructure for training and fine-tuning LLMs and SLMs, leveraging distributed GPU workloads, efficient clustering, and compute optimization.
Automate the ML Lifecycle: Architect robust, automated pipelines for continuous training (CT) and deployment (CD) of models, ensuring a seamless flow from raw data collection to production environments.
Build Model Infrastructure: Own the serving architecture for LLMs/SLMs, balancing latency, throughput, and GPU utilization under production traffic.
Implement Advanced Monitoring: Establish comprehensive observability systems to monitor live model performance, data drift, and computational metrics, feeding insights back into the automated training loops for continuous improvement.
Collaborative Architecture: Partner closely with data scientists and security researchers to productize complex model architectures and streamline their workflows, while collaborating with our DevOps team to integrate with core cloud infrastructure.

We are looking for a Senior Data Engineer to build and scale the data foundation behind platform and products. You will own complex data end-to-end - from ingestion and transformation through modeling, quality, observability, and production delivery.
This is a hands-on senior IC role for a strong builder who can solve difficult data problems independently, set a high technical bar, and collaborate closely with engineering, DS, and product. You will turn large, fragmented datasets into reliable, reusable capabilities that power every product.
Responsibilities
Build and own scalable data pipelines- Design, implement, and operate robust pipelines for high-volume structured and unstructured data, with validation, monitoring, lineage, and recovery built in.
Scale the platform for growth- A key near-term initiative is re-architecting the system to support a significantly larger customer base. You will own performance and cost-efficiency across pipelines and services, keeping reliability and operating costs under control as the platform scales.
Build across the stack- This is not a pipelines-only role. You will also write backend services and some frontend, including the internal backoffice the team runs on. We hire builders, not narrow specialists.
Own the core data tables- Own schema design and evolution, data contracts, and the modeling standards the team follows - naming, shared dimensions, normalization, documentation. Be accountable when a table is wrong, late, or drifting.
Level up the teams data work- Pair with and advise software engineers and data scientists on Spark, SQL, and modeling, and help turn notebook-grade code into production-grade pipelines.
Partner cross-functionally- Translate product, client, compliance, and business requirements into clear technical designs and dependable production systems.

Join the Cortex team as a Principal Backend Engineer and own the backend strategy for our integrated detection and response engine. You will be responsible for transforming raw endpoint, network, and cloud data into actionable intelligence, using AI-driven techniques to dramatically reduce false positives and focus our defense on what truly matters: stopping the worlds most critical threats.
Key Responsibilities
backed by the massive scale and resources of a global leader, to architect and scale a world-class data infrastructure capable of processing billions of events per second in real-time.
Integrate state-of-the-art AI technologies to transform raw telemetry from the industry's leading cyber products into high-fidelity, actionable intelligence.
Own the end-to-end software development life cycle, from conceptualizing complex distributed systems to deploying high-performance code, building AI-augmented systems that are fast, maintainable, and resilient.

Principal Software Engineer Data Platform ( Cortex)
Job Summary:
Join the Cortex team as a Principal Backend Engineer and own the backend strategy for our integrated detection and response engine. You will be responsible for transforming raw endpoint, network, and cloud data into actionable intelligence, using AI-driven techniques to dramatically reduce false positives and focus our defense on what truly matters: stopping the worlds most critical threats.
Key Responsibilities:
backed by the massive scale and resources of a global leader, to architect and scale a world-class data infrastructure capable of processing billions of events per second in real-time.
Integrate state-of-the-art AI technologies to transform raw telemetry from the industry's leading cyber products into high-fidelity, actionable intelligence.
Own the end-to-end software development life cycle, from conceptualizing complex distributed systems to deploying high-performance code, building AI-augmented systems that are fast, maintainable, and resilient.