דרושים » תוכנה » ML Software Engineer, Data Plane

The MLIL DataPlane team is looking for a Senior Software Development Engineer to own the design and implementation of our inference data plane. We build the software that makes large models run efficiently on custom hardware - spanning model execution, memory management, data movement, and serving integration.
Our work covers the full inference path: integrating serving engines with custom hardware, developing high-performance compute kernels, enabling efficient data movement, and driving models from early validation through production. We operate at frontier scale with large distributed models.
This is a ground-up effort with rapidly evolving hardware and software. We need a senior IC who can write and optimize low-level code for custom hardware, validate model architectures end-to-end, build test and profiling infrastructure, and drive performance across the stack.

Key job responsibilities
- Develop and optimize compute kernels for a custom ML accelerator architecture, targeting production-level performance for large language model inference.
- Implement and validate LLM architectures (decoder-only, mixture-of-experts) end-to-end - from PyTorch model definition through distributed execution on custom hardware.
- Integrate custom accelerator backends into open-source ML serving frameworks (vLLM, PyTorch), including scheduler extensions, memory management, and model parallelism.
- Build and maintain test infrastructure for model correctness validation across CPU, GPU, simulator, and hardware targets.
- Profile and optimize inference workloads - identify bottlenecks, instrument critical paths, and drive latency and throughput improvements from simulation through hardware bringup.
- Own features end-to-end: from design through implementation, testing, and integration into the broader software stack.
- Contribute to CI/CD pipelines that gate model and kernel changes on correctness and performance regressions.
- Mentor engineers, drive design reviews, and raise the engineering bar across the team.

The MLIL DataPlane team is looking for a Software Development Engineer to own the design and implementation of our inference data plane. We build the software that makes large models run efficiently on custom hardware - spanning model execution, memory management, data movement, and serving integration.
Our work covers the full inference path: integrating serving engines with custom hardware, developing high-performance compute kernels, enabling efficient data movement, and driving models from early validation through production. We operate at frontier scale with large distributed models.
This is a ground-up effort with rapidly evolving hardware and software. We need an individual contributor who can write and optimize low-level code for custom hardware, validate model architectures end-to-end, build test and profiling infrastructure, and drive performance across the stack.

Key job responsibilities
- Develop and optimize compute kernels for a custom ML accelerator architecture, targeting production-level performance for large language model inference.
- Implement and validate LLM architectures end-to-end - from PyTorch model definition through distributed execution on custom hardware.
- Integrate custom accelerator backends into open-source ML serving frameworks (vLLM, PyTorch), including scheduler extensions, memory management, and model parallelism.
- Build and maintain test infrastructure for model correctness validation across CPU, GPU, simulator, and hardware targets.
- Profile and optimize inference workloads - identify bottlenecks, instrument critical paths, and drive latency and throughput improvements from simulation through hardware bringup.
- Own features end-to-end: from design through implementation, testing, and integration into the broader software stack.
- Contribute to CI/CD pipelines that gate model and kernel changes on correctness and performance regressions.

We seek a versatile Senior Software Engineer who is passionate about performance optimization and generative AI. Our team brings the latest research in LLM inference - from novel decoding strategies to quantization schemes - into production across our hardware lineup, from large data center servers to powerful edge devices. We work on the most advanced architectures in the field, with a focus on NVIDIA's own.

What you'll be doing:

Implement and optimize inference algorithms for LLM and omnimodal architectures, including hybrid Mamba-Transformer and mixture-of-experts models.

Profile inference pipelines using NVIDIA's profiling and simulation tools. Correlate simulation predictions against real hardware across data center and edge devices.

Write and tune GPU kernels (CUDA, Triton) for operators like fused MoE layers, SSM state updates, and quantized GEMMs.

Solve distributed inference problems: expert parallelism, communication-compute overlap, collective tuning, multi-node deployment.

Build production-grade software inside major open-source libraries - vLLM, SGLang, Dynamo, FlashInfer.

Own optimization features end-to-end, from scoping through delivery, collaborating with research, product, and engineering teams worldwide.

We are looking for a Principal MLOps Engineer with a deep focus on ML Platforms and Infrastructure to join our Data & AI group at Cortex Research. Our team is responsible for designing, building, and scaling the foundational MLOps and LLMOps platforms that power both our Data Scientists and Security Researchers. You will architect the high-performance core infrastructure that enables these roles to build, train, and deploy advanced AI systems-ranging from optimized Small Language Models (SLMs) to complex agentic workflows and RAG systems. If you are passionate about building scalable compute platforms and automating the full ML lifecycle to solve complex data and security challenges, we want to hear from you.
Key Responsibilities
Scale Distributed Training: Design and optimize infrastructure for training and fine-tuning LLMs and SLMs, leveraging distributed GPU workloads, efficient clustering, and compute optimization.
Automate the ML Lifecycle: Architect robust, automated pipelines for continuous training (CT) and deployment (CD) of models, ensuring a seamless flow from raw data collection to production environments.
Build Model Infrastructure: Own the serving architecture for LLMs/SLMs, balancing latency, throughput, and GPU utilization under production traffic.
Implement Advanced Monitoring: Establish comprehensive observability systems to monitor live model performance, data drift, and computational metrics, feeding insights back into the automated training loops for continuous improvement.
Collaborative Architecture: Partner closely with data scientists and security researchers to productize complex model architectures and streamline their workflows, while collaborating with our DevOps team to integrate with core cloud infrastructure.

We are building a high-performance inference and fine-tuning platform designed to push foundation models to their hardware limits. Our mission is to maximize throughput, minimise latency, and optimise cost-per-token across tens of thousands of GPUs.



Some directions we are currently working on, and which you can be a part of:

Inference Optimization: Identifying LLM inference bottlenecks to drive production speedups. Squeezing the maximum performance for a wide range of LLM architectures at scale (e.g., GPT-OSS, Kimi K2.5, DeepSeek V3.1/V3.2, GLM-5).
Inference engines support: Implement novel speculative decoding architectures, optimise components of various LLM designs (dense/MoE, autoregressive/parallel), and contribute to open-source inference engines.
Low Precision Training & Inference: Design and productionise low-precision (FP8, NVFP4/MXFP4) training and inference pipelines with measurable gains in throughput and cost-efficiency.

The MLIL FLOW team is looking for a Software Development Engineer to design and build automation, tooling, and monitoring systems for our next-generation ML accelerator servers. We build production software to validate, initialize, monitor, and qualify these servers - from first silicon through fleet-scale deployment. Our work spans hardware diagnostics, manufacturing test automation, CI/CD pipelines, operational dashboards, and data-driven fleet health monitoring.

Key job responsibilities
Design and develop software infrastructure - automation frameworks, deployment systems, and test orchestration platforms that run at scale across manufacturing and production environments.
Work cross-functionally with Hardware, Manufacturing, and EC2 teams to automate coordinated software delivery and qualification workflows.
Debug and root-cause hardware/software interaction failures using systematic data analysis and automation-assisted triage.
Build and own CI/CD pipelines end-to-end: from code commit through build, test, deploy, and production validation - driving fast, reliable software delivery for hardware teams.
Create data pipelines and analytics systems (ETL, aggregation, real-time reporting) that transform raw hardware test results into actionable engineering insights.
Develop monitoring dashboards, alerting systems, and data visualization tools for fleet health, yield tracking, and performance benchmarking.
Own features end-to-end: from design through implementation, testing, deployment, and operational excellence.

We are seeking an AI Networking Architect to join the Networking Research Group. This role will help bridge the gap between emerging tasks supported by advanced technologies and the data center infrastructure that powers them. In this role, you will work at the intersection of AI applications, distributed systems, networking hardware, and software architecture.

You will join a focused team of multidisciplinary engineers driving AI workload optimization through deep application understanding, network analysis, and end-to-end systems thinking. Your insights will directly shape our products across the full stack - from applications and software libraries to hardware architecture and physical design.

What Youll Be Doing:

Model the performance of complex AI workloads to identify bottlenecks and recommend system-level optimizations.

Analyze brand-new AI models, distributed training techniques, and inference workloads to understand their infrastructure requirements.

Build Platforms, simulations and HW platforms, execute AI workloads and build analytical tools to evaluate trade-offs across compute, memory, storage, and network behavior.

Translate research insights and workload behavior into actionable software, hardware, and networking architecture requirements.

Partner with architecture, software, and product teams to influence our future networking and AI infrastructure roadmaps.

Drive architectural innovation by applying deep workload analysis to real-world advanced machine learning frameworks.

Were looking for a Senior MLOps Engineer to be a core driver in how our product empowers security teams. You will be expected to deeply understand customer needs and translate them directly into product features that deliver real value. You'll own key parts of our frontend stack, drive key architectural decisions, and turn complex security data into clear, actionable business insights.

As we scale our AIDR product and expand deeper into model-driven security intelligence, we are looking for a Senior MLOps Engineer to own the infrastructure, tooling, and operational foundations that power our NLP and LLM training, evaluation, and deployment workflows.

You will architect and operate the systems that enable us to train, fine-tune, deploy, and monitor models at scale making ML reliable, fast, cost-efficient, and production-ready.

This is a high-visibility, high-impact role where you will partner closely with DevOps, Backend, Data, and Product to establish world-class ML infrastructure from the ground up.

What Youll Do

Build & Scale ML Pipelines
Design, build, and maintain pipelines for training, fine-tuning, evaluating, and deploying NLP and LLM models across GPU and CPU environments.
Establish LLM-Focused CI/CD
Implement automated CI/CD workflows for ML models, including benchmarking, testing, performance gating, and production deployment.
Optimize Runtime & Inference
Select and optimize serving frameworks for low-latency, high-throughput inference, ensuring reliability and scalability.
Own ML Infrastructure
Manage training environments, experiment tracking, model registries, artifact versioning, and distributed training systems.
Operational Excellence
Monitor and optimize production models for performance, cost efficiency, availability, and observability.

We are seeking a highly skilled and versatile Performance Research and Analysis Manager to join our Performance Group. This role will drive end-to-end performance strategy and execution for next-generation our data centers and solutions based on GPU systems, NIC, Switch, DPU and Networking technologies. The ideal candidate will oversee, evaluating, and optimizing end-to-end AI GPU cluster-level performance for scaling out large scale distributed training and inference jobs communication. The role will focus heavily on RDMA, Networking Protocols, Collective Communication, Congestion Control, and Load Balancing algorithms. Secondarily, you will lead our DPUs and Storage technologies for N-S use cases to support AI Inference jobs. Third, you will drive our Performance Dashboards and Observability for cluster-level performance analysis from a stream line telemetry across NICs, Switches, GPUs, and NVlink.

What you'll be doing:

Drive end-to-end performance strategy, characterization, test plans, and optimization for next-generation our AI GPU clusters, focusing on large-scale distributed training and inference workloads.

Deeply evaluate and optimize our Networking core technologies performance, including RDMA/PRDMA, networking protocols, collective communication (NCCL), congestion control, and load-balancing algorithms.

Work on performance research and analysis of NVIDIA DPUs and storage technologies in North-South (N-S) use cases and deployment scenarios to maximize performance and efficiency for AI inference jobs.

Drive the strategy for performance observability and dashboards across next-generation NVIDIA data center solutions and supercomputers by leveraging scalable, streamlined telemetry pipelines to build performance dashboards and automated analytics based on real-time performance metrics across NICs, Switches, GPUs, and NVLink boundaries.

Perform deep root-cause analysis (RCA) on complex multi-node performance bottlenecks, driving actionable mitigation plans across hardware, firmware, and software teams.

Help build an Always-On, low-overhead GPU profiling service that runs in production, scales across cluster environments, and delivers actionable insights for ML workloads. You will be hands-on delivering our profiling solutions across system software, drivers, and CUDA to make profiling continuously available and reliable.

What youll be doing:

Develop low-overhead, high-reliability implementations in C/C++, with bounded CPU/memory budgets.

Lead end-to-end feature delivery spanning user-mode components, driver/platform layers, and performance counter/trace providers.

Establish profiling models that integrate with existing ML/AI workflows (e.g., PyTorch/XLA) to turn low-level signals into actionable insights.