דרושים » תוכנה » Senior ML Engineer (Token Factory)

Some directions we currently working on and which you can be a part of:
Advanced Fine-Tuning: Enhancing fine-tuning methodologies - both LoRA-based and full-parameter - for cutting-edge LLMs (e.g., GPT-OSS, Kimi K2.5, DeepSeek V3.1/V3.2, GLM-4.7), focusing on both model quality and training efficiency.

Inference Optimization: Identifying LLM inference bottlenecks to drive production speedups. This involves building model training and evaluation pipelines in JAX for speculative decoding, experimenting with architectures (dense/MoE, auto-regressive/parallel), and deriving scaling laws to guide resource allocation.
Low Precision Training & Inference: Investigating low-precision (FP8, NVFP4/MXFP4) methodologies for supervised fine-tuning and reinforcement learning - spanning both inference and training - optimized for modern hardware

We seek a versatile Senior Software Engineer who is passionate about performance optimization and generative AI. Our team brings the latest research in LLM inference - from novel decoding strategies to quantization schemes - into production across our hardware lineup, from large data center servers to powerful edge devices. We work on the most advanced architectures in the field, with a focus on NVIDIA's own.

What you'll be doing:

Implement and optimize inference algorithms for LLM and omnimodal architectures, including hybrid Mamba-Transformer and mixture-of-experts models.

Profile inference pipelines using NVIDIA's profiling and simulation tools. Correlate simulation predictions against real hardware across data center and edge devices.

Write and tune GPU kernels (CUDA, Triton) for operators like fused MoE layers, SSM state updates, and quantized GEMMs.

Solve distributed inference problems: expert parallelism, communication-compute overlap, collective tuning, multi-node deployment.

Build production-grade software inside major open-source libraries - vLLM, SGLang, Dynamo, FlashInfer.

Own optimization features end-to-end, from scoping through delivery, collaborating with research, product, and engineering teams worldwide.

We are looking for a Software Development Engineer to own the design and implementation of our inference data plane. We build the software that makes large models run efficiently on custom hardware - spanning model execution, memory management, data movement, and serving integration.
Our work covers the full inference path: integrating serving engines with custom hardware, developing high-performance compute kernels, enabling efficient data movement, and driving models from early validation through production. We operate at frontier scale with large distributed models.
This is a ground-up effort with rapidly evolving hardware and software. We need an individual contributor who can write and optimize low-level code for custom hardware, validate model architectures end-to-end, build test and profiling infrastructure, and drive performance across the stack.

Key job responsibilities
- Develop and optimize compute kernels for a custom ML accelerator architecture, targeting production-level performance for large language model inference.
- Implement and validate LLM architectures end-to-end - from PyTorch model definition through distributed execution on custom hardware.
- Integrate custom accelerator backends into open-source ML serving frameworks (vLLM, PyTorch), including scheduler extensions, memory management, and model parallelism.
- Build and maintain test infrastructure for model correctness validation across CPU, GPU, simulator, and hardware targets.
- Profile and optimize inference workloads - identify bottlenecks, instrument critical paths, and drive latency and throughput improvements from simulation through hardware bringup.
- Own features end-to-end: from design through implementation, testing, and integration into the broader software stack.
- Contribute to CI/CD pipelines that gate model and kernel changes on correctness and performance regressions.

The MLIL DataPlane team is looking for a Software Development Engineer to own the design and implementation of our inference data plane. We build the software that makes large models run efficiently on custom hardware - spanning model execution, memory management, data movement, and serving integration.
Our work covers the full inference path: integrating serving engines with custom hardware, developing high-performance compute kernels, enabling efficient data movement, and driving models from early validation through production. We operate at frontier scale with large distributed models.
This is a ground-up effort with rapidly evolving hardware and software. We need an individual contributor who can write and optimize low-level code for custom hardware, validate model architectures end-to-end, build test and profiling infrastructure, and drive performance across the stack.

Key job responsibilities
- Develop and optimize compute kernels for a custom ML accelerator architecture, targeting production-level performance for large language model inference.
- Implement and validate LLM architectures end-to-end - from PyTorch model definition through distributed execution on custom hardware.
- Integrate custom accelerator backends into open-source ML serving frameworks (vLLM, PyTorch), including scheduler extensions, memory management, and model parallelism.
- Build and maintain test infrastructure for model correctness validation across CPU, GPU, simulator, and hardware targets.
- Profile and optimize inference workloads - identify bottlenecks, instrument critical paths, and drive latency and throughput improvements from simulation through hardware bringup.
- Own features end-to-end: from design through implementation, testing, and integration into the broader software stack.
- Contribute to CI/CD pipelines that gate model and kernel changes on correctness and performance regressions.

Were looking for a Senior MLOps Engineer to be a core driver in how our product empowers security teams. You will be expected to deeply understand customer needs and translate them directly into product features that deliver real value. You'll own key parts of our frontend stack, drive key architectural decisions, and turn complex security data into clear, actionable business insights.

As we scale our AIDR product and expand deeper into model-driven security intelligence, we are looking for a Senior MLOps Engineer to own the infrastructure, tooling, and operational foundations that power our NLP and LLM training, evaluation, and deployment workflows.

You will architect and operate the systems that enable us to train, fine-tune, deploy, and monitor models at scale making ML reliable, fast, cost-efficient, and production-ready.

This is a high-visibility, high-impact role where you will partner closely with DevOps, Backend, Data, and Product to establish world-class ML infrastructure from the ground up.

What Youll Do

Build & Scale ML Pipelines
Design, build, and maintain pipelines for training, fine-tuning, evaluating, and deploying NLP and LLM models across GPU and CPU environments.
Establish LLM-Focused CI/CD
Implement automated CI/CD workflows for ML models, including benchmarking, testing, performance gating, and production deployment.
Optimize Runtime & Inference
Select and optimize serving frameworks for low-latency, high-throughput inference, ensuring reliability and scalability.
Own ML Infrastructure
Manage training environments, experiment tracking, model registries, artifact versioning, and distributed training systems.
Operational Excellence
Monitor and optimize production models for performance, cost efficiency, availability, and observability.

Join our companys AI research group, a cross-functional team of ML engineers, researchers and security experts building the next generation of AI-powered security capabilities. Our mission is to leverage large language models to understand code, configuration, and human language at scale, and to turn this understanding into security AI capabilities which will drive our company AI future security solutions.
We foster a hands-on, research-driven culture where youll work with large-scale data, modern ML infrastructure, and a global product footprint that impacts over 100,000 organizations worldwide.
Key Responsibilities
Your Impact & Responsibilities
As a Senior ML Research Engineer, you will be responsible for the end-to-end lifecycle of large language models: from data definition and curation, through training and evaluation, to providing robust models that can be consumed by product and platform teams.
Own training and fine-tuning of LLMs / seq2seq models: Design and execute training pipelines for transformer-based models (encoder-decoder, decoder-only, retrievalaugmented, etc.), and fine-tune open-source LLMs on our company-specific data (security content, logs, incidents, customer interactions).
Apply advanced LLM training techniques such as instruction tuning, preference / contrastive learning, LoRA / PEFT, continual pre-training, and domain adaptation where appropriate.
Work deeply with data: define data strategies with product, research and domain experts; build and maintain data pipelines for collecting, cleaning, de-duplicating and labeling large-scale text, code and semi-structured data; and design synthetic data generation and augmentation pipelines.
Build robust evaluation and experimentation frameworks: define offline metrics for LLM quality (task-specific accuracy, calibration, hallucination rate, safety, latency and cost); implement automated evaluation suites (benchmarks, regression tests, redteaming scenarios); and track model performance over time.
Scale training and inference: use distributed training frameworks (e.g. DeepSpeed, FSDP, tensor/pipeline parallelism) to efficiently train models on multi-GPU / multi-node clusters, and optimize inference performance and cost with techniques such as quantization, distillation and caching.
Collaborate closely with security researchers and data engineers to turn domain knowledge and threat intelligence into high-value training and evaluation data, and to expose your models through well-defined interfaces to downstream product and platform teams.

Join our company, an innovative startup building a fully-managed LLM-inference platform, that enables data heavy enterprises to perform any AI task at any scale without limits.
We're looking for an Experienced Performance Researcher to join our founding team. Youll be responsible for building and optimizing scalable cloud infrastructure solutions tailored for AI workloads. This role offers a unique opportunity to directly shape our infrastructure strategy, improve system reliability and performance, and contribute to establishing our company as a leader in adaptive AI compute management.
Join us to tackle the magic that make AI tick under the hood and build the backbone powering the AI revolution.
What Youll Do
- Design and build high-performance distributed inference pipelines for LLMs, focused on large-batch, non-real-time scenarios.
- Optimize GPU memory usage, kernel execution, and communication across nodes (NCCL, MPI, etc.).
- Own CUDA kernels, compiler-level tricks, and multi-GPU scheduling logic.
- Lead profiling and performance tuning for throughput, and cost- down to the kernel level.
- Collaborate with infra, product, and research teams to define SLAs, resource allocation logic, and runtime behaviors.
- Help build the core infrastructure that will run LLM workloads across hybrid GPU environments (cloud/on-prem/self-hosted).

We are seeking a highly motivated Senior Deep Learning Researcher to join our team! This is an outstanding opportunity to conduct impactful research and develop the next generation of large language model (LLM) inference algorithms. You will work on technologies that directly enhance our software, making the latest LLMs more efficient and accessible for users worldwide.

By joining us, you will be part of a strategic effort to establish us as the definitive platform for high-performance LLM inference. You will engage with our skilled problem-solvers and top organizations, crafting AI technology advancements.

What you'll be doing:
Research, invent, and implement groundbreaking algorithms for LLM inference to advance the state of the art in both low-latency and high-throughput scenarios.

Translate research into practical software solutions that directly impact our products and customers.

Collaborate with internal research, engineering, and product teams across the globe to drive the development of advanced inference technologies.

Analyze the performance of new algorithms on our latest hardware, identifying bottlenecks and opportunities for algorithmic optimizations.

Partner with leading scientific organizations and industry pioneers to remain at the forefront of technological advancements and integrate the latest innovations into practical applications.

we are seeking a Senior Product Manager focused on ML Platform to be a key member of our Product Management team. Join a dynamic and forward-thinking company at the forefront of AI infrastructure. We leverage advanced technologies to develop innovative solutions that drive efficiency, scalability, and exceptional compute performance. Collaborate with the industry's best as we partner with hyperscalers, emerging NeoClouds, and enterprises building large-scale AI clusters, shaping the future of heterogeneous AI infrastructure. Our environment fosters creativity, teamwork, and growth, and offers you the opportunity to make a meaningful impact while working on groundbreaking projects.
As a Senior Product Manager for the ML Platform, you will own the strategy, roadmap, and feature definition of ' heterogeneous inference serving platform - a system designed to enable efficient inference across diverse and mixed compute environments. You will work directly with our R&D teams and end customers to shape the product, engage compute and storage partners to co-define reference architectures, and serve as an internal expert on performance benchmarking and collective communication tuning in support of ' cluster and performance engineering teams.

We're looking for our **first Forward Deployed Engineer**. You'll sit directly with our customers' engineering teams and get their workloads onto Impala - from the first scoping conversation to a production endpoint carrying real traffic, at a cost and latency profile they couldn't hit anywhere else.
This is an engineering role. You will write code every day, in customer repos and in ours. It also carries pieces of solutions architecture, product management, and pre-sales, and you should want that mix rather than tolerate it. Ambiguous business goals come in; observable, benchmarked, production services go out.
As the founding FDE you also define the function: what a POC looks like, what we promise and measure, which patterns get productized, and how the field feeds the roadmap. The next FDEs will work from what you build here.
What You'll Do
- **Own customer outcomes end to end:** problem framing, evaluation design, migration, deployment, benchmarking, monitoring. You're the technical owner from first call through expansion.
- **Win the POC:** turn a vague objective into a tight spec and a working proof of concept fast, with explicit quality, latency, throughput, and cost-per-token targets - and hit them.
- **Tune serverless inference for real workloads:** model and engine selection, batching strategy, KV cache behavior, speculative decoding, quantization, parallelism, cold-start and autoscaling behavior under bursty traffic. Diagnose regressions down to the inference engine.
- **Migrate workloads onto Impala:** move customers off OpenAI-compatible APIs, self-managed vLLM, SageMaker, or their own GPU fleets - and prove out the quality and cost delta with numbers.
- **Guide model strategy:** advise on open-weight model selection, distillation, and fine-tuning for specific tasks; help customers get from a general-purpose frontier model to a smaller, faster, cheaper one that holds quality.
- **Close the product loop:** bring the field back into the roadmap - write the PRDs, land the PRs, and turn one-off customer work into platform features.
- **Be the technical anchor in the room:** support sales on complex evaluations, run technical onboarding, earn trust with staff engineers and CTOs.