דרושים » תוכנה » Senior Software Engineer, Deep Learning Inference

We are hiring exceptional software engineers to build and optimize the core inference infrastructure for large language models. Join the TensorRT‑LLM team - the group defining how generative AI performs at global scale on our GPUs. Were looking for engineers who love squeezing every drop of throughput, memory efficiency, and scalability out of modern model runtimes. Your work will directly shape the frameworks behind state‑of‑the‑art LLM inference used across the company and the AI community. Join us to redefine what fast means for LLM inference - building the frameworks that power the next generation of generative AI at scale.
What you'll be doing:
Design, implement, and optimize high‑performance inference pipelines for large language models running on GPUs
Profile and tune model execution across the stack - from scheduler design to kernel fusions and everything in-between
Design and experiment with memory management strategies for improved memory bandwidth optimization and cache efficiency
Innovate and Implement cutting-edge techniques such as Speculative Decoding, Context Caching, and FP8/INT4 quantization to push the boundaries of tokens-per-second-per-watt
Develop and maintain benchmarking and testing systems that quantify latency, utilization, and efficiency.

Required Senior ML Engineer - Applied AI Engineering Group
The Dream Job
It starts with you - an engineer driven to build the ML platform that turns research into reliable, production-grade intelligence. You care about reproducibility, low-friction experimentation, and infrastructure that earns the trust of the scientists and researchers who depend on it daily. You'll architect and ship our ML platform - training pipelines, model serving, feature stores, experiment tracking, and compute orchestration - turning models into production capabilities across cloud and on-prem, including air-gapped deployments. A significant part of the platform supports large language models, with unique challenges across training, evaluation, and inference in mission-critical environments.
If you want to make a meaningful impact, join our mission and build the ML platform that drives Sovereign AI products - this role is for you.
The Dream-Maker Responsibilities
Build and operate ML training infrastructure - distributed training pipelines, compute scheduling, and reproducible experiment workflows that data scientists rely on daily.
Own model serving and inference systems - packaging, deployment, autoscaling, A/B testing, canary rollouts, and latency/cost optimization for production models.
Run feature stores, model registries, and dataset versioning - enabling self-serve feature engineering, model lineage, and reproducible experiments across teams.
Build experiment tracking and evaluation infrastructure - automated evals, comparison dashboards, drift detection, and monitoring that give teams visibility into model behavior and performance.
Build and maintain production pipelines for training, fine-tuning workflows, and serving domain models - owning reliability, reproducibility, and scale.
Build and maintain the monitoring and observability layer - model performance tracking, data and prediction drift detection, data quality validation, and alerting.
Improve performance and cost across the ML stack - training throughput, inference latency, batch vs. real-time tradeoffs, and compute cost management.
Ship shared tooling - libraries, templates, CI/CD for models, IaC, and runbooks - while collaborating across Data Platform, AI, Data Science, Engineering, and DevOps. Own architecture, documentation, and operations end-to-end.

We are looking for a highly skilled Senior Machine Learning Engineer to lead our transition from on-demand, third-party LLM APIs to a fully self-hosted, scalable model ecosystem.
Our core product is an advanced, agentic support chatbot capable of complex reasoning, API tool calling, database lookups, and orchestrating specialized Small Language Models (SLMs) for targeted NLP tasks. As we scale, our current deployment infrastructure (AWS SageMaker) is becoming unsustainable. You will be responsible for architecting, deploying, and optimizing an infrastructure capable of supporting 50 to 100 distinct models ranging from 100M to 70B parameters.
What Youll Do:
Inference Optimization: Deploy and manage large-scale models using high-performance inference engines (like vLLM) to ensure low latency and high throughput for our agentic chatbot.
Agentic Workflows: Develop and refine the chatbot's agentic capabilities, ensuring reliable tool-use, routing, and interactions between massive LLMs and specialized SLMs.
Model Fine-Tuning: Design and execute fine-tuning strategies to improve model accuracy on specific domain tasks and tool-calling execution.
Rigorous Evaluation: Build comprehensive offline and online evaluation frameworks to constantly measure model performance and business impact through structured A/B testing.

Our team is looking for a Deep Learning Engineer.
We are one of the few companies to have trained multi-billion parameter Large Language Models (LLMs), a feat that involves the most advanced engineering (large scale distributed training on thousands of cores). Serving these LLMs efficiently requires cutting-edge technology as well. As a deep learning engineer on the team, you will be responsible for maintaining and improving our training infrastructure, developing/scaling/testing new ideas, and adapting our code to run on and best utilize the newest and most advanced hardware accelerators.
Role and Responsibilities
Develop Large Language Models as part of our applied research projects and in support of our Platform, including designing, implementing and training massive-scale deep language models
Implement, optimize, scale and test new cutting edge ideas and architectures
Perform large-scale evaluations and comparisons of trained models across a range of benchmarks, as well as adding support for new benchmarks

We are looking for an enthusiastic software engineer to join our AI networking acceleration team, to work on a groundbreaking open-source library, using hardware offloads, GPU Kernels and RDMA network cards. Our product is a performance-oriented low-level infrastructure, crafted to change the way inference works.
We thrive as a team in a deeply strong environment, and we're passionate about innovation. The rewards are sweet and include working with some of the brightest people in the industry, an aggressive compensation plan that rewards top performers, and the opportunity to collaborate on products that transform daily the way people work and play.
What you'll be doing:
Developing a highly optimized inference framework
Running on the worlds largest supercomputers and data centers.
The work environment is dynamic and challenging as our employees work on innovative, next-generation products at the forefront of technology in terms of performance, scalability, and features.

We are seeking an exceptional Senior Backend Engineer to join our Platform Group, where you will architect and develop shared infrastructure components that power enterprise security and identity products. This is a highly collaborative role requiring both technical excellence and strong cross-functional partnership skills.
What You'll Do:
As a Senior Backend Engineer on our Core Platform team, you will:
Design and build shared platform components used across multiple product teams, ensuring scalability, reliability, and maintainability
Architect distributed systems using microservices and event-driven patterns that support enterprise-scale workloads
Own critical backend services from conception through deployment, including code quality, performance optimization, and operational excellence
Collaborate extensively with product engineers, DevOps, and architects to define platform capabilities and technical standards
Mentor junior engineers through code reviews, pair programming, and technical guidance
Drive technical initiatives that improve developer productivity, system observability, and platform resilience
Contribute to architectural decisions and establish best practices for backend development across the organization
Analyze system performance and data patterns to identify optimization opportunities and inform future platform investments

we are looking for a Senior Machine Learning Engineer I.
As Senior Machine Learning Engineer, youll work with top notch engineers and data scientists from the team on bringing it to the next level and enabling optimal user experience. The work will focus on building, deploying and serving GenAI capabilities (Agents, Tools and the orchestration between them) using the most advanced technologies and models.
Key Job Responsibilities and Duties:
Deploying machine learning models: Design, develop and deploy in collaboration with scientists, scalable machine learning models and algorithms that provide content related insights and generative AI applications, ensuring scalability, efficiency, and accuracy.
Evaluating possible architecture solutions by taking into account cost, business requirements, emerging technologies, and technology requirements, like latency, throughput, and scale.
Generative AI Development: Contribute to the development of generative models such as GPT (Generative Pre-trained Transformer) variants or similar architectures for creative content generation, Q&A, translation or other innovative applications.
Deployment and integration: Work closely with software engineers to integrate machine learning models into production systems. Ensure seamless deployment and efficient model inference in real-time environments. Collaborate with DevOps to implement effective monitoring and maintenance strategies.
Owning a service end to end by actively monitoring application health and performance, setting and monitoring relevant metrics and acting accordingly when violated.
Maintain clean, scalable code, ensuring reproducibility and easy integration of models into production environments, including CI/CD.
Collaborate with multidisciplinary teams: Collaborate with product managers, data scientists, and analysts to understand business requirements and translate them into machine learning solutions.
20031

We are looking for a phenomenal engineer to join the chip simulation team for networking chips and GPUs.
This simulation platform enables our engineers across firmware, SDK, and OS domains to develop and test their code without relying on physical hardware. If you're a creative, self-driven engineer passionate about systems-level design and eager to build technology that empowers internal teams, we want to hear from you.
What Youll Be Doing:
Develop and maintain simulation components for the physical layer of our high-performance networking chips (e.g., GPUs, switches, NVLink, Ethernet...).
Collaborate with chip architects, firmware developers, and hardware design teams to accurately simulate complex physical-layer behaviors in software.
Define, implement, and validate simulations of features such as link training, error injection, and transceiver behavior, making the simulation platform a go-to internal platform for development and debugging.
Extend and optimize the simulation infrastructure by contributing to CI pipelines, automated test frameworks, and regression tools.
Support internal users by debugging simulation flows and collaborating on bug resolution.
Take part in future-facing innovation by enabling simulation for next-generation devices and features.

We are looking for a 100% hands-on Storage Services Software engineer to join the block storage group. You will be a member of a team that builds the next generation block storage capabilities. You will work closely with a variety of teams and architects including the networking team, and external customers. You will take part in defining the software architecture and implementation of the most advanced storage services! Services that will need to meet extreme performance and scalability demands! We have crafted a team of extraordinary people stretching around the globe, whose mission is to push the frontiers of what is possible today and define the platform of tomorrow.
We work, think and learn as a team. We thrive in a deeply strong environment, and we're passionate about a culture that demands innovation and the highest standards. The rewards are sweet and include collaborating with some of the smartest people in the industry, an aggressive compensation plan that rewards top performers, and the opportunity to work on products that transform the way people work and play.
What youll be doing:
100% hands-on coding role in C language, kernel and userspace
Access advanced AI tools and a token budget for code development provided by us, the world's AI factory leader.
Research, design, implement and test, new and existing, networking features for distributed storage services and features of our block storage solution, in both Host and DPU environments.
Acquire understanding of the algorithms, the technicalities and the interaction with other components across our block storage ecosystem.
Analyze and solve challenging bugs and customer cases in large scale production systems, identifying issues in our or inbox kernel modules and often in other components. Drive new solutions based on any issues that arise.

We are looking for a 100% hands-on Network Software engineer to join the block storage group. You will be a member of a team that builds the next generation block storage capabilities. You will work closely with a variety of teams and architects including the networking team, and external customers. You will take part in defining the software architecture and implementation of the most advanced storage services! Services that will need to meet extreme performance and scalability demands! We have crafted a team of extraordinary people stretching around the globe, whose mission is to push the frontiers of what is possible today and define the platform of tomorrow.
We work, think and learn as a team. We thrive in a deeply strong environment, and we're passionate about a culture that demands innovation and the highest standards. The rewards are sweet and include collaborating with some of the smartest people in the industry, an aggressive compensation plan that rewards top performers, and the opportunity to work on products that transform the way people work and play.
What youll be doing:
100% hands-on coding role in C language, kernel and userspace
Access advanced AI tools and a token budget for code development provided by us, the world's AI factory leader.
Research, design, implement and test, new and existing, advance networking service and features of our block storage solution, in both Host and DPU environments.
Acquire understanding of the algorithms, the technicalities and the interaction with other components across our block storage ecosystem.
Analyze and solve challenging bugs and customer cases in large scale production systems, identifying issues in our or inbox kernel modules and often in other components. Drive new solutions based on any issues that arise.