דרושים » תוכנה » Senior Software Engineer, Deep Learning Inference

we are seeking a sharp, innovative, and hands-on Architect to help shape the future of LLM inference at scale. Join our dynamic E2E Architecture group, where we build cutting-edge systems powering the next generation of generative AI workloads. In this role, you will work across software and hardware domains to design and optimize inference infrastructure for large language models running on some of the most advanced GPU clusters in the world.
Youll help define how AI models are deployed and scaled in production, driving decisions on everything from memory orchestration and compute scheduling to inter-node communication and system-level optimizations. This is an opportunity to work with top engineers, researchers, and partners across our company and leave a mark on the way generative AI reaches real-world applications.
What Youll Be Doing:
Design and evolve scalable architectures for multi-node LLM inference across GPU clusters.
Develop infrastructure to optimize latency, throughput, and cost-efficiency of serving large models in production.
Collaborate with model, systems, compiler, and networking teams to ensure holistic, high-performance solutions.
Prototype novel approaches to KV cache handling, tensor/pipeline parallel execution, and dynamic batching.
Evaluate and integrate new software and hardware technologies relevant to model inference (e.g., memory hierarchy, network topology, modern inference architectures).
Work closely with internal teams and external partners to translate high-level architecture into reliable, high-performance systems.
Author design documents, internal specs, and technical blog posts and contribute to open-source efforts when appropriate.

Required Machine Learning Performance Engineer
Description
Our team is responsible for the AWS Neuron software stack, which powers Generative AI and other advanced ML workloads on AWS's custom-built ML accelerators Inferentia and Trainium. These accelerators deliver best-in-class performance and cost-efficiency for ML inference and training in the cloud.
We're building a new core group of engineers in TLV (Tel Aviv) to drive innovation in ML systems performance and software. As a Machine Learning Performance Engineer, you'll help shape the direction of the team from the ground up and work on:
Optimizing system performance across the entire ML software stack
Analyzing high-performance ML workloads running on Annapurna hardware
Developing high-performance kernels for critical ML operations
Enhancing the Neuron SDK to improve developer experience and system capabilities
Collaborating across Compiler, Frameworks, and Hardware teams to maximize end-to-end performance
As part of the Performance Engineering Team, you'll contribute to projects involving instruction scheduling, memory management, parallelism, kernel optimization, and compiler enhancements to maximize end-to-end performance.
This is a unique opportunity to be at the intersection of ML and systems within AWS, helping to build the future of AI infrastructure right here in Tel Aviv.
Key job responsibilities
Our engineers collaborate across diverse teams, projects, and environments to have a firsthand impact on our global customer base. You will:
Solve challenging technical problems, often ones not solved before, at every layer of the stack.
Design, implement, test, deploy and maintain innovative software solutions to transform service performance, durability, cost, and security.
Research implementations that deliver the best possible experiences for customers.
A day in the life
As you design and code solutions to help our team drive efficiencies in software architecture, youll create metrics, implement automation and other improvements, and resolve the root cause of software defects. Youll also:
Build high-impact solutions to deliver to our large customer base.
Participate in design discussions, code review, and communicate with internal and external stakeholders.
Work cross-functionally to help drive business decisions with your technical input.
Work in a startup-like development environment, where youre always working on the most important stuff.
Considering the technical requirements of this role, and to ensure fair and equal candidate consideration, we are conducting full interview loops. If you meet the basic qualifications, and the team is interested in moving forward, we will be in touch to coordinate your interviews.

Required ML Engineer - AI Infra Group
Tel Aviv Full-time
We are on an expedition to find you, someone who is passionate about creating intuitive, out-of-this-world production-grade AI infrastructure. This group builds scalable, high-performance AI systems for internal users and external customers, designed to run seamlessly across cloud and on-premise environments using the latest hardware advancements.
Responsibilities
Design and optimize LLM serving infrastructure using inference engines (vLLM, TensorRT-LLM, Triton Inference Server)
Implement and tune distributed inference strategies including tensor parallelism, pipeline parallelism, and multi-node serving
Develop and apply model compression techniques to optimize cost, latency, and memory footprint while maintaining model quality
Build self-service fine-tuning platforms that enable data scientists to run experiments (LoRA, QLoRA, full fine-tuning) in a standardized, reproducible, and governed manner
Optimize inference performance through batching strategies, KV-cache tuning, and speculative decoding
Develop reusable APIs, abstractions, and platform services for model deployment, scaling, and lifecycle management
Collaborate with AI researchers and product teams to productionize models and meet latency/throughput requirements
Evaluate and benchmark new model architectures, compression methods, and serving frameworks.

Required Senior Algo Data Engineer
Realize your potential by joining the leading performance-driven advertising company!
As a Senior Algo Data Engineer on the Infra group, youll play a vital role in develop, enhance and maintain highly scalable Machine-Learning infrastructures and tools.
About Algo platform:
The objective of the algo platform group is to own the existing algo platform (including health, stability, productivity and enablement), to facilitate and be involved in new platform experimentation within the algo craft and lead the platformization of the parts which should graduate into production scale. This includes support of ongoing ML projects while ensuring smooth operations and infrastructure reliability, owning a full set of capabilities, design and planning, implementation and production care.
The group has deep ties with both the algo craft as well as the infra group. The group reports to the infra department and has a dotted line reporting to the algo craft leadership.
The group serves as the professional authority when it comes to ML engineering and ML ops, serves as a focal point in a multidisciplinary team of algorithm researchers, product managers, and engineers and works with the most senior talent within the algo craft in order to achieve ML excellence.
How youll make an impact:
As a Senior Algo Data Engineer, youll bring value by:
Develop, enhance and maintain highly scalable Machine-Learning infrastructures and tools, including CI/CD, monitoring and alerting and more
Have end to end ownership: Design, develop, deploy, measure and maintain our machine learning platform, ensuring high availability, high scalability and efficient resource utilization
Identify and evaluate new technologies to improve performance, maintainability, and reliability of our machine learning systems
Work in tandem with the engineering-focused and algorithm-focused teams in order to improve our platform and optimize performance
Optimize machine learning systems to scale and utilize modern compute environments (e.g. distributed clusters, CPU and GPU) and continuously seek potential optimization opportunities.
Build and maintain tools for automation, deployment, monitoring, and operations.
Troubleshoot issues in our development, production and test environments
Influence directly on the way billions of people discover the internet
Our tech stack:
Java, Python, TensorFlow, Spark, Kafka, Cassandra, HDFS, vespa.ai, ElasticSearch, AirFlow, BigQuery, Google Cloud Platform, Kubernetes, Docker, git and Jenkins.

We have improved AI infrastructure by merging GPU virtualization with Kubernetes-native tech to power innovative AI factories. We aim to speed up enterprise AI projects with smart orchestration, and scalability for AI workloads. Seeking a skilled Senior Software Engineer for our Infrastructure Group to innovate AI technology. The Infrastructure Group is tasked with composing and evolving the core systems responsible for thousands of GPUs and nodes driving enterprise AI. We invent the foundation that facilitates elastic, secure, and observable AI operations at extensive scale. We are seeking engineers who are passionate about distributed systems, modern cloud-native infrastructure, and AI performance optimization.

What youll be doing:

Crafting and developing enterprise-grade systems with a strong focus on scalability, reliability, and performance.

Building and optimizing microservices-based architectures using Kubernetes and cloud-native technologies.

Collaborating closely with backend engineers, product managers, and other partners to deliver impactful solutions.

Writing clean, maintainable, and testable code in Go, contributing to our CI/CD pipelines.

Conducting code and build reviews to uphold high-quality standards and mentor team members.

Leading the development and implementation of advanced identity management systems that secure our innovative AI and GPU cloud.

Developing scalable multi-tenant solutions that allow our diverse clientele to harness the power of our platforms securely and efficiently.

Collaborating with multi-functional teams to integrate identity and access management features seamlessly into our products, from cloud services to edge computing devices.

our company has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. Its a unique legacy of innovation thats fueled by great technologyand amazing people. Today, were tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing whats never been done before takes vision, innovation, and the worlds best talent. As a worker, youll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.
we are seeking a highly skilled and modern software engineer to develop and prototype brand new advancements in distributed training and inference using our companys Spectrum-X AI fabric. This role offers a rare chance to pioneer AI and networking technology, contributing to ground-breaking projects that will define the landscape of large-scale AI systems. Improve AI app-networking connection by refining communication, crafting congestion control, coding NIC firmware, and expanding switch SDK features for enhanced AI factory efficiency. Your work impacts large AI system development, scaling, and speed.
What youll be doing:
Prototype end-to-end solutions to improve distributed training and disaggregated inference performance.
Analyze and optimize communication flows across application, transport, and network layers.
Develop system software spanning communication libraries, drivers, and firmware integrations.
Collaborate with hardware, firmware, and SDK teams to co-design network features.
Validate and integrate prototypes into our companys AI infrastructure and products.

our company has been defining computer graphics, PC gaming, and accelerated computing for more than 25 years. With an outstanding legacy of innovation, driven by phenomenal technology, and extraordinary people, we are looking for a strong technical senior architect to join us in shaping the future. Senior Architects are innovators who can translate business needs into workable technology solutions. Their expertise is deep and broad. They are hands on, producing both detailed technical work and high-level architectural designs.
As a Senior architect in the Advanced Development team, you will explore technological challenges on accelerate networking and building AI data centers. Research new transport functions and semantics for optimizing AI workloads. You will also be leading architectural and development efforts across numerous technological fields, related to the modern data center, such as distributed AI and deep learning solutions, data analytics, High Performance Computing (HPC), Software Defined Networking (SDN), virtualization, storage, and more.
What youll be doing:
Enhance our company's GPU Networking offerings for accelerating AI workloads, such as our company Dynamo or our company NIXL.
Identify and evaluate new technologies, innovations and partner relationships for alignment with our technology roadmap and business value.
Lead architecture and design of such technologies.
Lead proof-of-concept development to evaluate and drive such technologies.

our company has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. Its a unique legacy of innovation thats fueled by great technologyand amazing people. We seek an Senior SW Performance Engineer to join our performance verification team. As a Performance Engineer at our company, you will have to work closely with our companys development and architecture teams responsible for Ethernet AI solution and gain a deep understanding of our company products and technologies.
What youll be doing:
Participate in an international team of software engineers working on products for testing our company products
Build automated verification environment for high-end hardware and software which is at the forefront of innovation
Identify, analyze, and report software defects, inconsistencies, and other quality issues.
Drive improvements for performance, quality, stability around SW acceleration solutions.
Stay up to date with industry standard methodologies, new technologies, and emerging trends in software verification.

In this role, you will lead a team of Machine Learning Scientists and Engineers dedicated to building, training, and deploying cutting-edge Generative AI models. This includes developing foundation models trained on our companys extensive textual data and creating fine-tuned models designed to tackle complex travel-related tasks. Your leadership will play a pivotal role in advancing the application of AI to transform the travel experience for millions of customers.
As a technical manager, you should be passionate about GenAI technology, keep up to date with recent breakthroughs in the field, define and shape the teams ML roadmap, and not be afraid to get your hands dirty with code when needed.
You are expected to be the focal point for all technical aspects, make sure your team members deliver on their tasks, and work together with other stakeholders to define and shape the roadmap of our products. You will work independently and will also be responsible for making technical decisions within your team.
When it comes to management, your expertise in handling people will motivate and inspire them to reach outstanding success! You should have experience in developing people. You will mentor and coach your team while working closely with a Product Manager.
Key Job Responsibilities and Duties:
Leadership in LLM Development- Build, Guide and mentor a team of ML scientists and ML engineers in the development, fine-tuning, and deployment of large language models (LLMs) tailored for the travel domain.
Expertise in the engineering aspects of deploying LLMs at scale with minimal latency. This includes optimizing model performance, scalability, and efficiency to meet the demands of real-time, high-traffic applications.
Define and communicate the technical vision and strategy for LLM-related initiatives, ensuring alignment with company goals and customer needs.
Foster a culture of collaboration, innovation, and excellence within the team.
Prioritize work in collaboration with Product Managers, depending on business needs and keeping stakeholders aligned at all times.
Translate machine learning vision and strategy into planning and execution, and ensure timely delivery of their plans.
Develop innovative ML models, algorithms, and engineering approaches or identify existing ones, with the potential to impact our business.
Design and execute applied research plans to understand, apply, test, evolve, and generalize these technologies into reusable frameworks.
Translate business problems into viable, reliable and robust ML and AI solutions, accounting for constraints of the production environment.
Monitor product health, performance and business impact and act accordingly when requirements are not met.
Identify underlying issues and opportunities across domains and situations that are not obviously related through application of structured thinking and logic.
Solve issues by applying methods and insights gained from a variety of disciplines, navigating a variety of environments.

As Senior Machine Learning Engineer, youll work with top notch engineers and data scientists from the team on bringing it to the next level and enabling optimal user experience. The work will focus on building, deploying and serving GenAI capabilities (Agents, Tools and the orchestration between them) using the most advanced technologies and models.
Key Job Responsibilities and Duties:
Deploying machine learning models: Design, develop and deploy in collaboration with scientists, scalable machine learning models and algorithms that provide content related insights and generative AI applications, ensuring scalability, efficiency, and accuracy.
Evaluating possible architecture solutions by taking into account cost, business requirements, emerging technologies, and technology requirements, like latency, throughput, and scale.
Generative AI Development: Contribute to the development of generative models such as GPT (Generative Pre-trained Transformer) variants or similar architectures for creative content generation, Q&A, translation or other innovative applications.
Deployment and integration: Work closely with software engineers to integrate machine learning models into production systems. Ensure seamless deployment and efficient model inference in real-time environments. Collaborate with DevOps to implement effective monitoring and maintenance strategies.
Owning a service end to end by actively monitoring application health and performance, setting and monitoring relevant metrics and acting accordingly when violated.
Maintain clean, scalable code, ensuring reproducibility and easy integration of models into production environments, including CI/CD.
Collaborate with multidisciplinary teams: Collaborate with product managers, data scientists, and analysts to understand business requirements and translate them into machine learning solutions.