Senior Software Engineer, AI Inference Systems
We are seeking highly skilled and motivated software engineers to join us and build AI inference systems that serve large-scale models with extreme efficiency. You’ll architect and implement high-performance inference stacks, optimize GPU kernels and compilers, drive industry benchmarks, and scale workloads across multi-GPU, multi-node, and multi-cloud environments. You’ll collaborate across inference, compiler, scheduling, and performance teams to push the frontier of accelerated computing for AI.
What you’ll be doing:
Contribute features to vLLM that empower the newest models with the latest NVIDIA GPU hardware features; profile and optimize the inference framework (vLLM) with methods like speculative decoding, data/tensor/expert/pipeline-parallelism, prefill-decode disaggregation.
Develop, optimize, and benchmark GPU kernels (hand-tuned and compiler-generated) using techniques such as fusion, autotuning, and memory/layout optimization; build and extend high-level DSLs and compiler infrastructure to boost kernel developer productivity while approaching peak hardware utilization.
Define and build inference benchmarking methodologies and tools; contribute both new benchmark and NVIDIA’s submissions to the industry-leading MLPerf Inference benchmarking suite.
Architect the scheduling and orchestration of containerized large-scale inference deployments on GPU clusters across clouds.
Conduct and publish original research that pushes the pareto frontier for the field of ML Systems; survey recent publications and find a way to integrate research ideas and prototypes into NVIDIA’s software products.
What we need to see:
Bachelor’s degree (or equivalent expeience) in Computer Science (CS), Computer Engineering (CE) or Software Engineering (SE) with 7+ years of experience; alternatively, Master’s degree in CS/CE/SE with 5+ years of experience; or PhD degree with the thesis and top-tier publications in ML Systems, GPU architecture, or high-performance computing.
Strong programming skills in Python and C/C++; experience with Go or Rust is a plus; solid CS fundamentals: algorithms & data structures, operating systems, computer architecture, parallel programming, distributed systems, deep learning theories.
Knowledgeable and passionate about performance engineering in ML frameworks (e.g., PyTorch) and inference engines (e.g., vLLM and SGLang).
Familiarity with GPU programming and performance: CUDA, memory hierarchy, streams, NCCL; proficiency with profiling/debug tools (e.g., Nsight Systems/Compute).
Experience with containers and orchestration (Docker, Kubernetes, Slurm); familiarity with Linux namespaces and cgroups.
Excellent debugging, problem-solving, and communication skills; ability to excel in a fast-paced, multi-functional setting.
Ways to stand out from the crowd
Experience building and optimizing LLM inference engines (e.g., vLLM, SGLang).
Hands-on work with ML compilers and DSLs (e.g., Triton, TorchDynamo/Inductor, MLIR/LLVM, XLA), GPU libraries (e.g., CUTLASS) and features (e.g., CUDA Graph, Tensor Cores).
Experience contributing to containerization/virtualization technologies such as containerd/CRI-O/CRIU.
Experience with cloud platforms (AWS/GCP/Azure), infrastructure as code, CI/CD, and production observability.
Contributions to open-source projects and/or publications; please include links to GitHub pull requests, published papers and artifacts.
At NVIDIA, we believe artificial intelligence (AI) will fundamentally transform how people live and work. Our mission is to advance AI research and development to create groundbreaking technologies that enable anyone to harness the power of AI and benefit from its potential. Our team consists of experts in AI, systems and performance optimization. Our leadership includes world-renowned experts in AI systems who have received multiple academic and industry research awards. If you’re excited to build systems, kernels, and tools that make large-scale AI faster, more efficient, and easier to deploy, we’d love to hear from you.
#LI-Hybrid
Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. For Poland: The base salary range is 292,500 PLN - 507,000 PLN for Level 4, and 375,000 PLN - 650,000 PLN for Level 5.


Similar roles
Quantitative Researcher / Signal Generator - Alpha Signal Program
Moreton Capital Partners·Germany
QUANTITATIVE RESEARCHER / SIGNAL GENERATOR - ALPHA SIGNAL PROGRAM ABOUT US We're a small, elite team of AI researchers and engineers building predictive models across commodities, prediction markets, and global macro. We're well capitalized, we move fast, and we care about one thing above all: whether a signal actually works. The Alpha Signal Program is how we bring in talent…
- Remote
- Contract
- Blue Card — tied; settle 21–33mo
QA Engineer
veritree·Vancouver, Canada
VERITREE AND JOB OVERVIEW veritree is an award-winning climate tech start-up based in Vancouver. Launched in 2021, our technology measures and verifies the impact of global restoration efforts from the ground up. We are on a mission to plant 1 billion verified trees by 2030, collaborating with businesses, planting organizations, and consumers who believe in the transformative power of verified…
- Hybrid
- Full-time
- Express Entry — PR day one, no employer
WerkstudentIn AI Agent Developer
logen.ai·Berlin, Germany
Du hast Erfahrung mit KI und willst sie an echten Kundenprojekten einsetzen? Bei uns baust du AI Agents, die in Produktion gehen. Keine Proof-of-Concepts für die Schublade. Wir sind ein KI-Startup aus Berlin, spezialisiert auf Automatisierung im Kundenservice. Wir bauen AI Agents, Voicebots und Automatisierungslösungen für Unternehmen im deutschen Mittelstand. Gegründet 2023 von Oscar Schwarz und Ludwig Sickert, aktuell ein…
- On-site
- Full-time
- Blue Card — tied; settle 21–33mo
Account Manager (m/w/d) - Eschborn
TimeLapse Go·Eschborn, Germany
Über TimeLapse Go TimeLapse Go entwickelt intelligente Lösungen für die digitale Baustellendokumentation. Mithilfe von künstlicher Intelligenz (KI), vernetzten Technologien (IoT), hochauflösenden Kameras und Zeitrafferlösungen begleiten wir Bauunternehmen, Projektentwickler, Industrieunternehmen und öffentliche Auftraggeber bei der Überwachung, Dokumentation und Analyse ihrer Bauprojekte in Echtzeit. Im Zuge unseres Wachstums auf dem deutschen Markt suchen wir einen Account Manager (m/w/d), der unsere Kunden nach…
- On-site
- Full-time
- Blue Card — tied; settle 21–33mo
Werkstudent (m/w/d) Projektmanagement
Streamergy GmbH·Augsburg, Germany
Die Streamergy bringt Energie und Daten zusammen. Als innovativer Software-, Ingenieurs- und Lösungsanbieter digitalisieren wir Prozesse im Bereich der Erneuerbaren Energien, des Netzanschlusses und der Anlagenzertifizierung. Mit unserer 360°-Lösung begleiten wir unsere Kunden von der Netzanschluss-Beantragung bis in die Betriebsphase – und sorgen dafür, dass Photovoltaik- und Batteriespeicherprojekte effizient, sicher und reibungslos ans Netz gehen. Zur Verstärkung unseres Teams suchen…
- On-site
- Full-time
- Blue Card — tied; settle 21–33mo
Embedded Systems Engineer (m/w/d)
PCB Arts GmbH·Fürth, Germany
**"We drive the integration of intelligent embedded systems - pioneering innovation by processing data on the Edge! **Mit unseren Produkten beschleunigen wir die Entwicklung von Edge-AI Anwendungen in Europa. Von der Konzeption bis zur Umsetzung setzen wir auf deine Kreativität und auf deine Fähigkeiten in Koordination & Management. Du willst den nächsten Technologiesprung in Europa aktiv gestalten? Dann werde Teil…
- On-site
- Full-time
- Blue Card — tied; settle 21–33mo
Test Engineer (Java/.Net)
Solirius Reply·London, United Kingdom
About Us: Solirius Reply, part of the Reply Group, is a technology consultancy and digital transformation partner that helps organisations solve complex challenges through strategy, design, engineering, and delivery. We work closely with our clients to deliver secure, accessible, user-focused services that evolve with their needs. By combining deep technical expertise with people-centred design, we create solutions that deliver meaningful,…
- Hybrid
- Full-time
Platform Engineer (Mid-level)
Team17 Digital·Wakefield, United Kingdom
About the Role We are seeking a Platform Engineer (Mid-level) to join our Platform Engineering function. This hands-on technical role focuses on the platforms, infrastructure, automation, and tooling that support our development teams and business operations. The successful candidate will help maintain, improve, and modernise our platform estate across cloud services, infrastructure, monitoring, automation, CI/CD, and operational tooling. The role…
- Full-time
