Toplisters
Nvidia brand banner

Software Engineering Intern, NCCL - 2026

NvidiaShanghai, ChinaPosted 3h ago
via Workday

NVIDIA is leading the way in groundbreaking developments in Artificial Intelligence, High-Performance Computing and Visualization. The GPU, our invention, serves as the visual cortex of modern computers and is at the heart of our products and services. Our work opens up new universes to explore, enables amazing creativity and discovery, and powers what were once science fiction inventions from artificial intelligence to autonomous cars. NVIDIA is looking for phenomenal people like you to help us accelerate the next wave of artificial intelligence.

We are looking for a highly motivated software engineer intern for an exciting role in our communication libraries and network software team. The position will be part of a fast-paced crew that develops and maintains software for complex heterogeneous computing systems that power disruptive products in High Performance Computing and Deep Learning.

What you will be doing:

  • Design, implement and maintain highly-optimized communication runtimes for Deep Learning frameworks (e.g. NCCL for TensorFlow/Pytorch) and HPC programming interfaces (e.g. UCX for MPI/OpenSHMEM) on GPU clusters.

  • Participating in and contributing to parallel programming interface specifications like MPI/OpenSHMEM.

  • Design, implement and maintain system software that enables interactions among GPUs and interactions between GPUs and other system components.

  • Creating proof-of-concepts to evaluate and motivate extensions in programming models, new designs in runtimes and new features in hardware.

What we need to see:

  • You are pursuing a Ph.D. in CE/CS/EE with a strong background in computer architecture, operating systems, communication library and/or AI/ML.

  • Excellent C/C++ programming and debugging skills.

  • Strong experience with Linux.

  • Experience with parallel programming interfaces and communication runtimes.

  • Ability and flexibility to work and communicate effectively in a multi-national, multi-time-zone corporate environment.

Ways to stand out from the crowd:

  • Deep knowledge of high-performance networks like InfiniBand, RoCE etc.

  • Background with HPC applications. Experience with Deep Learning Frameworks such PyTorch, TensorFlow, JAX/XLA, vLLM/SGLang etc.

  • Experience with AI/DL communication patterns such as Expert Parallelism (EP), TP, DP, PP and how these patterns can be implemented with NCCL. Experience with CUDA kernel optimization and profiling.

  • Experience with large-scale model training and production inference software stack.

  • Strong collaborative and interpersonal skills, specifically a proven ability to effectively guide and influence within a dynamic matrix environment.

Similar roles

  • GVB logo

    Storingscoördinator Techniek

    GVB·Diemen, Netherlands

    Een storing komt binnen en op hetzelfde moment kunnen er meerdere meldingen spelen. Als storingscoördinator techniek ben jij het centrale aanspreekpunt voor technische storingen binnen de railinfrastructuur en ondersteunende technische systemen van GVB. Je werkt op het snijvlak van techniek en operatie en krijgt te maken met uiteenlopende situaties en betrokken partijen. Van een defecte installatie op een station tot…

    • Full-time
    • Highly Skilled Migrant — sponsor-tied, fast settlement
  • Ashby logo

    Support Operations Specialist

    Ashby·United States

    HELLO! I'm Allie, Head of Support at Ashby. 👋 As Ashby continues to scale its Support organization, we're investing in the operational foundation that helps our team deliver exceptional customer experiences with a high bar for quality, efficiency, and consistency. We're looking for a Support Operations Specialist to strengthen the operational engine behind Support. You'll help ensure our tooling and…

    • Remote
    • Full-time
  • The Extend Group logo

    OOSH Coordinator- Vacation Care- Bowral, NSW

    The Extend Group·Bowral, Australia

    JOB DETAILS & ROLE About your service: Bowral Public School, Bowral Hourly Rate: $47- $50 Per Hour + 12% Super. Additional payment of $2.00- $2.15 per hour available under Worker Retention Payment Scheme. * Days Required: Monday to Friday. Five days per week required during vacation care only. * Vacation Care: Starting from 28 September 2026. Upto 38 Hours per…

    • Full-time
    • Skilled Independent 189 — PR, no sponsor
  • Kubota Australia and New Zealand logo

    Storeperson (Parts)

    Kubota Australia and New Zealand·Ravenhall, Australia

    At Kubota Australia, we distribute high value, exceptional quality construction and agricultural machinery and products, including all the parts they need across their lifespans. As a company, we are going from strength to strength, continuing to expand and grow across Australia and New Zealand. As an experience Storeperson, you will love this diverse role and relish the opportunity to work…

    • Full-time
    • Skilled Independent 189 — PR, no sponsor
  • The Extend Group logo

    OSHC Relief Coordinator - Narre Warren, VIC

    The Extend Group·Narre Warren, Australia

    JOB DETAILS & ROLE Extend OSHC at Waverley Christian College, Wantirna South Hourly Rate: $47.81 to $50.00 Per Hour + 12% Super. Additional payment of $2.07 to $2.15 per hour available under Worker Retention Payment Scheme. * Before School Care: 6:45 am to 8:45 am. * After School Care: 2:15 pm to 6:30 pm. * Hours: Work 2-3 shifts or…

    • Full-time
    • Skilled Independent 189 — PR, no sponsor
  • Nvidia logo

    Senior Project Delivery Manager, NVIS

    Nvidia·Australia

    Do you thrive in a fast-paced, high-growth technology organization? Are you excited to help enterprises build next-generation digital infrastructures powered by cloud and accelerated computing? At NVIDIA, our team is passionate about parallel computing, visual computing, and AI infrastructure. We are advancing how computing solves the world’s most sophisticated challenges in science, industry, and enterprise. We are looking for driven,…

    • Full-time
    • Skilled Independent 189 — PR, no sponsor
  • Specsavers logo

    Optical Assistant Apprentice

    Specsavers·Widnes, United Kingdom

    Are you a caring individual looking to start your career in an Optics environment? You've come to the right place. As well as being passionate and motivated in everything you do, it's important that you're also a great listener and communicator - as that's what excellent service is all about. In addition to having great communication skills, you'll also be…

    • On-site
    • Full-time