Senior Software Engineer, Matrix Multiplication

NVIDIA is hiring a Senior Software Engineer to work on inference systems software, specifically focused on matrix multiplication and GPU kernel development. The role sits on a team building libraries, code generators, and GPU kernel technologies for NVIDIA's hardware architecture. This is for engineers with deep experience in deep learning frameworks, GPU kernel optimization, and large language model inference.

What you will do

• Develop new AI systems technologies for efficient inference
• Design, implement, and optimize kernels for high-impact AI workloads
• Design and implement extensible abstractions for LLM serving engines
• Build efficient just-in-time domain-specific compilers and runtimes
• Collaborate with engineers across deep learning frameworks, libraries, kernels, and GPU architecture teams
• Contribute to open source communities including FlashInfer, vLLM, and SGLang

What they are looking for

• Masters degree in Computer Science, Electrical Engineering, or a related field, or equivalent experience; PhD preferred
• 6+ years of academic or industry experience with ML/DL systems development preferred
• Strong experience developing or using deep learning frameworks such as PyTorch, JAX, TensorFlow, or ONNX, and ideally inference engines and runtimes such as vLLM, SGLang, and MLC
• Strong Python and C/C++ programming skills
• Strong experience in GPU kernel development and performance optimizations, especially using CUDA C/C++, cuTile, Triton, or similar, with hands-on experience with matrix multiplication

Ways to stand out

• Background in domain-specific compiler and library solutions for LLM inference and training, such as FlashInfer or Flash Attention
• Expertise in inference engines like vLLM and SGLang
• Expertise in machine learning compilers such as Apache TVM or MLIR
• Open source project ownership or contributions

Pay and benefits

The base salary range is 184,000 USD to 287,500 USD per year. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. You will also be eligible for equity and benefits.

Location and application details

The role is open to candidates in Canada and the United States. Applications will be accepted at least until July 18, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes.

About NVIDIA

NVIDIA is an equal opportunity employer. The company states that it values diversity in its current and future employees and does not discriminate on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status, or any other characteristic protected by law.