Inference & Performance Engineer
Engineers in this role optimize how AI models train and run in production, focusing on the full stack from GPU kernels and inference runtimes to distributed training systems and cluster orchestration. They bridge research breakthroughs and production reality, writing CUDA/Triton kernels, tuning serving frameworks like vLLM, profiling end-to-end inference pipelines, and solving the performance and efficiency challenges that emerge when deploying transformer models at scale. They typically sit in infrastructure, platform, or systems teams at AI labs and companies, working closely with researchers and product teams to ensure models meet latency, throughput, and cost targets in real deployments.
Measured across 157 of 159 open postings.
This role is advertised at 3 levels, so a single figure for the role would describe none of them. Experience and pay are the midpoints for each level on its own.
| Level | Share | Median years | Median pay |
|---|---|---|---|
| Mid | 37%(58) | 3 | $308k |
| Senior | 20%(32) | 5 | — |
| Staff / Principal | 38%(59) | 8 | $300k |
A dash means too few postings stated it to report a midpoint. Most companies do not publish a salary band, so pay is indicative rather than a market rate. 2 levels with fewer than 10 open postings are not shown.
“own model and endpoint optimization from model artifacts through production deployment”
“AI fluency as a practitioner: You use AI tools seriously in your own work and have opinions about where they help and where they don't.”
“making large language models run faster, cheaper, and more reliably in production. That means owning the inference stack end to end”
“dive into the complexities of distributed inference, compiler-aware optimization, and novel inference-time computation strategies”
Requirements are a share of every open posting, so a role missing from this list is one where almost nobody asks. Work mode is different: many postings never say, so that figure counts only the ones that do. A posting stops being advertised when it is filled, cancelled or reorganised, so read the last figure as how long these stay on the market, not as time to hire.
Skills
What companies are looking for in this role.
Systems performance optimization
Distributed systems architecture
Monitoring and observability
Backend and API engineering
ML systems development
Large-scale model training
Hardware and embedded systems
Cloud infrastructure operations
Incident response and reliability
Developer platform engineering
Infrastructure automation and IaC
Inference performance optimization
AI and GPU infrastructure operations
ML platform engineering
LLM deployment support
AI evaluation design
Mentoring and code review
Technology
The tools and technologies that define this role.
Open Jobs
159 open Inference & Performance Engineer jobs across 44 companies.
Other Engineering roles
General-purpose software engineering roles focused on building and maintaining software systems. Covers generalist SWE positions that don't clearly fall into frontend, backend, fullstack, or other specialized tracks.
Engineers focused on server-side systems, APIs, services, and data processing pipelines. Includes roles explicitly labeled as backend or server-side development.
Engineers specializing in user-facing interfaces, web applications, and client-side development. Includes UI/UX engineering and web development roles.
Engineers working across the entire application stack, handling both frontend and backend responsibilities.
Engineers building and maintaining internal platforms, cloud infrastructure, compute systems, and developer tooling.