Director of Engineering, Inference Serving Engine

About the role

DigitalOcean is looking for a Director of Engineering to lead the Inference Serving Engine team. You will own the design, development, and scaling of a large-language-model inference platform that powers millions of users worldwide.

Key responsibilities

• Recruit, mentor, and grow a high-performing engineering team, fostering ownership and technical excellence.
• Translate product strategy into technical roadmaps, define architecture, and ensure on-time delivery of features.
• Design and implement a distributed inference platform optimized for NVIDIA and AMD GPUs, including multi-scheduler and batch dispatching at massive scale.
• Drive observability, benchmarking, and auto-tuning frameworks, and contribute to open-source inference engines.
• Maintain production health, on-call rotation, security isolation, and fault-tolerance mechanisms such as checkpoint/restore.

Required profile

• 10+ years of software engineering experience with at least 6 years in technical leadership, preferably in AI/ML infrastructure or cloud platforms.
• Deep knowledge of distributed systems, Kubernetes at scale, and LLM inference workloads.
• Strong understanding of GPU architectures, interconnects (NVLink, PCIe) and topology-aware scheduling.
• Proven ability to communicate technical decisions to non-technical stakeholders and align cross-functional teams.

Required skills

• Distributed systems design
• Kubernetes (large clusters, custom schedulers)
• GPU programming (CUDA, ROCm)
• LLM inference frameworks (vLLM, SGLang, NVIDIA Grove)
• Container runtimes and isolation (Kata Containers, gVisor, microVMs)
• Observability, benchmarking and auto-tuning
• Checkpoint/Restore technologies (CRIU, NVIDIA cuda-checkpoint)

What we offer

• Competitive salary, performance bonus and equity participation.
• Career development resources including conference reimbursement, training budget and LinkedIn Learning access.
• Flexible time-off policy and hybrid work model.