Staff Backend Engineer – AI Inference Platform

About the role

As a Staff Backend Engineer you will partner with platform and product leaders to design and deliver highly scalable, reliable AI inference infrastructure. You will drive the architecture of Coupang’s next‑generation inference gateway, supporting mission‑critical machine‑learning workloads across cloud and on‑prem environments.

Key responsibilities

• Architect and build a high‑performance AI inference gateway platform with request routing, load balancing, failover, caching and policy enforcement.
• Design multi‑tenant capabilities such as authentication, authorization, quota management, cost attribution and governance controls.
• Develop core infrastructure components in Go, Java or Python with a focus on performance, resiliency and operational excellence.
• Lead architecture and design reviews, set engineering standards and mentor senior engineers across multiple teams.
• Define observability standards using metrics, tracing, logging and SLO‑based operations for business‑critical AI services.

Required profile

• 8+ years of professional software development experience, including 5+ years building large‑scale distributed systems.
• Strong hands‑on expertise in Go, Java or Python and proven track record of delivering highly available platform services.
• Deep understanding of microservices architecture, Kubernetes, containerized workloads and cloud‑native technologies.
• Experience operating services on AWS, Azure or GCP and implementing observability, reliability engineering and capacity planning.

Required skills

• Go, Java, Python
• Kubernetes, Docker, container orchestration
• AWS, Azure, GCP
• Kafka, distributed data systems
• gRPC, HTTP/2, high‑performance networking
• Service mesh technologies (Envoy, Istio, Linkerd)
• API gateways, microservices, observability (metrics, tracing, logging)