Staff Backend Engineer - K8 (Envoy)

About the role

As a Staff Backend Engineer, you will work closely with platform and product leaders to design and deliver solutions for complex infrastructure problems. You will drive the development of highly scalable, reliable, and efficient platform services while providing technical direction across teams using Java, AWS, Kafka, Kubernetes, Kubeflow, Argo CD, and gRPC.

Key responsibilities

• Architect and build Coupang’s next‑generation AI Inference Gateway platform for mission‑critical machine learning workloads.
• Design high‑performance request routing, traffic shaping, load balancing, failover, caching, and policy enforcement for inference services.
• Develop core infrastructure components in Go, Java, or Python with a focus on performance and resiliency.
• Define multi‑tenant capabilities such as authentication, authorization, quota management, and governance controls.
• Partner with ML Platform, Model Serving, Data, and Product Engineering teams to enable seamless deployment of AI workloads.
• Optimize system performance, latency, throughput, and infrastructure efficiency for large‑scale inference workloads.
• Define observability standards through metrics, tracing, logging, and SLO‑based operations.
• Investigate complex production issues, drive root‑cause analysis, and implement long‑term architectural solutions.
• Lead architecture and design reviews, raise engineering standards, and mentor senior engineers across teams.

Required profile

• 8+ years of professional software development experience.
• 5+ years designing and operating large‑scale distributed systems in production.
• Proven track record building highly available, mission‑critical platform or infrastructure services.
• Deep understanding of microservices architecture, distributed systems design, and cloud‑native technologies.
• Experience operating services on AWS, Azure, or GCP.
• Strong understanding of observability, reliability engineering, capacity planning, and production operations.

Required skills

• Strong programming expertise in Go, Java, or Python.
• Experience with Kubernetes, containerized workloads, and cloud platforms (AWS, Azure, GCP).
• Knowledge of microservices architecture, gRPC, API gateways, and service mesh technologies (Envoy, Istio, Linkerd).
• Familiarity with distributed data systems such as Kafka, Cassandra, Redis, MongoDB.
• Experience with service‑mesh and networking technologies such as Gateway API and Ingress Controllers.
• Understanding of high‑performance networking, HTTP/2, streaming protocols, and API gateway architectures.
• Experience with AI/ML inference platforms, LLM gateways, and GPU‑accelerated workloads.