GCP Data Engineer

About the role

Capco is seeking a seasoned GCP Data Engineer to design, build and support large‑scale data processing systems on Google Cloud. The role involves creating both batch and real‑time streaming pipelines for major banking and financial services clients.

Key responsibilities

• Design, develop and maintain distributed data processing systems using GCP services such as BigQuery, Dataflow, Pub/Sub and Cloud Composer.
• Build and optimise batch and streaming data pipelines, integrating Apache Kafka and other real‑time technologies.
• Lead feature teams, coordinate with clients and ensure delivery of high‑quality outcomes.

Required profile

• 8‑12 years of hands‑on experience in GCP data engineering and production‑grade data pipelines.
• Strong coding skills in Python, SQL and at least one of Java, Scala or Go.
• Proven experience with distributed frameworks like Spark, Flink, Beam or Hadoop.
• Familiarity with containerisation (Docker, Kubernetes) and CI/CD tools such as Jenkins, GitHub and Spinnaker.

Required skills

• Google Cloud Platform services: BigQuery, Dataflow, Cloud Storage, Pub/Sub, Cloud Composer.
• Apache Kafka and real‑time streaming technologies.
• Programming languages: Python, SQL, Java, Scala, Go.
• Big data frameworks: Spark, Flink, Beam, Hadoop.
• Container platforms: Docker, Kubernetes.
• CI/CD tools: Jenkins, GitHub, Spinnaker.