Data Engineer – Build and Scale Vulnerability Data Pipelines

About the role

We are looking for a Data Engineer to assist in scaling a vulnerability data pipeline that meets enterprise standards. The role focuses on designing, implementing, and maintaining robust data processing workflows using modern cloud and container technologies.

Key responsibilities

• Develop and enhance data pipelines for vulnerability data ingestion, transformation, and storage.
• Implement scalable solutions using Python and Apache Spark to handle large‑scale data workloads.
• Containerize and orchestrate pipeline components with Docker and Kubernetes.
• Automate infrastructure provisioning and configuration management using Terraform.
• Collaborate with cloud teams to deploy and optimize solutions on AWS, GCP, and Azure environments.

Required profile

• Strong knowledge of Python programming.
• Hands‑on experience with Apache Spark for big‑data processing.
• Familiarity with container technologies (Docker) and orchestration platforms (Kubernetes).
• Experience using Terraform for infrastructure as code.
• Good understanding of cloud platforms such as AWS, GCP, and Azure.

Required skills

• Python
• Apache Spark
• Kubernetes
• Docker
• Terraform
• AWS
• GCP
• Azure