Lead Data Engineer (DA)

About the role

Labcorp is looking for a Lead Data Engineer (DA) to join its Bengaluru office. The role focuses on designing, building, and maintaining high‑performance data ingestion pipelines that support the company’s diagnostic and drug‑development initiatives.

Key responsibilities

• Design and implement highly performant data ingestion pipelines from multiple sources.
• Develop scalable, reusable frameworks for ingesting diverse data sets.
• Integrate end‑to‑end pipelines to move data from source systems to target repositories while ensuring data quality and consistency.
• Work with event‑based and streaming technologies (e.g., Kafka) to process data in real time.
• Evaluate tools and present proofs of concept to stakeholders.

Required profile

• Bachelor’s degree in Computer Science, Engineering or a related field.
• 5+ years of hands‑on experience with PySpark, Python, Spark, Scala and data validation.
• 5+ years of experience implementing Snowflake or Databricks cloud data warehouses and handling XML, JSON, Avro, Parquet formats.
• 2+ years of experience with ETL processes, big‑data modeling, and CI/CD pipelines using Jenkins, Terraform, Docker, AWS and Kubernetes.
• Strong understanding of data‑warehousing concepts, including fact and dimension tables.

Required skills

• PySpark
• Python
• Spark
• Scala
• Snowflake
• Databricks
• XML
• JSON
• Avro
• Parquet
• Hadoop
• Kafka
• Airflow
• Matillion
• Tableau
• Power BI
• PL/SQL
• AWS
• Jenkins
• Terraform
• Docker
• Kubernetes