Senior Data Engineer – AWS & Databricks

About the role

Join a global healthcare biopharma leader in Hyderabad as a Senior Data Engineer, designing and building trusted, scalable data platforms on AWS and Databricks. You will lead the development of batch and streaming pipelines, ensure data quality, and mentor a team of engineers.

Key responsibilities

• Design, develop, and operate end‑to‑end ETL/ELT pipelines that ingest data into an AWS‑based lakehouse and data warehouse.
• Build curated, dimensional datasets (star/snowflake, SCD, conformed dimensions) for BI and self‑service analytics.
• Implement data quality controls, observability, and SLAs while maintaining data catalog, lineage, and metadata.
• Develop transformations on Databricks using Python/PySpark and Spark SQL, optimizing for performance and cost.
• Provision and manage cloud resources with Terraform, Docker, and CI/CD pipelines.

Required profile

• 8+ years of data engineering experience building production‑grade platforms.
• Strong AWS expertise (S3, Glue, Lambda, Step Functions, EMR/ECS/Fargate, CloudWatch).
• Proficiency in Python, PySpark, and complex SQL.
• Hands‑on experience with Databricks Lakehouse and a cloud data warehouse (e.g., Redshift).
• Experience working in Agile teams and mentoring junior engineers.

Required skills

• AWS
• Databricks
• Python
• PySpark
• SQL
• CI/CD
• Terraform
• Docker
• GitHub