Senior Data Engineer – Azure & Databricks (Remote, India)

About the role

We are looking for an experienced Senior Data Engineer to design, build, and maintain scalable data pipelines on cloud platforms. The role is fully remote and supports multiple Indian locations, offering the chance to work with cutting‑edge Azure and Databricks technologies.

Key responsibilities

• Design and develop robust, scalable pipelines using Azure Data Factory.
• Build and optimise data processing workflows with Databricks and Apache Spark.
• Implement declarative Lakehouse‑style pipelines and best practices.
• Write and optimise complex SQL queries for transformation and analysis.
• Develop PySpark applications for large‑scale data processing.
• Collaborate with data scientists, analysts and business stakeholders to capture requirements.
• Ensure data quality, security and compliance across all pipelines.
• Monitor, troubleshoot and improve pipeline performance.
• Document technical solutions and maintain best‑practice guides.
• Mentor junior engineers and share knowledge within the team.

Required profile

• 5+ years of professional data‑engineering experience.
• 3+ years hands‑on with cloud‑based data platforms.
• Proven track record of delivering production‑grade data solutions.
• Detail‑oriented with a focus on data quality and performance.
• Strong communication skills and ability to work cross‑functionally.

Required skills

• Azure Data Factory
• Databricks
• Apache Spark
• Lakeflow / declarative pipeline frameworks
• Advanced SQL
• PySpark
• Azure (and optionally AWS or GCP)
• Git version control
• CI/CD and DevOps practices
• Data‑quality testing frameworks
• Big‑data and distributed computing concepts