Manager, Databricks Data Engineer - GCC Advisory

About the role

PwC is looking for an experienced Manager, Databricks Data Engineer within its GCC Advisory practice in Bangalore. The role centres on designing, building and operating scalable, high-performance data pipelines on the Databricks Lakehouse Platform, working closely with analytics, AI/ML and business teams.

Key responsibilities

• Design, build and maintain end-to-end data pipelines using Databricks (PySpark / Spark SQL)
• Implement batch and incremental data processing with Delta Lake and multi-hop architecture
• Develop and optimise Databricks notebooks, jobs and workflows
• Ingest, transform and curate large-scale structured and semi-structured datasets
• Support analytics, reporting and downstream data consumption use cases
• Ensure data quality, reliability, lineage and governance
• Collaborate with data scientists, analysts and architects on AI/ML workloads
• Optimise Spark jobs for performance and cost efficiency
• Follow enterprise security, access control and compliance standards
• Provide production support and troubleshoot pipeline issues
• Document technical designs, data flows and operational runbooks
• Mentor junior engineers and share best practices

Required profile

• 7+ years of experience as a Data Engineer with strong Databricks expertise
• Bachelor's or Master's degree in Computer Science, Engineering or a related field (60% above)
• Experience in Agile teams

Required skills

• Apache Spark, PySpark and Spark SQL
• Delta Lake and Lakehouse architecture
• Advanced SQL
• ETL/ELT patterns and data warehousing concepts
• At least one cloud platform (Azure / AWS / GCP)
• Distributed computing concepts
• Preferred: Unity Catalog, Auto Loader or streaming frameworks, CI/CD for data pipelines, Python for data engineering, Databricks certification