Azure Databricks Engineer

About the role

We are looking for an experienced Azure Databricks Engineer to design, build, and optimise modern data platforms. You will work on real‑time streaming, Delta Lake architecture and Generative AI‑driven analytics, taking end‑to‑end ownership of data‑engineering deliverables from pipeline development to production support.

Key responsibilities

• Design and implement scalable data pipelines using Azure Databricks, PySpark and Azure Data Factory.
• Develop and maintain Delta Lake tables, Unity Catalog, and Medallion Architecture.
• Orchestrate workflows and jobs, ensuring reliable real‑time streaming and CDC processes.
• Collaborate with business and analytics stakeholders to translate requirements into technical solutions.
• Implement CI/CD pipelines, version control (Git, Databricks Repos) and automated data‑quality checks.
• Provide production support, troubleshoot issues and continuously improve platform performance.

Required profile

• 5–10 years of data‑engineering experience, preferably on Azure.
• Strong hands‑on expertise with Azure Databricks, PySpark/SQL and Delta Lake.
• Proven ability to own deliverables and resolve issues independently.
• Excellent communication skills for interaction with business and analytics teams.
• Valid passport with at least one year of validity and willingness to be based at any listed location.

Required skills

• Azure Databricks
• Delta Lake & Delta Table Streaming
• PySpark / SQL
• Unity Catalog
• Medallion Architecture
• Azure Data Factory
• Generative AI (LLM, RAG)
• Lakeflow (Declarative Pipelines/Connect), CDC, data‑quality validation
• Databricks AI/BI or Genie (semantic layers, certified datasets)
• CI/CD, Git, Databricks Repos, Asset Bundles