Data Scientist – Databricks & AI Specialist

About the role

We are seeking a highly skilled Data Scientist with deep expertise in Databricks, Machine Learning, and AI‑driven analytics to join our growing data organization. In this role you will design, build, and deploy scalable models and data workflows that power enterprise insights, automation, and decision‑making, collaborating across engineering, analytics, and business teams.

Key responsibilities

• Develop, train, and deploy machine learning models using Databricks notebooks, MLflow, and the Lakehouse architecture.
• Build scalable ETL/ELT pipelines leveraging Delta Lake, PySpark, and Databricks workflows.
• Implement AI/LLM‑based solutions such as retrieval‑augmented generation, vector search, and enterprise agent workflows.
• Partner with data engineering to optimise datasets for analytics, modelling, and real‑time inference.
• Conduct exploratory data analysis, feature engineering, and statistical modelling to uncover actionable insights.
• Use MLflow for experiment tracking, model versioning, and lifecycle management.
• Collaborate with business stakeholders to translate ambiguous problems into measurable, data‑driven solutions.
• Deploy models into production using Databricks Model Serving, serverless compute, or API endpoints.
• Ensure governance, security, and compliance through Unity Catalog and enterprise data standards.
• Continuously evaluate new AI/ML technologies and recommend platform improvements.

Required profile

• Bachelor’s or Master’s degree in Data Science, Computer Science, Statistics, or a related field.
• 3–7+ years of experience building machine learning models in Python (Pandas, Scikit‑learn, PySpark, TensorFlow or PyTorch).
• Hands‑on experience with Databricks, including notebooks, Delta Lake, MLflow, and Databricks SQL.
• Strong understanding of Lakehouse architecture, distributed computing, and scalable data processing.
• Experience deploying ML models into production environments.
• Proficiency in SQL and Python for data manipulation and analysis.
• Familiarity with LLMs, embeddings, vector databases, or AI agent frameworks.
• Ability to communicate complex technical concepts to non‑technical stakeholders.

Required skills

• Python
• Pandas
• Scikit‑learn
• PySpark
• TensorFlow
• PyTorch
• SQL
• Databricks (notebooks, Delta Lake, MLflow, Databricks SQL)
• Lakehouse architecture
• MLOps and CI/CD
• Azure, AWS, GCP
• Unity Catalog, RBAC, ABAC
• Vector Search, LLMs, embeddings, vector databases
• SAP BDC, Snowflake