Staff Software Engineer (Data Platform)

About the role

Databricks is looking for a Staff Software Engineer to lead the Data Platform team in Bengaluru. You will help build the Data Intelligence Platform that powers decision‑making across the company, working closely with product, data science and applied AI teams.

Key responsibilities

• Design, implement and operate the Databricks metrics store for company‑wide metric aggregation and high‑performance queries.
• Build and run the cross‑company Data Intelligence Platform, balancing data protection with shareability as Databricks transitions to a public company.
• Develop tooling and infrastructure to manage Databricks on Databricks at scale across multiple clouds, geographies and deployment types, including CI/CD pipelines and test frameworks.
• Design the base ETL framework used by all internal data pipelines.
• Partner with engineering teams to define long‑term product vision and requirements.
• Build reliable multi‑geo data pipelines using Databricks, partner products and open‑source tools.
• Establish conventions and create APIs for telemetry, debugging, feature and audit event logging.
• Represent Databricks at academic and industry conferences and events.

Required profile

• 12+ years of industry experience.
• 4+ years building large‑scale distributed systems.
• 5+ years providing technical leadership on large ETL, metrics store or infrastructure projects.
• Experience building, shipping and operating reliable multi‑geo data pipelines at scale.
• Hands‑on experience with workflow/orchestration frameworks such as Airflow and DBT.
• Experience with large‑scale messaging systems like Kafka or RabbitMQ.
• Excellent cross‑functional communication and consensus‑building skills.
• Passion for data infrastructure and enabling others to access data easily.

Required skills

• Airflow
• DBT
• Kafka
• RabbitMQ
• CI/CD processes
• Test frameworks for pipelines
• Infrastructure‑as‑code tooling
• ETL framework design
• Metrics store design
• Databricks platform