Databricks Developer – Build Scalable Data Pipelines

About the role

Appbay Technologies is looking for a Databricks Developer to join its data engineering team in Jaipur. You will design, develop and maintain robust data pipelines that power business intelligence and analytics across the organization.

Key responsibilities

• Design, develop and maintain scalable data pipelines using Databricks and Apache Spark.
• Build and optimise ETL/ELT workflows for large‑volume structured and unstructured data.
• Develop data ingestion and transformation processes from databases, APIs and third‑party systems.
• Collaborate with analysts, architects and stakeholders to translate data requirements into effective solutions.
• Write clean, efficient code in Python, PySpark and SQL.
• Monitor, troubleshoot and optimise data jobs for performance, reliability and scalability.
• Implement data quality checks, validation and governance best practices.
• Work with cloud platforms such as Azure, AWS or Google Cloud to support data engineering initiatives.
• Participate in code reviews, testing, deployment and documentation activities.

Required profile

• Bachelor’s degree in Computer Science, IT, Engineering, Data Science or a related field.
• 2–4 years of experience in data engineering, big data or data analytics projects.
• Hands‑on experience with Databricks and Apache Spark.
• Strong proficiency in Python, PySpark and SQL.
• Experience building and maintaining ETL/ELT pipelines.
• Good understanding of data warehousing, data lakes and modern data architectures.
• Familiarity with relational and non‑relational databases.
• Knowledge of cloud platforms (Azure, AWS, Google Cloud).

Required skills

• Databricks
• Apache Spark
• Python
• PySpark
• SQL
• ETL / ELT
• Azure
• AWS
• Google Cloud
• Azure Databricks
• Azure Data Factory
• AWS Glue
• Delta Lake
• Unity Catalog
• Databricks Lakehouse architecture
• Relational databases
• Non‑relational databases
• Data modeling
• Performance tuning
• Optimization techniques