📢 New: get today's jobs on our WhatsApp Channel
Jobiglo

No results.

Senior Data Engineer (Databricks & Spark)

Koantek · Hyderabad

New
Senior 🇬🇧 English
Databricks Apache Spark PySpark Scala SQL Delta Lake Unity Catalog Azure DevOps Git Kafka Spark Streaming ETL/ELT

Job description

About the role

Koantek is looking for an experienced Full Stack Data Engineer with 5-6 years of industry experience, preferably in the manufacturing or energy sectors. The role focuses on developing and maintaining scalable data solutions on Databricks and related technologies.

Key responsibilities

  • Develop and deploy end-to-end data pipelines and solutions on Databricks, integrating with various data sources and systems.
  • Collaborate with cross-functional teams to deliver effective BI solutions.
  • Implement data ingestion, transformation and processing workflows using PySpark/Scala, SQL and Databricks notebooks.
  • Build and maintain data models, ETL/ELT processes and APIs and data services supporting analytics, reporting and application integration.
  • Ensure data quality, integrity and security across the data lifecycle, and monitor, troubleshoot and optimise pipeline performance in the cloud.
  • Integrate data from APIs, relational and non-relational databases, IoT devices and external providers, following data governance, security and compliance policies.

Required profile

  • Bachelor's or Master's degree in Computer Science, Engineering or a related field.
  • 5-6 years of hands-on data engineering experience, with a strong focus on Databricks and Apache Spark.
  • Experience building and maintaining ETL/ELT pipelines in production environments.
  • Strong communication skills and the ability to work independently or in a team.

Required skills

  • Python/PySpark and/or Scala; strong SQL skills.
  • Experience with Delta Lake, Unity Catalog or similar technologies.
  • Knowledge of data structures, algorithms, data modelling, ETL/ELT and data warehousing.
  • Familiarity with Kafka, Spark Streaming and Databricks Genie.
  • CI/CD tools such as Azure DevOps and Git.

Questions fréquentes

Le salaire n'est pas communiqué publiquement par le recruteur. Vous pouvez postuler et négocier directement avec Koantek.
Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.
Source : ats:zoho_recruit

Why are you reporting this job?

Thank you for your report. We will review this job.

Explore further

Salaries, guides and searches in India.

Apply in 30 seconds

Enter your email to apply. An account will be created automatically.

Apply now →

By continuing, you accept our terms of use.

Already have an account? Login

A question about this job?

Ask it here: you will get the full job summary by e-mail, right away.

Published 1 day ago

Expires 1 month from now

4 views · 0 interested

Boost your chances

Upload your CV — we will match you with relevant openings.

Analyzing your CV...

Koantek

Hyderabad