Senior Data Engineer - PySpark, Snowflake, AWS & GenAI

About the role

An experienced Data Engineer is sought in Hyderabad, India, with strong expertise in SQL, PySpark, Python and Snowflake. The role covers designing and maintaining scalable data pipelines and engineering frameworks, cloud data services, Medallion Architecture, ETL/ELT workflows and orchestration, with GenAI exposure as an added advantage.

Key responsibilities

• Design, develop and maintain scalable data pipelines using PySpark, SQL, Python and Snowflake.
• Build reusable data engineering frameworks for efficient data processing and transformation.
• Implement Medallion Architecture across Bronze, Silver and Gold layers.
• Develop ingestion and processing pipelines using AWS Glue and AWS Lambda.
• Build event-driven and streaming pipelines using Kafka.
• Orchestrate workflows with Apache Airflow or equivalent tools.
• Optimize SQL queries, Snowflake workloads and PySpark jobs for performance and scalability.
• Ensure data quality, reliability and consistency across pipelines.
• Explore GenAI-based solutions using LLMs, RAG and LangChain.

Required profile

• 6-12 years of relevant data engineering experience.
• Experience building scalable data pipelines and reusable engineering frameworks.
• Strong understanding of Medallion Architecture and ETL/ELT processes.
• Strong problem-solving skills and a focus on data quality and performance.

Required skills

• SQL, PySpark, Python and Snowflake (hands-on)
• Kafka, AWS Glue and AWS Lambda
• Apache Airflow or other workflow orchestration tools
• Good to have: GCP, additional AWS services, MCP, LLMs, RAG and LangChain

Candidates apply by sending their CV to the recruiter's email.