Data Engineer (PySpark)

About the role

We are looking for a Data Engineer to design, develop, and maintain robust data solutions that support data generation, collection, and processing. The role focuses on building efficient pipelines, ensuring data quality, and implementing ETL processes to move data across systems.

Key responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark.
• Implement and manage ETL processes for data migration and deployment.
• Monitor, troubleshoot, and optimise pipeline performance and reliability.
• Ensure data quality through validation and quality‑assurance methods.
• Document data workflows and processes for knowledge sharing.
• Collaborate with cross‑functional teams to gather requirements and deliver solutions.
• Provide guidance and support to junior team members.

Required profile

• Minimum 3 years of professional experience with PySpark.
• 15 years of full‑time education (equivalent to a bachelor’s degree).
• Ability to work independently and become a subject‑matter expert.
• Strong problem‑solving and analytical skills.
• Effective communication for cross‑team collaboration.

Required skills

• Proficiency in PySpark.
• Experience building and managing scalable data pipelines.
• Knowledge of distributed computing frameworks.
• Ability to troubleshoot complex data workflows.
• Familiarity with data storage and integration techniques.
• Understanding of data quality assurance and validation methods.