Senior Data Engineer – Big Data & Analytics

About the role

Mastercard is seeking a Senior Data Engineer to design, build, and operate scalable data pipelines that power analytics products, reporting, and advanced modeling. The role works closely with product, data science, and platform teams to ensure reliability, performance, data quality, and governance across batch and streaming workloads.

Key responsibilities

• Build and maintain robust ETL/ELT pipelines for ingestion, transformation, and aggregation of large‑scale datasets on Hadoop and enterprise data platforms.
• Develop high‑performance data processing jobs using PySpark/Spark, Python, and SQL (including Impala where applicable).
• Partner with product and analytics stakeholders to translate requirements into reusable, governed data models (facts, dimensions, curated layers, and semantic‑ready datasets).
• Implement and automate data quality checks, reconciliation, lineage documentation, and monitoring to ensure trust in downstream analytics and AI use cases.
• Optimize pipeline performance and cost through partitioning strategies, columnar file formats (Parquet, ORC, Delta), compute tuning, caching, and efficient query patterns.
• Contribute to CI/CD for data workflows, including testing, code reviews, and deployment automation.

Required profile

• Senior‑level experience in data engineering, preferably in a large‑scale, enterprise environment.
• Proven ability to work with cross‑functional product, data science, and platform teams.
• Experience delivering reliable, high‑performance data pipelines with strong focus on data quality and governance.

Required skills

• Hadoop ecosystem
• PySpark / Spark
• Python
• SQL (including Impala)
• ETL / ELT pipeline development
• Columnar file formats: Parquet, ORC, Delta
• Data modeling (facts, dimensions, curated layers)
• Data quality frameworks and monitoring
• CI/CD for data workflows