Analyst, Big Data Analytics & Engineering

About the role

The Data Engineer will design, build, and operate scalable data pipelines and curated datasets that power analytics products, reporting, and advanced modeling. Working closely with the Lead and cross‑functional partners, the role focuses on reliability, performance, data quality, and governance across batch and streaming workloads.

Key responsibilities

• Build and maintain robust ETL/ELT pipelines for ingestion, transformation, and aggregation of large‑scale datasets on Hadoop and enterprise data platforms.
• Develop high‑performance data processing jobs using PySpark/Spark, Python, and SQL (including Impala where applicable).
• Partner with Product and Analytics stakeholders to translate requirements into reusable, governed data models (facts/dimensions, curated layers, and semantic‑ready datasets).
• Implement and automate data quality checks, reconciliation, lineage documentation, and monitoring to ensure trust in downstream analytics and AI use cases.
• Optimize pipeline performance and cost through partitioning strategies, columnar file formats (Parquet, ORC, Delta), compute tuning, caching, and efficient query patterns.
• Contribute to CI/CD for data workflows (testing, code reviews, deployment automation), promoting engineering best practices and maintainable codebases.
• Support data governance, privacy, and security requirements (PII handling, access controls, auditability) in collaboration with platform teams.

Required profile

Required skills

• PySpark / Spark
• Python
• SQL
• Impala
• Hadoop
• ETL / ELT
• Parquet
• ORC
• Delta Lake
• CI/CD
• Data governance
• PII handling