Lead Big Data Engineer

About the role

The Data semantic solution team at Capco, a Wipro company, is looking for a Big Data Engineer in Pune to unlock the potential of data assets by innovating how users access data from Big Data repositories and enforcing standards and principles in the Big Data space. The candidate will work on fast-paced data engineering solutions in the data and analytics domain.

Key responsibilities

• Develop high-quality, secure and scalable data pipelines using Spark, Scala/Python on Hadoop or object storage.
• Use new technologies and approaches to handle increasingly large data sets.
• Drive automation and efficiency in data ingestion, data movement and data access workflows.
• Implement and enforce software development standards and engineering principles in the Big Data space.
• Work closely with business stakeholders and embedded engineering teams to help them build scalable products quickly.
• Contribute ideas to ensure standards and processes are in place and improve process efficiency.
• Perform assigned tasks and production incidents independently.

Required profile

• 10+ years of experience in data warehouse projects, including multiple end-to-end implementations in a Big Data environment.
• Strong analytical skills for debugging production issues, finding root causes and implementing mitigation plans.
• Strong verbal and written communication, collaboration and organizational skills.
• Ability to multi-task across projects, interface with external and internal resources and lead junior team members technically.
• High-energy, detail-oriented, proactive and able to work under pressure independently, with flexibility for matrix-based, geographically distributed teams.

Required skills

• Building data pipelines with Spark using Scala, Python or Java on Hadoop or object storage.
• NiFi pipeline experience.
• Oracle and Netezza database experience is an added advantage.
• Ability to quickly learn new technologies and run POCs to find the best solution.