Data Engineer (ETL Specialist) - VFX Data Pipeline

About the role

BOT VFX, a visual effects services company with global clients, is hiring a Data Engineer (ETL Specialist) for a full-time contract role, strictly on location in Chennai, India. The role transforms petabytes of unstructured, legacy VFX data (heavy video plates, multi-layered OpenEXR sequences and proprietary DCC workfiles) into structured, sanitized and compressed datasets. It reports to the CIO and works closely with the IT/IO and Creative Innovation teams.

Key responsibilities

• Lead the architecture, implementation and automation of scalable ETL/ELT pipelines that move and process tens of terabytes of unstructured media data daily.
• Implement automated "Purge & Keep" business rules to clean directory trees, removing render caches, temporary scratch files and duplicates while preserving core visual assets.
• Build high-throughput pipelines to transcode, compress and restructure image sequences (linear EXR, DPX) and video formats into AI-friendly data structures such as WebDataset shards.
• Integrate and normalize metadata scraped from VFX workfiles (Silhouette, Nuke) and bind it deterministically to its visual assets.
• Manage fast data movement across tiered storage (hot NVMe/SSD staging, warm object storage, deep cloud and tape archives).
• Set up data quality checks to catch corrupted frames, missing frame ranges and accidental inclusion of restricted or sensitive assets.

Required profile

• 3-5 years of experience as a Data Engineer building large-scale, automated ETL/ELT pipelines, with a track record of unstructured data, heavy media assets or computer vision datasets.
• Solid problem-solving under strict hardware constraints, balancing egress speeds with staging capacity.
• Ability to communicate across multidisciplinary teams such as VFX technical directors, systems engineers and AI research teams.

Required skills

• Python and bash/shell scripting, with heavy data processing frameworks such as Apache Spark, Ray or Dask.
• Containerization (Docker, Kubernetes) and cloud or on-premise block and object storage.
• Preferred: basic familiarity with VFX file structures, image sequence formatting and tools such as FFmpeg and the OpenEXR toolset.