Senior Data Engineer – Regulatory Data Pipelines

About the role

We are looking for a Senior Data Engineer to design and operate data pipelines that ingest, transform, and enrich legal and regulatory documents. The role focuses on building knowledge‑graph‑ready data assets that power AI and retrieval‑augmented generation (RAG) solutions.

Key responsibilities

• Develop and maintain end‑to‑end ingestion pipelines for heterogeneous source formats such as XML, HTML, and structured exports.
• Normalize and transform content using XSLT and custom scripts to preserve semantic provenance.
• Implement embedding generation, indexing, and enrichment workflows for downstream AI/RAG systems.
• Design RDF‑based knowledge representations and expose them via SPARQL endpoints.
• Integrate storage and processing components across containerised and cloud environments (Docker, Kubernetes, Azure Storage).
• Support event‑driven architectures using Apache Camel, message brokers, and CloudEvents.
• Ensure pipelines are reproducible, maintainable, and ready for operational hand‑over.

Required profile

• 6‑8 years of hands‑on experience as a Data Engineer, preferably in regulatory or compliance domains.
• Strong background in building document‑centric pipelines and knowledge graphs.
• Experience with container orchestration (Docker, Kubernetes) and cloud storage services.
• Ability to work independently and collaborate with AI and domain experts.

Required skills

• Python
• Docker / Docker Compose
• Kubernetes
• RDF and SPARQL
• XSLT
• Java (for RDF handling)
• Embeddings pipelines / vector preparation
• Azure Storage (or equivalent cloud storage)
• Apache Camel
• Git
• Docling (or similar document conversion tool)
• CloudEvents
• Kafka or other message brokers
• Dev Containers
• GitOps