Data & AI Architect – AWS Databricks & GenAI

About the role

We are looking for an experienced Data & AI Architect to design and implement scalable hybrid analytics and AI solutions on the Databricks platform. The role will focus on building robust data architectures, secure pipelines, and reliable workflows that enable advanced insights for business teams in a collaborative onsite environment.

Key responsibilities

• Design end‑to‑end data and analytics architectures leveraging Databricks SQL, Delta Lake, and PySpark.
• Develop secure data ingestion patterns from Amazon S3 and other enterprise sources.
• Define best practices for Delta Lake table management, including partitioning and optimization.
• Implement Databricks Workflows to orchestrate batch and near‑real‑time pipelines.
• Apply GenAI fundamentals to create solution patterns that augment analytics and reporting.
• Collaborate with BI and AI product teams to build semantic layers and reusable data models.
• Create Python and PySpark framework components for logging, error handling, and testing.
• Review designs, notebooks, and configurations to ensure compliance with security and regulatory standards.
• Partner with security and operations teams to align data architectures with encryption, masking, and monitoring controls.
• Guide optimization of Databricks clusters, job configurations, and SQL queries to control cost and maximize performance.

Required profile

• Proven experience as a data or AI architect with strong expertise in GenAI fundamentals.
• Hands‑on experience with Python, PySpark, Databricks, and Amazon S3.
• Ability to design scalable hybrid analytics solutions and work effectively in an onsite team.

Required skills

• Python
• PySpark
• Databricks (SQL, Delta Lake, Workflows)
• Amazon S3
• SQL
• GenAI fundamentals