Architect and Lead Data Engineer – Data Architecture, Governance

Job Description

Senior Data Engineer | Optiver

The Tone:
This is a full-time Senior Data Engineer role at Optiver, a leading global market-making institution, with location in Australia. Optiver’s mission is to continuously improve financial markets by injecting liquidity, providing accurate pricing, and increasing transparency. This position is central to establishing a unified, governed data foundation across critical business functions like Finance, People, Procurement, and Compliance. The work will involve replacing fragmented, siloed data systems with a modern, greenfield stack, providing the essential data layer for advanced AI tools, conversational analytics, and robust reporting.

The TL;DR
• Role: Full Time
• Type: Full-time
• Location: In-person – Australia
• Team: Part of a team establishing Optiver’s business intelligence foundations.
• Mission: Design and build a unified semantic layer that consolidates fragmented business data across multiple domains into a single truth.
• Tech Stack: Databricks, dbt, SQL, PySpark, Unity Catalog, DataHub, Databricks Genie, Delta table.

What You’ll Actually Do
• Architect: Own the architecture and delivery of shared data products across Finance, People, Procurement, and Compliance, from design decisions through to production.
• Build: Construct and evolve the semantic layer that powers reporting, self-service analytics, conversational analytics, and Databricks Genie.
• Implement: Design and build scalable data pipelines, data models, and governed data products using Databricks, dbt, SQL, and PySpark.
• Govern: Implement end-to-end governance capabilities including Unity Catalog access controls, column-level security, data classification, lineage, and data quality standards.
• Translate: Convert complex and ambiguous senior stakeholder requirements into production-grade data solutions, overseeing the process from conversation to deployment.

The Must-Haves
• Background: Senior-level professional with 8+ years of experience designing and building production data solutions, demonstrating a strong track record of architectural ownership and leading complex technical initiatives end-to-end.
• Experience: Deep experience with Databricks, Lakehouse architectures, or comparable modern data technologies, including Delta table design, Unity Catalog governance, compute trade-offs, and downstream BI or AI workloads. Proven experience implementing governance at scale, covering access control, PII handling, column-level security, data lineage, and data quality management in production environments. Additionally, experience building trusted business-facing data products within domains such as Finance, People/HR, Procurement, or Operations is required.
• Skills: Expert-level SQL and data transformation skills, coupled with strong hands-on experience in dbt, PySpark, or both. The ability to design data products for scale, write tests without friction, and articulate clear opinions on tool limitations is essential. Furthermore, understanding business logic well enough to challenge unclear or incorrect requirements is critical.