Tō kamupene hou | Your new company

An organisation that works on meaningful data services that supports the entire of New Zealand.

Tō tūranga hou | Your new role

• Design, build, and operate data pipelines on Databricks using Python, PySpark, R, SQL, Delta Lake, and endorsed data tooling
• Develop and maintain production‑ready R and Python notebooks, libraries, and scheduled jobs
• Implement governed data products, including data ingestion, schema enforcement, validation, lineage, and metadata management
• Build batch pipelines using Databricks Workflows, Jobs, and Delta Live Tables where appropriate
• Collaborate with technical and statistical stakeholders to accurately implement transformation and business logic
• Implement data quality checks and validation frameworks (e.g. Great Expectations or equivalent)

Ngā uara me ngā pūkenga e angitu ai koe | What you'll need to succeed

• Proven experience building data pipelines on Databricks or similar lakehouse platforms
• Strong hands‑on experience with Python and R in production environments
• Advanced PySpark and Spark SQL development capability
• Experience packaging, testing, and deploying reusable R or Python code
• Solid understanding of Delta Lake concepts, including versioning, history, and schema evolution
• Experience implementing data quality and validation frameworks

Contract duration – Until June 2026 (with potential extension)