Need only USC/GC
Data Engineering (DE) Lead-Remote
Platform Focus
GCP, MDM, Dataflow
Role Description
Own data-engineering delivery on GCP/DataFlow, including ingestion, transformation, standardisation, reconciliation, publication and operational engineering standards.
Key Responsibilities
Define reusable DataFlow/GCP ingestion and transformation patterns.
- Implement source ingestion from Epic, ELMS, Oracle, terminology and other approved sources.
- Build raw/source-preserving, standardised/cleansed and curated/mastered processing layers.
- Implement standardisation, validation, transformation and reference-data enrichment.
- Build reconciliation controls between source, MDM/RDM and downstream publication.
- Build publication pipelines to GCP data products, APIs, events and tables/files.
- Implement replay, restart, idempotency, audit and monitoring; optimise performance and cost.
- Establish CI/CD, code quality and production support procedures.
Required Experience & Skills
- 8+ years in data engineering, hands-on with GCP and DataFlow (or equivalent).
- Large-scale batch, incremental and event-driven pipeline experience.
- Experience feeding and reconciling MDM/RDM data.
- Strong on operationalisation: monitoring, replay, CI/CD.
