Data Engineer, Data Quality & Provenance

Overview

In this role you will design and operate scalable data pipelines and data products that power autonomous-driving development. You’ll build and manage versioned, reproducible datasets for training, evaluation and safety analysis, enabling cross-functional teams to access trustworthy data. You’ll collaborate with ML, autonomy, simulation and safety engineers to define schemas and data contracts, improving reliability and cost efficiency of large-scale data platforms. This is a hands-on, impact-driven opportunity to shape the data backbone of Wayve’s AI platform with petabyte-scale datasets.

Pay / Benefits

• competitive salary
• meaningful equity
• relocation support and visa sponsorship
• hybrid working with core hours
• learning and development budget
• comprehensive benefits including health insurance and family leave

Responsibilities

• Design and operate scalable batch and streaming pipelines for multimodal fleet and simulation data
• Build data models, catalogs, indexes and query capabilities for sensor, vehicle-state, map and event data
• Create versioned, reproducible datasets for training, evaluation, replay, scenario mining and safety analysis
• Develop workflows for ingestion, synchronisation, transformation, curation, labelling and quality validation
• Partner with autonomy, ML, simulation and safety engineers to define schemas, APIs and data contracts
• Improve reliability, performance and cost efficiency of large-scale storage and compute workloads
• Work on petabyte-scale data products supporting autonomy development
• Maintain trusted datasets with strong versioning and reproducibility guarantees
• Implement metadata, geospatial and temporal indexing for data discovery and scenario mining
• Integrate real-world fleet logs with simulation and synthetic data for closed-loop development
• Define data-quality standards, observability, access controls, retention policies and incident response processes
• Build cloud-based storage and distributed compute infrastructure for reliability and cost efficiency

Key requirements

• strong hands-on Python and SQL skills
• designed and operated large-scale distributed data systems
• production experience with distributed-processing and workflow-orchestration (Spark, Flyte or Airflow)
• built cloud-based data platforms using object storage and understood data organisation, versioning, querying, governance and cost management
• owned data quality, lineage, observability, reproducibility and incident response for production data workflows
• ability to translate ambiguous requirements from ML/data-science/robotics teams into durable platform capabilities
• comfort with ambiguity and helping define standards and operating model for a growing data platform

• collaboration with cross-functional teams
• ability to operate in ambiguity
• strong problem-solving mindset
• Python
• SQL
• Spark

…