Porsche / MHP
Central data backbone and monitoring tools for Porsche's global vehicle management system, processing order data across international supply chain.
The challenge
Porsche needed a unified data platform to process vehicle orders, dealer quotas, stock levels, and logistics data flowing from multiple on-premise and cloud sources. The challenge was bridging legacy on-premise systems with modern cloud infrastructure while maintaining data consistency, compliance, and real-time visibility across the entire supply chain.
My role
I designed and built the multi-layer data pipeline architecture, developed the full-stack order monitoring application, and managed infrastructure-as-code across the platform. My responsibilities included Kafka topic management, Lambda function development, security policy enforcement, and delivering satellite data product repositories.
System architecture
What I built
Three-stage pipeline architecture separating ingestion, transformation, and persistence into independently deployable layers. Each layer has its own error handling, retry logic, and dead-letter queues, enabling granular monitoring and recovery without reprocessing entire data flows.
Data architectureConcurrency model aligned with Kafka partition topology to maximize throughput while preserving message ordering guarantees. Processing scales horizontally by adding partitions, with each consumer instance handling a deterministic subset of the data stream.
PerformanceReal-time change detection on upstream data sources feeding incremental updates into the pipeline. Only modified records flow through transformation and persistence stages, dramatically reducing processing volume and warehouse compute costs.
Data syncPolicy-as-code enforcement with 40+ security rules validated on every deployment. Infrastructure definitions are checked against organizational compliance requirements before provisioning, preventing non-compliant resources from reaching production.
SecurityMulti-region deployment strategy ensuring data pipeline continuity during regional outages. Kafka topic replication and stateless compute layers allow traffic redirection with minimal data loss and recovery time.
ReliabilityDeclarative Kafka topic configuration managed through version-controlled definitions. Topic creation, schema evolution, and partition changes flow through pull request review and automated validation before applying to the cluster.
OperationsTechnologies used
Key achievements
- Built central data backbone processing 60+ Kafka topics
- Designed multi-layer pipeline architecture (ingest → transform → persist)
- Built full-stack order monitoring application
- Managed 40+ security policy rules via policy-as-code
- Delivered 16 satellite data product repositories