Inca — Message Tracing and Loss Detection For Streaming Data @Netflix
Netflix engineering post introducing Inca, a message tracing and loss detection system for Netflix's real-time data infrastructure, built on a multi-cluster Kafka architecture and Flink-powered stream processing.
Link
Related resources
- Common Media Client Data (CMCD)Original academic paper (ACM NOSSDAV 2021) introducing Common Media Client Data for streaming analytics, initial findings.
- eBandit: eBPF Kernel-Level Network Monitoring with MAB ABR SelectionAcademic paper implementing kernel-level (eBPF/sockops) network monitoring combined with multi-armed-bandit ABR selection for adaptive stre…
- ML-Driven Open-Source Framework for QoE in Multimedia NetworksEnd-to-end ML framework predicting and optimizing QoE from network conditions, based on ITU-T P.1203, code released open source.
- multifluxSelf-hostable multistreaming platform powered by Cloudflare Workers/Pages/Stream with edge fanout.
- OpenIO SDSOpen-source object storage system for big-data/media workloads with erasure coding.
- Build a Self-Hosted CDN with OpenResty EdgeStep-by-step guide to building a self-hosted CDN using OpenResty Edge (LuaJIT edge gateway), from the OpenResty core team.
- OpencastOpen-source platform for capturing, processing, managing and distributing academic video, with its own storage and distribution architectur…
- jsDelivrProduction multi-CDN (Cloudflare+Fastly) delivery network with RUM+synthetic monitoring load balancer and dual-DNS failover architecture, o…
- Survive CDN Failures with Redundant StreamsTechnical deep-dive on client-side multi-CDN failover pattern using redundant streams with HLS.js.
- cXnExtensible open-source CDN with classic nginx→varnish→S3 edge cache architecture, MIT licensed.
- video-transcode-queueReference architecture for a video upload and transcode pipeline running on Kubernetes with a job queue.