Byte Down: Making Netflix’s Data Infrastructure Cost-Effective

Netflix engineering blog post describing strategies for reducing data infrastructure costs in large-scale video operations.

Link

• Visit resource

Related resources

• LeoFSHighly-available, distributed, eventually-consistent object/blob storage system used for large-scale media storage.
• Coffee: Cost-Effective Edge Caching for 360° Live VideoAcademic paper on predictive tile-prefetch edge caching for 360-degree live video streaming, achieving 76% backhaul traffic reduction.
• YIGS3-compatible distributed object storage server, alternative to Ceph/MinIO for large-scale media deployments.
• OpenIO SDSOpen-source object storage system for big-data/media workloads with erasure coding.
• Common Media Client Data (CMCD)Original academic paper (ACM NOSSDAV 2021) introducing Common Media Client Data for streaming analytics, initial findings.
• Build a Self-Hosted CDN with OpenResty EdgeStep-by-step guide to building a self-hosted CDN using OpenResty Edge (LuaJIT edge gateway), from the OpenResty core team.
• ML-Driven Open-Source Framework for QoE in Multimedia NetworksEnd-to-end ML framework predicting and optimizing QoE from network conditions, based on ITU-T P.1203, code released open source.
• Survive CDN Failures with Redundant StreamsTechnical deep-dive on client-side multi-CDN failover pattern using redundant streams with HLS.js.
• cXnExtensible open-source CDN with classic nginx→varnish→S3 edge cache architecture, MIT licensed.
• video-transcode-queueReference architecture for a video upload and transcode pipeline running on Kubernetes with a job queue.
• OpencastOpen-source platform for capturing, processing, managing and distributing academic video, with its own storage and distribution architectur…