Building a Petabyte-Scale Video Archive with Fixed-Cost Ceph

Description

Architecture write-up on tiering a video archive in one Ceph cluster: hot pool with 2x replication plus an archive pool using 5+2 erasure coding.

URL

• Visit resource

Related resources

• OpenIO SDSOpen-source object storage system for big-data/media workloads with erasure coding.
• GarageLightweight, geo-distributed S3-compatible object store with erasure coding, built for storing media blobs across sites with high-latency l…
• SeaweedFSDistributed object storage (S3 API), O(1) disk access, chunked large-file handling ideal for video segments, erasure coding.
• LeoFSHighly-available, distributed, eventually-consistent object/blob storage system used for large-scale media storage.
• Video Streaming in Distributed Erasure-coded Storage Systems: Stall Duration AnalysisAcademic analysis modeling playback stall duration for video streaming over distributed erasure-coded storage systems.
• Build a Self-Hosted CDN with OpenResty EdgeStep-by-step guide to building a self-hosted CDN using OpenResty Edge (LuaJIT edge gateway), from the OpenResty core team.
• OpencastOpen-source platform for capturing, processing, managing and distributing academic video, with its own storage and distribution architectur…
• cXnExtensible open-source CDN with classic nginx→varnish→S3 edge cache architecture, MIT licensed.
• video-transcode-queueReference architecture for a video upload and transcode pipeline running on Kubernetes with a job queue.
• jsDelivrProduction multi-CDN (Cloudflare+Fastly) delivery network with RUM+synthetic monitoring load balancer and dual-DNS failover architecture, o…
• RustFSRust S3-compatible object storage claiming 2.3x MinIO performance on small-payload workloads, drop-in migration from MinIO/Ceph, good fit f…