Building In-Video Search at Netflix

Netflix Tech Blog article about developing an internal system for searching within video content (e.g., finding scenes across the catalog). It covers the indexing of visual and audio data to enable search.

Link

• Visit resource

Related resources

• PanakoContent-based audio search and fingerprinting system robust to noise and distortion, with stream-monitoring subapps, academic-backed.
• SparseSync: Audio-Visual Synchronisation with Trainable SelectorsBMVC 2022 Spotlight paper and code for sparse-in-space-and-time audio-visual sync detection.
• Evaluating Video Quality Metrics for Neural and Traditional Codecs (4K/UHD-1)Academic comparison of VMAF, AVQBits, and FasterVQA no-reference quality metrics on 4K content.
• VQA² (Visual Question Answering for Video Quality Assessment)ACM MM 2025 paper and code for VQA²-Assistant, a unified video/image quality scoring and interpretation model.
• audioprintPython MFCC-based perceptual-hash audio fingerprinting library.
• CompressedVQA: Full/No-Reference VQA for Compressed UGC VideoICME 2021 grand challenge-winning full-reference and no-reference video quality models for compressed user-generated content.
• Bitrate Ladder Construction via Transfer Learning and Spatio-Temporal FeaturesML-based bitrate ladder optimization achieving 94.1% complexity reduction at only 1.71% BD-rate cost using transfer learning.
• VMAF-torchPyTorch reimplementation of VMAF metric, GPU/gradient-friendly for use in learned codec optimization loops.
• 2BiVQA: Double Bi-LSTM No-Reference Video Quality AssessmentNo-reference VQA model for UGC video using double Bi-LSTM, companion code to arXiv:2208.14774.
• nr-vqa-consumervideoNo-reference video quality metric tuned for consumer video capture artifacts (sensor noise, motion blur, shake) rather than compression art…