Glance, a mobile-first content platform, transforms 1-2 hour long-form videos into 30-180 second vertical clips for mobile lock screens. With daily volumes projected to grow from 3,500 to over 10,000 videos, manual editing became infeasible. The solution uses an AI pipeline with three modules: video clipping, intelligent reframing, and finishing.

Module 1 (Video Clipping) extracts audio, transcribes with Google Cloud Speech-to-Text v2, uses Gemini 2.5 Flash to identify engaging 60-second segments, clips the video, and validates transcripts.

Module 2 (Intelligent Reframing Engine) converts 16:9 frames to 9:16 portrait. Active speaker detection uses Google Cloud Vision API for frame-by-face analysis, including a liveness check (mouth movement, head pose) to distinguish live speakers from static images. A liveness ratio (animated frames / total frames) identifies the primary speaker. Split-screen detection handles interview layouts using two approaches: continuous face tracking with Samurai (open-source object tracker) and frame-by-frame detection with Vision API, both identifying vertical dividing lines and contrasting backgrounds. The system then applies reframing rules: single-speaker crop, split-screen vertical stacking, multi-speaker crop, or fallback center crop. Short scenes are merged and camera smoothing prevents abrupt cuts.

Module 3 (Finishing) adds dynamic caption highlighting using word-level timestamps (karaoke-style with mustard yellow on black) via MoviePy, and applies brand masks and logos programmatically.

The pipeline demonstrates how AI can automate repetitive video editing while preserving narrative context, offering a template for organizations with long-form video archives.