Video Streaming

Web, mobile, TV, and console clients first need a reachable endpoint and a compatible playback experience. DNS and edge routing choose an appropriate serving location, while the CDN caches application assets and reusable media segments. TLS protects each network hop; an edge that terminates it can see plaintext and must protect its own keys, logs, and connection to the origin.

WAF rules, bot defenses, and rate limits keep abusive traffic away from expensive origin paths. The gateway authenticates API calls and routes upload, catalog, and playback requests to their owners. Access to a particular video still depends on visibility and entitlement checks; a valid login token alone does not grant access to every media object.

The upload API creates a session and returns scoped, expiring URLs for direct multipart transfer to object storage. The client retries missing parts rather than resending the whole file. Completion verifies that the final raw object is durable and consistent with the session before recording upload success; the creator can then see processing without holding an HTTP connection open for encoding.

Ingestion validates the file and extracts metadata, with scanning and unsupported-format outcomes recorded explicitly. The catalog owns title, tags, visibility, and processing status independently of the byte object. A durable upload-completed event, commonly keyed by video ID, notifies the job orchestrator; Kafka carries that event, while the orchestrator owns stage dependencies, retries, and workflow recovery.

The orchestrator records jobs for the chosen resolution and codec ladder, such as H.264, H.265, or AV1 outputs from mobile sizes through 4K where supported. Separate workers execute those jobs with bounded parallelism, and priority queues allocate capacity to time-sensitive uploads. A retry uses the same job identity and output version so two workers cannot publish conflicting results.

Packaging turns encoded media into HLS playlists and segments or a DASH manifest and its segment references. Quality checks validate duration, audio/video synchronization, required outputs, and other product criteria such as unexpected black frames. Write and validate the immutable media objects first, then atomically publish the catalog pointer to the usable manifest. A partial encode must remain processing or failed rather than expose a broken playlist.

Object storage holds raw masters, encoded segments, thumbnails, preview sprites, and captions. The catalog stores searchable facts and references: ownership, processing state, channel, available renditions, manifest version, and entitlement rules. Keeping bytes separate from metadata makes catalog transactions small and allows different lifecycle policies for originals, published media, and temporary failed outputs.

A search index projects titles, tags, and permitted transcript content for discovery; its lag does not make an existing playable object disappear. Replication and geographic placement should follow recovery and delivery requirements rather than automatically copying every asset everywhere. Retain a clear deletion workflow across catalog, derived indexes, cached access, and object versions so removing a video also removes its usable serving paths.

The streaming API checks identity, visibility, and entitlement, then returns a manifest URL with the appropriate access contract. The manifest describes compatible codecs, bitrates, and segment locations; the player ultimately chooses a rendition using device support, measured throughput, and buffer health. Seeking translates a playback time into segment requests rather than asking an application server to retransmit the entire file.

CDN edges serve those reusable segment bytes, with an origin shield coalescing misses before they reach object storage. Adaptive bitrate changes quality as conditions change, ideally avoiding a stall by reducing requested bitrate before the buffer empties. Measure startup delay and rebuffering from the player, because a fast manifest API does not prove that later segment downloads are healthy.

User profiles, subscriptions, playlists, and watch-later lists are separate product services; comments, likes, and notifications can use durable asynchronous side effects. Search and recommendations combine offline models with fresher behavioral features. Keep those optional enrichments outside routine segment fetches so a ranking outage can produce a simpler browsing experience while an already authorized video continues playing.

Advertising and billing require an explicit product contract. VAST or VMAP may describe ad responses and schedules, while entitlement and payment systems decide access. Some services may skip an unavailable optional ad; others may pause playback or require a valid ad decision. Bound those dependencies and provide a defined fallback rather than universally asserting that monetization must always fail open.

Player events flow through a durable stream into aggregation, warehouse, and machine-learning consumers. They report plays, progress, errors, buffering, and quality choices, but telemetry must have bounded resource use so it does not compete with playback. Track startup latency, rebuffer ratio, CDN and shield hit rates, segment failures, origin egress, and transcode queue age as separate symptoms.

Carry upload-session and job IDs through ingestion, encoding, validation, and manifest publication so a stuck video can be traced end to end. Recovery must restore both media bytes and catalog references, including access keys and processing state. Replicate or back up according to loss and recovery objectives, then test serving from the recovery location; active-active deployment alone does not prove those dependencies are independent.