Cloud File Storage

Web and mobile clients interact with APIs, while desktop agents also watch local changes and maintain a sync database. DNS routes them to an available service region; a CDN serves static application assets and permitted downloads. The gateway authenticates users and devices, applies quotas and request limits, and routes metadata operations without proxying every large file through the application tier.

Account sign-in can use OAuth or OpenID Connect, with multi-factor authentication and device management where the product requires them. Those mechanisms establish identity; each file operation still needs a current permission decision. A device with an open sync connection must lose access when its session or membership is revoked, so long-lived connections cannot bypass policy changes.

An upload session records the intended file, base version, expected chunks, and expiry. The client uploads missing chunks with checksums and resumes only failed parts after interruption. Download authorization returns the exact committed version and supports ranged or parallel chunk reads. A completed transfer is distinct from publishing a new file version in the catalog.

Metadata operations create, move, rename, share, and delete stable file identifiers; paths are mutable views of those identifiers. Sync returns ordered changes since a device cursor, including deletion markers. Trash and restore preserve references according to a retention policy. These services must agree on version and permission checks so a stale rename or restore cannot overwrite a newer change.

The authoritative catalog stores stable file and folder identifiers, parent relationships, permissions, version pointers, and the chunk manifest for each retained version. Pick a database and transaction model that can enforce these invariants; naming a distributed key-value store does not automatically make a move, permission update, or version commit strongly consistent. Use conditional writes or appropriate transactions for the actual access pattern.

A committed metadata mutation also records a durable journal or outbox entry. A relay publishes that change to Kafka for sync notifications, search indexing, and audit processing, with replay-safe identifiers. Search can lag, but its results still need authorization before content is disclosed. The audit trail records who changed which version or permission, allowing failures and unexpected access changes to be reconstructed.

Store immutable chunks under a verified content hash and represent a file version as an ordered manifest of hashes. Fixed-size chunks are simple; content-defined chunking finds boundaries in the bytes so some small edits preserve more existing chunks. Savings depend on the workload, chunk sizes, and encryption design, so measure reuse rather than assuming a fixed percentage. Deduplication must stay within the intended tenant and authorization boundary.

Publish a manifest only after all referenced chunks are durably available and their references are protected from garbage collection. Retained versions, trash, and active uploads can all keep chunks live. A delayed reference-count event is insufficient proof that deletion is safe. Coordinate reference creation with a deleting state or pins, wait through the retention grace period, and recheck authoritative reachability before removing bytes.

Each device keeps a local database, pending edits, and the last journal position it applied successfully. A sync response contains added, changed, and deleted entries after that cursor. Apply the delta and advance the local cursor together so a crash causes safe replay instead of skipping changes. WebSockets or SSE can announce that newer changes exist, but the client still fetches authoritative deltas.

An edit names the version on which it was based. If another device already committed a newer version, the server detects the conflict rather than silently replacing it. Preserve both versions, create a conflict copy, or apply a content-specific merge policy. If a cursor is older than journal retention, request a fresh snapshot and reconcile pending local edits instead of pretending the missing history was empty.

Permissions combine account or workspace roles with the product’s file and folder inheritance rules. Specify how inherited grants, explicit exceptions, and moves between folders interact; deny-overrides is one possible policy, not a universal filesystem rule. A share link delegates a bounded capability, such as viewing one file until expiry, and should not silently grant editing or access to sibling files.

Revocation must reach download authorization, existing sync subscriptions, and relevant caches. A previously issued signed download URL may remain usable until its own expiry unless the delivery system provides a stronger revocation mechanism, so choose the lifetime deliberately. Sharing and commenting publish asynchronous notification events, while real-time collaborative editing requires an additional merge protocol beyond ordinary file synchronization.

Upload completion can enqueue scanning, preview, and thumbnail jobs while metadata tracks whether the file is pending, ready, or rejected. Workers use a file-version identifier so an old preview cannot replace the output for a newer edit. Quota accounting must define logical bytes, retained versions, and physical deduplicated bytes separately; otherwise concurrent uploads can exceed a workspace limit or make the bill impossible to explain.

Measure upload completion, metadata latency, device catch-up age, failed transformations, and unreferenced-byte growth. Keep backups of the catalog and chunks with compatible recovery points, and test a restore that reconstructs an actual folder tree. Multiple regions need explicit metadata ownership, permission behavior, and recovery objectives; cache clusters, orchestration, and key management support those decisions rather than replace them.