GPU & LLM hosting is live — dedicated L4 and RTX 4090 in India, from ₹9,999/mo

Storage

"S3-compatible" is a promise about the API, not the product

What the label actually guarantees, where implementations diverge, and how to test a compatible endpoint before you build a backup or media pipeline on it.

Zoophost4 min read

The S3 API is the closest thing object storage has to a universal interface. aws-cli, rclone, restic, boto3 and most backup tools speak it, so any endpoint that accepts S3 requests can claim compatibility. The claim is usually true for the basics and frequently untrue for the features you discover later.

What compatibility reliably means

A server that calls itself S3-compatible will handle:

  • Bucket and object CRUD over HTTP, addressed path-style (endpoint/bucket/key) or virtual-hosted-style (bucket.endpoint/key).
  • SigV4 request signing, which is what your access key and secret are for.
  • Multipart uploads for large objects: initiate, upload parts, complete, abort.
  • Presigned URLs for time-limited GET and PUT without exposing credentials.
  • ListObjectsV2 with prefix, delimiter and continuation tokens.

That covers a backup tool, a media pipeline, or an app that stores uploads. It is genuinely most of what people need.

Where implementations diverge

The gaps show up in the features that sound optional until you need them:

  • Versioning. Often partial or absent. If a tool assumes object versions for point-in-time recovery, verify it rather than assume.
  • Object Lock and WORM retention. Rarely implemented. If a compliance requirement depends on immutability, this is a blocker, not a detail.
  • Lifecycle rules. Transition and expiry policies vary widely. Where they exist, the syntax may accept only a subset of the AWS form.
  • Replication and event notifications. Usually platform-specific or absent.
  • IAM policies versus bucket policies. Many implementations support bucket policies and ignore IAM entirely, or offer their own permission model alongside.
  • Storage classes. Durability tiers are a provider concept; the header may be accepted and ignored.
  • Server-side encryption modes. SSE-S3 and SSE-KMS are not uniformly supported, and key management is usually the provider's own.

None of these are dishonest, but "S3-compatible" without a qualifier tells you nothing about them. Ask for the list.

The chunked-encoding trap

One interoperability detail bites more people than any other. The AWS SDKs can sign a request with a streaming signature, then send the body in chunks framed as aws-chunked, with each chunk carrying its own signature under STREAMING-AWS4-HMAC-SHA256-PAYLOAD.

If a gateway or proxy in front of the storage endpoint does not decode that framing, the bytes that get stored are not the bytes the client sent. Uploads report success and the object is subtly corrupt. Some clients only switch to this mode above a size threshold, which is why small test files pass and the first real backup does not.

Test with a file measured in hundreds of megabytes, and verify the checksum of what you read back, not just the HTTP status.

Egress is the business model

On the large hyperscaler object stores, storage and requests are cheap relative to bandwidth. Reading your own data back is billed per gigabyte. That single line item changes architecture:

  • Restores become expensive, so teams postpone testing them.
  • Serving media or downloads through storage encourages a CDN in front, mostly to move the cost rather than the bytes.
  • Cross-region or cross-provider copies get costed twice.

An endpoint without egress fees changes what is economically reasonable: you can verify backups weekly, serve assets directly, and keep an offsite copy without modelling your growth against a per-gigabyte read charge. Treat any such claim as a testable statement, and check whether request charges or minimum retention replace it.

A ten-minute acceptance test

Before you commit a pipeline to a compatible endpoint, run this against a test bucket:

  1. Round-trip a small object with aws s3 cp and compare sha256sum of source and destination.
  2. Upload 300 MB in one PUT and again with multipart, then read both back and compare checksums. This is where chunked-signature bugs surface.
  3. List more than 1,000 keys and confirm pagination returns all of them exactly once.
  4. Presign a PUT with a five-minute expiry, upload through it, then confirm the URL is rejected after expiry.
  5. Delete a prefix recursively and confirm the listing is empty afterwards.
  6. Trigger the error paths: wrong secret (expect SignatureDoesNotMatch), missing key (expect NoSuchKey), wrong bucket (expect NoSuchBucket). Distinct, correct codes matter because SDK retry logic branches on them.

If all six behave like AWS, the endpoint behaves like S3 for the operations your software actually uses.

Data residency is a separate question

API compatibility says nothing about where the bytes physically sit. For workloads with Indian data-residency obligations, the location of the cluster and the provider's ability to show you that location matter more than the endpoint's compatibility score.

  • object storage
  • s3
  • backups

Keep reading

All articles

Point your S3 client at a bucket you own

S3-compatible object storage on our own Indian cluster, with quota-managed buckets and no egress line item.

30-day money-back guaranteeInstant activationFree migration