S3 bucket replication: methods, examples and pitfalls

WebDisk Blog · category: Object Storage · reading time: ~27 minutes

In short:- Bucket replication – a second copy of your objects somewhere else – comes in five families of solutions: a rule inside S3 itself, cluster replication (multisite), copying with tools (rclone), an event-driven worker and dual writes in the application.- They all share one property: replication propagates deletion and overwriting as well. A replica is not a backup – it becomes one only when paired with versioning and a write lock (S3 Object Lock).- In WebDisk Files there is Bucket replication: a one-off import of an S3-compatible storage into a new storage space in WebDisk, with progress tracking – with no transfer charges, within the migration pool included in your subscription. >Not working with a terminal? Skip the command blocks – the description of the methods, the section on pitfalls and the WebDisk part give you the full picture.

The question "do we have a second copy?" usually comes up at the worst possible moment: when someone has deleted a directory, when the provider has announced a region outage, or when the board asks what happens if we have to change providers tomorrow. The answer "but we have the cloud" is not enough, because the cloud is not one place – it is a specific bucket, in a specific location, with a specific account that someone can take over.

Bucket replication (a bucket – a container for files in S3 storage, the equivalent of a disk or a network share; in WebDisk services we call it a storage space) is the answer to that question – but a partial answer, and one that is easy to misunderstand. In this article we go through every replication method genuinely used with object storage: from a ready-made rule switched on with a single command, through replication of whole clusters, to your own worker listening for events. For each one we show an example, the cost of entry and what that method does not do. At the end we describe how we solved replication in WebDisk Files.

Why replicate a bucket? Five different reasons, five different solutions

Before you choose a technique, name the problem – because "replication" is one word for five different jobs, and a solution that is good for one can be useless for another.

Resilience to the loss of a location (DR, disaster recovery). You want to survive the loss of an entire region, data centre or cluster. What you need is a copy physically somewhere else, ideally in a different failure domain and ideally with its own credentials.
Migration and exit from a provider. You move data from one S3 to another – once, but completely and verifiably. This is not continuous replication, it is a controlled transfer with an acceptance check; we wrote about it at greater length in the context of vendor lock-in.
Geographical proximity. A copy closer to users or closer to compute, to shorten response times. What matters here is propagation delay, not durability.
Separating environments and roles. A copy of production data for analytics, testing or archiving – often filtered, trimmed to a single prefix and stripped of sensitive data.
Isolating the copy from the production account. A copy in a place that production credentials cannot reach – so that an account takeover does not mean losing everything. This is the anti-ransomware scenario and it has its own rules of the game.

Remember this list, because we will come back to it when choosing a method: the first point is best served by one-way replication that preserves history, the second – by a one-off, verifiable copy made with a tool, the third – by the provider's native replication, the fourth – by copying with a filter, and the fifth – by a copy in a separate account with a write lock.

Replication, synchronization, backup – three words that are not synonyms

This distinction decides whether your second copy will save anything at all.

Replication keeps a copy consistent with the source. Its goal is for the target to look exactly like the source – including the fact that a file has been deleted. Good replication is fast and faithful; that also means it faithfully reproduces the disaster.

Synchronization (rclone sync, aws s3 sync --delete) is replication done periodically, in batches – with the same property: it removes from the target whatever is not in the source.

Backup is a copy with history and with protection against change: many points in time, retention, no way to delete before the deadline. A backup deliberately is not consistent with the source – its value lies precisely in remembering the state from before the mistake.

We covered the 3-2-1 rule and the difference between a copy and a synchronization in the article about cloud backup. Here one sentence is enough, and it is worth pinning above your desk: a replica protects against losing hardware, a backup protects against losing data. They are not interchangeable and most serious deployments have both.

Four questions that choose the method for you

Before we move on to the techniques, four questions. The answers usually narrow the choice down to one method, two at most.

What RPO (Recovery Point Objective) can you accept? How many minutes – or hours – of data can you afford to lose? Seconds mean event-driven or native replication; hours – scheduled copying with a tool is perfectly enough.
Are both ends with the same provider? Native S3 replication usually works within a single platform. Between different providers you are almost always left with tools or your own worker.
Does the copy have to remember history? If so, you need versioning on both sides and a method that carries versions across – or a deliberate decision that the copy is "flat" and the history is kept by the backup.
Who pays for the transfer and who holds the keys? A copy with the same provider can be free, a copy "outside" costs egress traffic. And a copy accessible with the same keys as production does not protect against an account takeover.

Method 1. A replication rule inside S3: the storage looks after the copy itself

Conceptually the simplest and the most "cloud-native" method: you tell the storage to look after the copy itself. The configuration lives on the source bucket as a replication rule, and all the work is done by the storage layer – with no machine on your side.

There are three things you need for it to work at all:

versioning on both sides – replication operates on object versions, not on names; without versioning there is nothing to replicate (this is the same foundation that Object Lock stands on),
an identity with permissions to both buckets – in AWS this is an IAM role that the S3 service assumes on your behalf,
a rule – with a filter (the whole bucket, a prefix or a tag), a priority and a description of the target.

Before you start. The examples use the aws CLI and rclone configured as in the previous articles in this series (aws configure + --endpoint-url of your provider – we omit it below for readability). Be careful with real data: rclone sync and aws s3 sync --delete remove from the target whatever is not in the source, and a typo in the source prefix can wipe out the copy. Practise on test buckets.

A practical example in AWS. First versioning, then the rule itself:

# 1. versioning on the source and on the target — a necessary condition
aws s3api put-bucket-versioning --bucket firma-produkcja \
  --versioning-configuration Status=Enabled
aws s3api put-bucket-versioning --bucket firma-kopia-dr \
  --versioning-configuration Status=Enabled

# 2. replication rule: everything that lands in "raporty/" ends up in the second bucket
aws s3api put-bucket-replication --bucket firma-produkcja \
  --replication-configuration '{
    "Role": "arn:aws:iam::111122223333:role/s3-replikacja",
    "Rules": [{
      "ID": "raporty-do-dr",
      "Priority": 1,
      "Status": "Enabled",
      "Filter": { "Prefix": "raporty/" },
      "DeleteMarkerReplication": { "Status": "Disabled" },
      "Destination": {
        "Bucket": "arn:aws:s3:::firma-kopia-dr",
        "StorageClass": "STANDARD_IA"
      }
    }]
  }'

Checking whether a particular object has already been copied is just as simple – the replication status is visible in its metadata:

aws s3api head-object --bucket firma-produkcja --key raporty/2026-08.csv
# "ReplicationStatus": "COMPLETED" ← or PENDING / FAILED
# on the copy the same object reports "REPLICA"

Note DeleteMarkerReplication. This single field decides the character of the whole solution: enabled – the copy is a faithful mirror and the "disappearance" of a file propagates to the other side; disabled – a deleted file stays in the copy. For replication understood as DR, disabling it is sometimes a deliberate choice, but you have to know that you are then making a copy which drifts away from the source over time and will not tidy up after itself.

The second thing that is easy to forget: deleting a specific version is never replicated: if you pass a version identifier in the DELETE request, it will disappear in the source only. This is a deliberate protective property – and good news for the "someone took over the account and is wiping the data" scenario. Watch out for one migration detail: older configurations (without a Filter element) replicated delete markers from user actions by default, and newer ones do not. Adding a filter to an old rule therefore quietly changes the deletion behaviour.

Three surprises this method keeps up its sleeve:

The rule works from the moment it is switched on. Objects that were already in the bucket will not be copied by themselves – with a hyperscaler you have to run a separate bulk job for them (in AWS: S3 Batch Replication), and elsewhere simply copy them with an ordinary tool (method 3, described below). The same applies to any interruption in the rule's operation – we come back to that in pitfall 6.
Replication does not build chains. A copy in bucket B will not travel on to C, even if B has its own rule. Chains are not built – every target is connected separately to the source.
A tag filter only catches tags applied at write time. Tagging a file after the fact does not trigger replication, which effectively blows up the popular "upload first, classify later" pattern. A prefix filter has no such problem.

If replication is to have contractual time bounds, with a hyperscaler you buy them separately: S3 Replication Time Control commits the provider to moving 99.9% of new objects within 15 minutes and adds metrics and events for threshold breaches to that. Without this option AWS makes no promise at all: the documentation only says that most objects replicate within 15 minutes, that with large ones it can take several hours, and in extreme cases even several dozen – with no SLA. In practice it is usually a matter of seconds, but in an audit what counts is the commitment, not the practice.

When it is worth it: both ends are with the same provider, you care about a small RPO and you do not want to maintain your own copying infrastructure. What it will not solve: moving data to another provider, the history from before the rule was switched on, or protection against deletion – that last one is added separately, with versioning and a lock on the destination bucket.

Method 2. Replication at cluster level: multisite, or a property of the platform

A method for those who have two object storage systems – two clusters, two server rooms – and want replication to be a property of the platform, not of every bucket separately. In Ceph RGW it is called multisite, in MinIO – installation replication.

In Ceph the hierarchy looks like this: a realm (a namespace) contains a zonegroup, and inside it live zones (zone) – one per location, each based on its own Ceph cluster. The zones exchange two independent streams between themselves: metadata (users, buckets, policies – managed centrally by the master zone) and data (objects – synchronized by each zone separately). The RGW daemons do this themselves, with no external agent.

# location A — realm, zonegroup and master zone
radosgw-admin realm create --rgw-realm=firma --default
radosgw-admin zonegroup create --rgw-zonegroup=eu --endpoints=https://s3-a.firma.pl --master --default
radosgw-admin zone create --rgw-zonegroup=eu --rgw-zone=eu-a --endpoints=https://s3-a.firma.pl --master --default

# system user — the zones authenticate to each other with this key
radosgw-admin user create --uid=sync --display-name="Sync User" --system \
  --access-key=<klucz-systemowy> --secret=<sekret>
radosgw-admin zone modify --rgw-zone=eu-a --access-key=<klucz-systemowy> --secret=<sekret>
radosgw-admin period update --commit

# location B — the second zone in the same group; it first pulls the realm and the period
radosgw-admin realm pull --url=https://s3-a.firma.pl \
  --access-key=<klucz-systemowy> --secret=<sekret> --default
radosgw-admin period pull --url=https://s3-a.firma.pl \
  --access-key=<klucz-systemowy> --secret=<sekret>
radosgw-admin zone create --rgw-zonegroup=eu --rgw-zone=eu-b --endpoints=https://s3-b.firma.pl \
  --access-key=<klucz-systemowy> --secret=<sekret>
radosgw-admin period update --commit

# day-to-day diagnostics: is the zone falling behind
radosgw-admin sync status
# you are looking for "behind" shards (to be caught up) and "recovery" (an error, a retry scheduled)

Replication is asynchronous, with eventual consistency – changes propagate in the background, and successive batches of the change log are polled every ten to a few dozen seconds. That means precisely this: right after a write the second location does not have the object yet, and switching over to the standby zone is an operator decision, not an automatic one: you promote it manually, ideally after making sure it has caught up with the backlog.

Newer Ceph releases allow selected buckets to be replicated instead of the whole zone – this is done with a granular sync policy built from three layers: the group (whether replication is allowed), the flow (which way the data goes – bidirectionally or one way) and the "pipe" (what exactly goes where). The recommended pattern is to allow it broadly at zonegroup level and enable it selectively at bucket level:

# at zonegroup level: "allowed, but I am not enabling everything"
radosgw-admin sync group create --group-id=grupa-dr --status=allowed
radosgw-admin sync group flow create --group-id=grupa-dr --flow-id=a-do-b \
  --flow-type=directional --source-zone=eu-a --dest-zone=eu-b
radosgw-admin sync group pipe create --group-id=grupa-dr --pipe-id=wszystko \
  --source-zones='*' --dest-zones='*'
radosgw-admin period update --commit

# at the level of a specific bucket: "replicate this one" (no period update)
radosgw-admin sync group create --bucket=firma-backup \
  --group-id=backup-default --status=enabled
radosgw-admin sync group pipe create --bucket=firma-backup \
  --group-id=backup-default --pipe-id=pipe1 \
  --source-zones='*' --dest-zones=eu-b

radosgw-admin sync info --bucket=firma-backup # what will actually flow and where

A pipe can also filter along the way (prefix, tags) and transform the target: replicate to a bucket with a different name, to a different owner or to a different storage class. There is also a separate module that pushes data one way to an external S3 – that is, to a completely foreign provider, without standing up a second Ceph cluster.

The most interesting variant from a security point of view is the archive zone (archive zone in Ceph): a zone that enforces versioning and keeps successive versions of everything that has passed through the cluster, and from which objects can be removed only through its own gateways. It is the only flavour of cluster-level replication that defends against an operator's mistake and against ransomware, because it breaks the key assumption that "the copy should look like the original". In practice the layout looks like this: production runs on ordinary, unversioned zones, and next to it stands the archive, which nobody in production has any way of reaching.

When it is worth it: you have (or are buying) two locations and you want replication that is transparent to clients – including accounts and policies. What it will not solve: nothing, if the second location does not exist. Multisite is an infrastructure and cost decision, not a switch in a panel; it requires a second cluster, a link that can carry the stream of changes, and discipline around upgrades (new features are enabled only once all the zones understand them).

Does your provider even have this? A review of nine platforms

As of August 2026. Before you plan replication based on a rule in the bucket, check whether there is anything to switch it on with. This is the most common disappointment in this area: native replication is a platform feature, not part of the S3 standard. The protocol is shared, but whether a replication rule exists at all and what exactly it does varies from provider to provider.

AWS S3 – full native replication, within the same region and across regions, with an optional contractual propagation time and a separate bulk job for existing objects.
OVHcloud – native, asynchronous replication, configured with the standard put-bucket-replication; versioning has to be enabled by you on the source and on the target (the only thing that does it by itself is the separate out-of-location replication option, ticked when creating a bucket in three-zone regions), and there is also a separate batch mode for objects written earlier. Both buckets have to be in the same project.
Wasabi – native replication, driven by S3-compatible calls. A curiosity: it does not require versioning, but it does require a matching state on both sides (versioned to versioned, unversioned to unversioned) and the same bucket owner.
Backblaze B2 – native replication, but configured with its own API and CLI, not through the S3 interface; up to two rules per bucket, with an option to cover existing files. You do not pay for the replication traffic itself – you pay for the second copy of the data.
Google Cloud Storage – replication is a property of the bucket, not a rule: you choose a dual-region or multi-region bucket and that is that. The nominal target is a complete set of objects within 12 hours, and for an extra fee (turbo) – 15 minutes.
Azure Blob – two separate layers: geo-redundancy of the whole account (GRS/GZRS) and object replication between containers, which requires versioning on both sides and a change feed on the source.
MinIO – it has both per-bucket replication and replication of whole installations (together with users and policies), but you need to know the state of the project: the community edition repository was archived in April 2026, and in the commercial AIStor replication belongs to the paid plans. Server-side replication also connects MinIO installations to each other only.
Ceph RGW – multisite and the granular sync policy described above; with a provider based on Ceph the question is therefore simply: do you have a second zone.
DigitalOcean Spaces, Scaleway, Cloudflare R2 – no native bucket-to-bucket replication. R2 makes up for it with two migration tools (a bulk move and incremental on-demand pulling of objects), and with the others you are left with method 3.

The practical conclusion is that in the "a copy with another provider" scenario you almost always come back to tools – because even when both platforms have native replication, they can almost never agree on it with each other.

Method 3. Copying with tools: rclone, mc, s5cmd and aws s3 sync

The most universal method and – contrary to appearances – the one most often used in practice. It needs no agreement or support from the provider on either side: you take two S3 endpoints, two sets of keys and push the data across. It works between any platforms, so this is the one that saves you during migrations and when leaving a provider.

The king of this category is rclone, because it understands several dozen backends and has exactly the switches this job needs:

# configuring two remotes: rclone config (type: s3, provider: AWS / Ceph / Minio / Other)

# 1. incremental copy WITHOUT deleting anything in the target — the safe default choice
rclone copy zrodlo:firma-produkcja cel:firma-kopia --progress

# 2. full mirror: the target looks exactly like the source (DELETES surplus files!)
rclone sync zrodlo:firma-produkcja cel:firma-kopia --progress

# 3. production version: S3-tuned parallelism, comparison by checksums,
# bulk listing and a bandwidth limit that varies during the day (values in BYTES/s)
rclone sync zrodlo:firma-produkcja cel:firma-kopia \
  --checksum --transfers 32 --checkers 64 --fast-list \
  --bwlimit "08:00,20M 18:00,off" \
  --log-file /var/log/rclone-dr.log --log-level INFO

# 4. acceptance check — do both sides really have the same thing
rclone check zrodlo:firma-produkcja cel:firma-kopia --one-way --checksum

The difference between copy and sync is not a nuance but a choice of strategy: copy gives you a copy that will never delete anything by itself (it grows, but it survives a deletion in production), sync gives you a mirror (it is consistent, but it will repeat every disaster – including a typo in the source prefix, which can wipe the target). For DR purposes you usually want copy – or sync on a bucket with versioning enabled, where "deletion" is only a marker.

Four details that separate replication that works from replication that only looks like it works:

By default rclone compares size and modification time, and it stores the modification time on S3 in its own metadata. Objects uploaded with a different tool do not have that metadata, so the comparison then falls back to the server-side LastModified, which changes with every copy – and the next run can decide that everything differs. Hence the --checksum in the example above.
--checksum on S3 compares MD5, which multipart objects do not have in their ETag. rclone works around this with its own metadata on its own uploads, but for large files uploaded with something else there is simply nothing to compare. This is the same pitfall we write about below in the section on verification.
A copy from S3 to S3 goes through your machine by default. Between two different configurations rclone downloads the data and sends it back, unless you explicitly pass --server-side-across-configs (and between different providers server-side copying will never work). At 50 TB that is the difference between transferring 50 and 100 TB – and a matching egress bill.
Run it with a lock. The most common way this kind of replication falls over by itself is overlapping runs from cron. The minimum is flock -n, more elegantly – a systemd timer with Type=oneshot and Persistent=true, which will not start a second instance of the same unit and will catch up a missed run after a host restart.

# /etc/cron.d/replikacja — hourly, with a lock against overlapping runs
17 * * * * root /usr/bin/flock -n /var/lock/replikacja.lock \
/usr/bin/rclone copy zrodlo:firma-produkcja cel:firma-kopia \
--checksum --fast-list --log-file /var/log/rclone-dr.log

Alternatives worth knowing:

s5cmd – when raw throughput matters with hundreds of thousands of objects; it parallelizes far more aggressively than aws s3 sync. The price is no bandwidth limiting (it can take up the whole available link) and no proper integrity verification; on top of that it works with a single set of credentials, so a transfer between two different providers has to pass through the local disk with it.
aws s3 sync – it is everywhere the AWS CLI is, and it is enough for simple jobs. It has a one-way "source is newer" test, and when copying from S3 to S3 it carries over tags and some object properties by default (--copy-props default) – at the cost of extra requests. Recently it can also surprise you outside AWS. Newer CLI releases compute checksums on every write by default; older S3-compatible implementations do not know about this and respond with an error. This is the most common "it suddenly stopped working" of recent months; the cure is to switch checksums to "when required" mode in the CLI configuration.
mc mirror --watch (the MinIO client) – it tempts you with the promise of continuous watching instead of cron runs, but the watching relies on a MinIO-specific extension. On AWS S3 or Ceph RGW it will end with an error or a silent fallback to ordinary polling – which is exactly what you wanted to avoid. Outside MinIO treat mc as an ordinary copying tool – object tags on an S3-to-S3 copy are in fact carried over by aws s3 cp/sync as well; the one that does not carry tags is, as it happens, rclone.

When it is worth it: different providers on both sides, migration, trimming the copy with a filter, full control over what is copied and when. What it will not solve: a small RPO – the copy is only as fresh as the last run. It will not carry history: none of these tools copies object versions or delete markers, and they will not restore Object Lock or the bucket configuration even approximately. It will also not spare you the egress cost, which you pay at the source provider on every cycle. With millions of small objects the listing alone can cost more than the transfer; at that scale method 4 looks more sensible.

And what if the bucket holds a backup repository rather than files?

A separate and often overlooked path is replicating the backup repository instead of replicating the data. If a backup tool writes to the bucket – restic, Kopia or Veeam – you make the second copy with its own means (kopia repository sync-to, a backup copy job in Veeam) or – in the case of a restic repository, which is a set of immutable files – with an ordinary rclone sync --checksum. You just have to know what you lose in the process: a byte-for-byte copy of the repository shares the encryption key with the original and is subject to the same pruning operations, whereas restic copy does the opposite – it creates a cryptographically independent copy, but it has to download and re-encrypt everything. With Veeam the matter is simple and worth repeating after the vendor: the lifecycle of data in object storage is managed exclusively by Veeam, and copying manually or adding lifecycle rules on its bucket can end in data loss.

Method 4. Event-driven replication: let the storage tell you what has changed

Instead of asking the storage "what has changed since yesterday?", let it tell you itself. Object storage systems can send a notification about every write and deletion: to a queue, to a broker, to an HTTP endpoint. Your own process receives the notification and copies exactly the one object it concerns.

In Ceph RGW this is configured as a "topic" and a notification attached to the bucket. The notification on the bucket itself is S3-compatible, but creating the topic below is already an RGW extension: the push-endpoint and persistent attributes are Ceph's own, and the topic identifier has the form arn:aws:sns:<zonegroup>:<tenant>:<topic>. On AWS the same effect is assembled differently – you create the topic separately, separately grant S3 the right to publish to it, and attach the receiver with a subscription:

# 1. topic: where the notifications should go (here: HTTP; Kafka or AMQP would do just as well).
# We point the sns calls at the RGW gateway endpoint: aws --endpoint-url https://s3-a.firma.pl ...
aws sns create-topic --name replikacja \
  --attributes '{"push-endpoint":"http://worker.wewn:9000/zdarzenia","persistent":"true"}'

# 2. notification on the bucket: we care about writes and deletions
aws s3api put-bucket-notification-configuration --bucket firma-produkcja \
  --notification-configuration '{
    "TopicConfigurations": [{
      "Id": "do-repliki",
      "TopicArn": "arn:aws:sns:eu:firma:replikacja",
      "Events": ["s3:ObjectCreated:*", "s3:ObjectRemoved:*"]
    }]
  }'

The worker on the other side is trivial to describe and demanding in the details: it receives the event, fetches the object from the source, writes it to the target, acknowledges. All the difficulty sits in what happens when something goes wrong – you need retries, a queue for failed events (dead-letter), resilience to the same event being delivered twice and an awareness that ordering is not guaranteed. You also have to remember that events describe the future: whatever was in the bucket before notifications were switched on still needs a one-off copy with a tool from method 3.

When it is worth it: large volumes, a lot of small writes, an RPO counted in seconds, a need for replication between different providers while keeping the copy fresh. What it will not solve: nothing here happens by itself – everything rests on your code and your maintenance. This is the most "do-it-yourself" of the solutions – it gives you the most control and the most things that have to be monitored.

Method 5. Dual writes in the application: sounds the simplest, works out the most expensive

The last family is replication moved into the application: on every write you send the object to two storage systems at once. It sounds the simplest of them all and can be tempting when storage is only an add-on to the product.

In practice dual writing hands you all the problems of distributed systems. What do you do when the first write succeeded and the second did not – reject the operation for the user, or accept it and fix it later? How do you catch up the divergence after an outage of the second storage? Where do you get copies of files written before you added the feature? Every answer is a piece of code that duplicates what the storage layer already has, ready and tested.

When it is worth it: when the second copy is meant to be different from the first – for example you write the original to S3 and its processed version to a completely different system. Then it is not replication but product logic, and it rightly sits in the application. What it will not solve: consistency without considerable effort. If both copies are meant to be identical, method 1, 3 or 4 from this article is almost always cheaper.

The layer people forget: durability inside the cluster

The methods above answer the question "what if we lose the whole location?". But there is a layer below that answers a far more common question: "what if a disk or a server fails?". In object storage this is handled by the cluster itself – and it is also called replication, which is the source of a great deal of confusion in conversations with providers.

n-way replication – every object kept in several full copies on different hosts (typically three). Simple, fast to rebuild, costs as much space as the multiplier.
Erasure coding – the object is split into data fragments and parity fragments spread across the nodes. Cheaper in capacity terms for comparable resilience, more expensive computationally and slower to rebuild.

Both techniques defend against hardware failure within a single cluster and neither of them is disaster recovery: an object deleted by a user propagates immediately to all the copies and fragments. When a provider says "we replicate your data three times", ask them two questions: are those three copies in different locations, and what happens when someone issues a DELETE.

Which method to choose? The comparison in a nutshell

The same five methods, arranged by the criteria that really decide the choice:

Typical RPO (how much data you can lose) – a rule in S3: seconds to minutes · multisite: seconds to minutes · tools from cron: as much as the interval between runs · event-driven: seconds · dual writes: zero (or divergence, when one write fails).
Does it work between different providers – a rule in S3: usually not · multisite: between clusters of the same platform, although the cloud sync module can push one way to a foreign S3 · tools: yes, always · event-driven: yes · dual writes: yes.
How much of your own infrastructure it requires – a rule in S3: none · multisite: a second cluster · tools: one machine with access to both ends · event-driven: a worker, a queue and monitoring for them · dual writes: application code.
Does it carry version history – a rule in S3: yes (it replicates versions) · multisite: yes · tools: no (they copy the current state) · event-driven: only what you copy yourself · dual writes: no.
Does it propagate deletion – a rule in S3: depends on DeleteMarkerReplication · multisite: yes, except in the archive zone · tools: sync yes, copy no · event-driven: depends on your worker · dual writes: depends on the code.
The main cost – a rule in S3: cross-region transfer and request charges · multisite: the second location · tools: egress traffic and machine time · event-driven: maintenance and being on call · dual writes: technical debt.

In short, by scenario: migration or leaving a provider – rclone (method 3) with a check at the end. DR within a single provider – a replication rule (method 1) on a bucket with versioning. DR between providers – a scheduled rclone copy, or an event-driven worker if the RPO is to be counted in seconds. A ransomware-resilient copy – any method, as long as the target has versioning and a write lock, and the credentials to it are different from the production ones.

Six pitfalls people fall into on their first deployment

1. The replica dies together with the original. The most important of the pitfalls and the one thing we repeat in this text on purpose: deletion, overwriting and encryption by ransomware are a change like any other and replication will carry it over to the copy – usually within a dozen or so seconds. This is not theory: in January 2025 a campaign was publicly described in which attackers holding valid access keys encrypted the contents of buckets using S3's own mechanism (with a customer key), leaving the victim with storage full of unreadable objects – without breaking into a server, using nothing but ordinary S3 requests. So a copy defends against a hardware and location failure, not against a person or an attacker. The remedy is versioning on the target side plus a write lock (Object Lock, an archive zone) or an "append-only" mode (copy instead of sync). The protocol itself helps here more than you might think: deleting a specific version is never replicated, and in newer configurations neither is a delete marker, by default.

2. Objects from before the rule was switched on will not be copied. Replication covers the future. History requires a one-off push – with a bulk job at the provider or rclone copy – and this is the easiest thing to forget, because the panel says "replication enabled" and everything looks fine.

3. A copy is not everything the bucket had. As standard, objects and their metadata come across. What does not come across by itself: bucket policies, CORS configuration, lifecycle rules, default encryption settings, notification configuration and – in the tool-based methods – version history and delete markers. They do not travel together with the objects, so after a migration you have to recreate them by hand – otherwise the new bucket looks the same and behaves differently.

4. Encryption changes the rules of the game – and does so differently in every method. In AWS native replication, objects encrypted server-side with the default key (SSE-S3) or a customer key (SSE-C) are copied with no extra configuration – but objects with a managed key (SSE-KMS) do not replicate at all by default: you have to enable them explicitly in the rule and point at a key created in the destination region. The trap is insidious, because saving such a configuration with an incorrect key ends with an "OK" response, and replication only fails on the first object. In the tool-based methods it is exactly the other way round: there it is SSE-C that blocks server-side copying – storage without the key will neither decrypt the source nor encrypt the copy, so the bytes have to pass through a process that knows the key (in Ceph RGW server-side copying of encrypted objects is not implemented at all – CopyObject responds NotImplemented). We took the three encryption variants apart in a separate piece on SSE-S3, SSE-KMS and SSE-C.

5. "It copied" is not the same as "it matches". Large files are uploaded in parts (multipart), and their ETag stops being a plain MD5 sum – comparing by it gives false results. The same pitfall comes back from another angle with native replication: if an object on the source is unencrypted and the destination bucket has default encryption enabled, the replica will get a different ETag from the original – and every script comparing ETags will announce a divergence that is not there. The acceptance check is therefore done with checksums (rclone check --checksum) or by comparing the number of objects and the total sizes on both sides. Until you have made that comparison, the migration is only a declaration.

6. A silent replication failure. Replication breaks discreetly: expired credentials, a changed policy, a full target, a stopped worker. The bucket still accepts writes, the panel does not shout, and the copy simply stops growing. Worse: in native replication an object that failed to copy will not be retried by itself – to catch it up you have to upload it again or run a bulk job. Switching the rule off "for a moment" works the same way: after switching it back on, the backlog is not caught up automatically. That is why monitoring has to be added to each of these methods – sync status (radosgw-admin sync status), latency metrics and the number of pending objects at the provider, notifications about failed replication. Or the simplest possible sensor: an alert when the number of objects in the copy stops keeping up with the source.

What does a second copy cost – and in which jurisdiction does it sit?

With hyperscalers, transfer leaving the region is chargeable; with continuous replication you pay it for every change, and with a full migration – for the whole volume. On top of that come charges for the requests themselves, and with native replication those can surprise you: copying a single object means as many as several reads and a write on the source side. So the bill for a second copy has three lines, not one: storage, requests and transfer. One piece of good regulatory news: the EU Data Act is phasing out charges for switching providers as such – from January 2027 they may not be levied – but this does not apply to ongoing replication, because that is ordinary egress traffic, not a migration on termination of a contract. We calculated what this can amount to over a year in the pieces on the public cloud bill and on the differences between a hyperscaler and a local provider.

The other side of the same bill is legal: a replica has a location. A copy "somewhere in the cloud" can be a copy outside the European Economic Area, which is a legal event, not only an operational one. Standing up a replica in another country is a change in the processing: it touches the data processing agreement, the record of processing activities and the risk assessment, not just a configuration file. So the location of both ends is settled before the first transfer, not after the audit – we wrote about why the question "where do the data and their copies physically sit" also has a jurisdictional dimension in the article on European cloud computing.

How do we do it in WebDisk Files?

In the organization panel of WebDisk Files – in the part available to the organization administrator – there is a feature with a name that says it plainly: Bucket replication. It answers the scenario from the list at the beginning that customers ask about most often – "we have data with another provider and we want it with you" – and it does so without a terminal and without an intermediary machine on your side, within the pool of free migrations and without asking us for permission.

It looks like this:

You provide the source endpoint address, the access key and the secret key – to any publicly available S3-compatible storage. From a list you pick the provider type (AWS, Ceph/RGW, MinIO, Wasabi, DigitalOcean Spaces or "other, S3-compatible"), because different implementations have their own small quirks and it is worth accounting for them up front.
You click "Show buckets". We check the credentials straight away and show the list of buckets visible to those keys – so a typo in the key shows up in the first second, not after an hour of copying. (The key has to have the right to list buckets – you do not type the source name by hand, you pick it from a list.)
You choose the source bucket, name the new storage space on our side and choose the copy speed. The basic speed is 50 MB/s, the highest available today – 250 MB/s; the chosen limit applies for the whole migration, so the copying does not saturate the link in unpredictable ways.
You press "Synchronize" and watch the progress: the bytes copied and the percentage of the whole, the current speed, the estimated time to completion, and if something went wrong – an error message on the migration. You can stop the migration at any moment – the files already copied stay in the new storage space.

Underneath it does exactly what we described in method 3, only as a service: for every migration we start a separate, single-use worker with rclone. The worker fetches its configuration with a token valid only for that one migration, reports progress to the panel with the same token, and disappears once the work is done. A few design decisions worth knowing about:

We keep the secret key to the source encrypted in the database (the access key identifier, as in any S3, is not a secret). To write to the destination storage the worker uses the organization's own S3 keys – the same ones you have in the panel; it receives them together with the configuration, for the duration of a single job, and they do not stay in the migration record.
The source address is validated before the server connects to it. We reject addresses pointing at internal networks and metadata services – this is standard protection against abuse of a form that by definition accepts any URL from the user.
The destination storage is created as an ordinary storage space, visible in the panel and accessible just like the others: through the browser, the desktop client and over S3.

The most important sentence about billing: this feature is included in the price of the service. We charge nothing for the transfer – neither for the volume nor for the chosen speed; every organization has a pool of free migrations, and if you need more, just write to us. A failed or stopped migration does not use up the pool, so an attempt can be repeated at no cost – just remember that the destination storage space with the files already copied stays and counts towards your storage, so if you do not want it, simply delete it.

We also have to say honestly what this feature is not. It is a one-off copy, not continuous replication: it copies the current state of the source bucket, it does not watch over it indefinitely and it does not carry version history across from the source. If you need a copy kept up to date, in Files you have everything method 3 requires: in the panel you will find the endpoint address and the S3 keys, so rclone, restic, Veeam, the AWS CLI or your own script will work with your storage space exactly as with any other S3 – in both directions. That, incidentally, is our answer to the question about vendor lock-in: the same tools that bring data in can take it out, without asking us for an export.

The layers that complete the picture of a second copy in WebDisk Files:

Durability inside the cluster – every object is kept in three copies on separate hosts of our Ceph cluster, the same one the Object Storage service runs on. This is the layer from the previous section: it defends against a disk and node failure, not against deletion.
Versioning – a write under an existing name creates a new version, and the previous ones remain available for restoring. This is the first real protection against "someone overwrote the file".
Anty-Ransomware – WORM-class storage based on S3 Object Lock, in which for a chosen period (from 90 days to 7 years) no S3 request will delete or change a written version. This is the layer replication will never replace – and the other way round: replication is what Object Lock does not do.
Encryption – SSE-S3 at the storage-space level or SSE-C with the key on your side, to choose from depending on who is to hold the key.

The set we recommend in practice to customers with a real continuity requirement: working data in Files, a backup copy to a separate storage space with Anty-Ransomware (because nobody will delete that one) and – if the DR scenario calls for it – a third copy with a completely different provider, made with a scheduled rclone copy. Three copies, two different systems, one outside our infrastructure – that is the 3-2-1 rule, just written out in specifics.

Frequently asked questions

Does replication replace a backup? No, and this is the most important thing to remember from this article. Replication keeps a copy consistent with the source, so it also propagates deletion, overwriting and the encryption of files by ransomware. It becomes a backup only once the target has versioning and a deletion lock – or when the replication never deletes anything by design.

Does replication slow down writes to the bucket? In methods 1–4 it does not: the copying is asynchronous and happens after the write has been acknowledged. The only one that slows things down noticeably is dual writing in the application, because there the user waits for both storage systems at once.

How long does moving 10 TB take? The arithmetic is merciless: at 50 MB/s that is about two and a half days of continuous transfer, at the highest speed tier – a dozen or so hours. In practice the bottleneck is often not the bandwidth limit but the source (request limits at the provider) or the number of objects: a million small files take far longer to copy than one file of the same total size.

Can data be replicated from another provider into WebDisk? Yes – that is exactly the job of the "Bucket replication" feature in the Files panel. All you need is an endpoint plus an access key and a secret key that can see the list of source buckets; we do the rest on our side and you watch the progress bar. The opposite direction is open too: the S3 keys to your storage space are in the panel.

Does replication carry the version history of files? Native S3 replication and multisite – yes, because they operate on versions. Tools such as rclone or aws s3 sync – no: they copy the current state of the objects. If history is to survive the migration, it has to be planned separately; more often it is solved by having the backup keep the history rather than the replica.

What about encrypted files – will they copy at all? It depends on who holds the key and on the platform. With server-side encryption (SSE-S3, SSE-KMS) the copying itself is transparent – with two caveats: in AWS native replication SSE-KMS objects have to be explicitly allowed in the rule with a key in the destination region indicated (see pitfall 4), and in Ceph RGW server-side copying of encrypted objects is simply not implemented. With customer-key encryption (SSE-C) every copy has to pass through a process that knows the key – which is why such objects are copied with a tool that has the key, not with a "copy server-side" instruction.

How do I check that the copy is complete? By comparison, not by trust. The minimum is a match in the number of objects and the total size on both sides; the serious version is rclone check --checksum (comparing checksums) or, with native replication, checking the replication status of the objects and the latency metrics. A migration without such an acceptance is not finished.

What does keeping a second copy cost? Three lines, not one: storage on the other side, request charges and egress from the source – the last of these you pay on every cycle, not once. With continuous replication count it from the volume of changes, with a migration – from all the data. A copy with the same provider can be free in terms of transfer, a copy "outside" never is.

Where to start if we have no second copy at all today? With the cheapest step that gives you the most: enable versioning where the working data sits, and point the backup copies at a separate storage space with a deletion lock. Only then add replication – and choose it for the problem you actually have (migration, DR, geographical proximity), not for the buzzword.

Is a copy in a second data centre included in the price? Let us separate two things. What is included in the price is durability inside the cluster (three copies of every object on separate hosts) and what is included in the price is migrating data into WebDisk. A copy outside our infrastructure is a separate DR scenario – you can do it yourself, with the S3 keys from the panel and any tool, or get in touch with us and we will design it together. We would rather say this plainly than sell the word "replication" as a promise that means something different with every provider.

Summary

S3 bucket replication is not one technology but five families of solutions with different entry costs and different purposes: a rule in the storage itself, cluster replication, copying with tools, an event-driven worker and dual writes in the application. The choice between them is settled by four questions – about the acceptable data loss, about whether both ends are with the same provider, about version history and about who pays for the transfer.

Three things are worth remembering:

a replica is faithful, and faithfulness can be a flaw – deletion and encryption propagate along with the data, so the second copy has to be hardened with versioning and a write lock if it is to survive a bad day,
a migration ends with verification, not with a "done" message – checksums, the number of objects, the size; and, while you are at it, recreating the policies, lifecycle rules and CORS that do not travel together with the objects,
replication is chosen for the problem, not for the buzzword – the same word means something completely different for a migration, for DR and for protection against ransomware.

In WebDisk Files you have migration from any S3-compatible storage ready in the panel – Bucket replication, with progress tracking, the option to stop it and at no extra charge. Try WebDisk Files or write to us if you want to talk through the architecture of a second copy for your data: we will also advise you when the best solution turns out to be a copy outside our infrastructure.

This article is part of a series on data security in WebDisk services – earlier we wrote about the basics of object storage, encryption at rest (SSE-S3, SSE-KMS, SSE-C), access via STS/SSO and immutable copies with Object Lock. You can run the examples on any S3-compatible platform; the --endpoint-url parameter in the aws CLI points to your provider's endpoint.

S3 bucket replication: five ways to keep a second copy – and what none of them solves