Object Storage in WebDisk: bucket, object, key – and what you get out of it in practice
WebDisk Blog · category: Cloud computing · reading time: ~9 minutes
In short:- Object storage keeps data as objects inside buckets – not as disk blocks and not as a tree of directories. Every object has its own key, and you talk to it over HTTP, just like to a web page.- It works well wherever a file is written once and read many times: backup, archives, media, static files, shared space for applications. It will not replace a disk for a database.- S3 is today the de facto standard for communicating with such storage – the same tools (aws,s3cmd,rclone) and the same libraries work with us and with other providers. >Not working with a terminal? You can safely skip the command blocks – the rest of the article reads perfectly well without them.
The company network drive fills up faster than anyone planned. Surveillance cameras produce recordings nobody watches, but which nobody is allowed to delete. An application runs in three instances and each one has to see the same files.
All of these problems share a common denominator: data that comes in large volumes, grows unpredictably and has to be available from many places at once. A classic disk in a server copes worse and worse with every terabyte you add. Object storage was created so that this scenario stops being a problem.
In this article we explain how object storage differs from a disk and from a network share, how to understand the three concepts everything revolves around (bucket, object, key), what S3 is really suited for – and what it is not – and how to start working with it in practice. At the end we describe what we offer in this area at WebDisk.
How do block, file and object storage differ?
Block storage is the lowest level: a disk (physical or a network volume) exposes raw blocks to the operating system, and only the file system gives them meaning. The advantage: the lowest latency and full freedom of modification – you can overwrite any 512 bytes in the middle of a file. The drawback: a volume is usually attached to a single machine.
File storage is a network share (NFS, SMB): a directory hierarchy, permissions, file locks, many clients at once. Trouble appears at scale – a tree with tens of millions of files becomes expensive to handle, and the file server can turn into a bottleneck.
Object storage gives up hierarchy and in-place modification. A file lands in the storage as an object: content plus metadata plus a unique key. Want to change it? You write the object anew, in its entirety. In return you get three things the two previous models do not offer so easily: practically linear scalability, access over plain HTTP from anywhere, and redundancy built into the cluster.
- Unit – Block: a block · File: a file in a directory · Object: an object in a bucket
- Access – Block: a disk driver · File: a network protocol (NFS/SMB) · Object: HTTP (the S3 API)
- Modifying a fragment – Block: yes · File: yes · Object: no – the whole object is written
- Typical client – Block: a single machine · File: many machines on the LAN · Object: any application on the internet
- Natural use – Block: a database, an operating system · File: shared documents, home directories · Object: backup, media, archives, application data
These are not competitors, but three layers of the same toolbox.
What are a bucket, an object and a key in S3?
A bucket is exactly what the name suggests: a container for objects – the equivalent of a disk or a network share. The bucket name has to be unique within the platform, because it becomes part of the address. It is at the bucket level that you set most of the things that matter: who has access, whether the data is encrypted, how long it lives.
Object is a single stored entity: the file content, its metadata (MIME type, date, custom headers) and a version identifier, if the bucket has versioning enabled.
The key is the full name of an object in a bucket – and this is where the most common misunderstanding lies. The namespace in S3 is flat: there are no directories, there is one long string of characters. The key 2026/07/faktury/FV-118.pdf does not mean three nested folders, only a single name containing slashes. Tools draw a tree out of it for your convenience, by asking the server for objects beginning with a given prefix.
The consequence is practical: "moving a directory" in S3 is in reality copying and deleting every object separately. It is worth remembering this when planning your key structure – a well-designed prefix (date, customer, data type) later makes searching, access policies and cleanup rules all easier.
What is object storage suited for, and what is it not?
Backup and archive. Today this is the default destination for backup copies – every backup system that matters can write straight to S3. The copy sits outside the server it protects, it is available over the network and it does not require maintaining a tape library. We write about this at greater length in the article Cloud backup – the foundation of IT security.
Media and static files. Photos, video, PDFs, attachments. The application does not have to pass them through itself – it can send the user straight to the storage, taking all the download traffic off its own servers.
A static site or documentation. A bucket with public read serves HTML, CSS and images without any application server. A cheap, resilient and practically maintenance-free solution. One caveat: full website hosting – with an index document, an error page and your own domain – requires website mode on the provider's side; public read alone is enough for serving files, for example images and downloads.
Shared space for applications. This is the scenario where object storage really shines. Several instances of the same application – on virtual machines or in Kubernetes pods – see the same bucket and the same files, without a shared volume and without wondering which node the upload landed on. One device or process pushes data in, another picks it up and processes it.
What object storage will not replace: a disk for a database, a file system for an application that modifies files in place, or storage that requires microsecond latency. Every operation is an HTTP request – it handles parallelism and large objects excellently, but a single write will never be as fast as a write to a local NVMe. There is also no sense in dumping millions of tiny files into S3 without thinking it through: the per-request overhead can then outweigh the cost of the data itself.
Your first bucket: three tools, the same story
To work with S3 you need three things: the provider's endpoint address, an access key and a secret key. The examples below use our public endpoint https://s3.dco.webdisk.io – with another provider, swap the address and everything will work the same way. That, incidentally, is one of the main advantages of the S3 standard: changing providers does not mean rewriting the integration, as we wrote in our article on vendor lock-in.
One thing worth knowing right away: the bucket address can be composed in two ways – by path (endpoint/bucket) or through the host name (bucket.endpoint). The default settings of clients vary, which is why in the examples below we specify path-style addressing explicitly.
s3cmd – the simplest start, convenient for manual operations:
s3cmd --configure # access key, secret key, host: s3.dco.webdisk.io # host_bucket: s3.dco.webdisk.io/%(bucket)ss3cmd mb s3://firma-archiwum # create a buckets3cmd put raport.pdf s3://firma-archiwum/2026/ # upload a files3cmd ls s3://firma-archiwum/2026/ # list a prefixs3cmd get s3://firma-archiwum/2026/raport.pdf # downloads3cmd du s3://firma-archiwum # how much space it takes
aws CLI – the official AWS client, works with any S3-compatible storage:
aws configure # access key, secret key, region: e.g. us-east-1aws configure set default.s3.addressing_style path # path-style addressing# you pass the endpoint with the command…aws --endpoint-url https://s3.dco.webdisk.io s3 ls# …or once, via an environment variable (aws CLI v2 from version 2.13)export AWS_ENDPOINT_URL=https://s3.dco.webdisk.ioaws s3 mb s3://firma-archiwumaws s3 cp ./raporty s3://firma-archiwum/2026/ --recursiveaws s3api head-object --bucket firma-archiwum --key 2026/raport.pdf
rclone – the best one for synchronising directories and moving data between providers:
rclone config # type: s3, provider: Ceph, endpoint as above # the Ceph profile sets path-style addressing itself# copy incrementally: only what has changed goes acrossrclone copy /srv/dane webdisk:firma-archiwum/dane --progress# note: sync DELETES files at the destination that are not present in the sourcerclone sync /srv/dane webdisk:firma-archiwum/dane --dry-run
A rule that saves nerves: run rclone sync with --dry-run first, and only run it without that switch once you have reviewed the list of changes.
How to control access to a bucket: policies and presigned URLs
By default a bucket is private – only the owner of the keys can see it. You open up access deliberately, in one of two ways.
A bucket policy is a JSON document describing whom we allow to do what. The classic example – a bucket serving static files, with public read of objects and private write:
{ "Version": "2012-10-17", "Statement": [{ "Sid": "PublicRead", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::moja-strona/*" }]}
aws s3api put-bucket-policy --bucket moja-strona --policy file://polityka.json
Note the Resource with an asterisk: the policy concerns the objects in the bucket, not the bucket itself. Replacing arn:aws:s3:::moja-strona/* with arn:aws:s3:::moja-strona is the most common beginner's mistake – suddenly nothing works or, worse, too much works.
A presigned URL solves a different problem: "I want to give one person one file, for an hour, without setting up an account for them". It is an ordinary link to an object with a signature and an expiry date appended. Whoever has the link will download the file; once the time is up the link becomes useless, and the bucket itself stays private:
# link valid for one hour (3600 seconds)aws s3 presign s3://firma-archiwum/2026/raport.pdf --expires-in 3600
A presigned URL also works the other way round – an application can generate a link authorising an upload, thanks to which the user's browser sends the file straight to the storage, bypassing the application server. Just bear in mind that such a link is a secret: whoever intercepts it has access to the object for the whole validity period. That is why you set short expiry times and do not send such links through channels you do not control.
A third mechanism – temporary credentials (STS) – replaces permanent keys with a pass valid for an hour and issued on the basis of a company login. We described it in detail in our article on S3 access with a company account (STS + SSO). We have also devoted a separate text to the encryption-at-rest options (SSE-S3, SSE-KMS, SSE-C) – they are worth discussing in every deployment.
What does object storage give you, and what does it not?
What object storage gives you: scaling without planning capacity up front, access over HTTP from anywhere, a standard you can carry between providers, billing for the space you have purchased – which you can expand without migrating data – and a natural place for data you do not want to keep on a production server.
What object storage does not give you: file system semantics (no in-place modification, no locks, no real directories), latency at the level of a local disk, automatic protection against deletion – if you want to survive a user's mistake or ransomware, you need versioning and object immutability (Object Lock), and those have to be deliberately enabled. Encryption at rest alone does not protect against a leak of access keys either: whoever holds valid credentials gets the data decrypted. Security is always a sum of layers, not a single switch.
What this looks like at WebDisk
Our object storage platform runs on Ceph and its S3 gateway (RADOS Gateway) – open source software used by many cloud providers. This is not a marketing detail: it means you talk to us with the same API you use to talk to AWS, and with the same clients – s3cmd, awscli, boto3, rclone, plugins in backup systems.
The public endpoint is https://s3.dco.webdisk.io. The storage is tuned for the cost of storing large data sets – archives, backup copies and media. A bucket can be encrypted server-side (an option when it is created, permanent). Versioning and object immutability (Object Lock) are available – immutability is enabled when the storage is created and is a one-way decision. Logging in to S3 with a company account (STS) is being prepared as a separate service – we described the mechanism itself in the article linked above. You order storage in the panel and buy extra space as your needs grow. The current scope of features and the price list can be found on the Object Storage service page – and if you are wondering where your data physically sits, we wrote about it in our article on Polish cloud computing.
Frequently asked questions
Do I have to be a developer to use S3?
No. For manual work, graphical clients are enough (e.g. Cyberduck, WinSCP, S3 Browser) or rclone in directory synchronisation mode. The terminal is convenient, but not obligatory.
How does a bucket differ from a network drive?
Above all in the absence of a file system. You will not map a drive letter to a bucket and you will not open a file in it for in-place editing – objects are downloaded and sent back whole. In return, the bucket is available from anywhere on the internet, and you increase capacity by buying additional packages – without moving data and without downtime.
Are there directories and folders in S3?
No – the namespace in a bucket is flat. Slashes in an object's key (e.g. 2026/07/raport.pdf) are part of the name, not nested folders; tools draw a directory tree out of the prefixes for your convenience. That is why "moving a directory" is in practice copying and deleting every object separately.
Is data in a bucket safe if somebody deletes a file?
Only if you enable versioning or an immutability lock. By default a deleted object is gone. This is the same principle as with any other storage: a backup copy and retention are a separate decision, not a built-in property.
How do I move a few terabytes from another provider?
The simplest way is rclone with two remotes configured – it copies between them directly and resumes an interrupted transfer. With large data sets it is worth raising the parallelism (--transfers) and moving the data prefix by prefix.
Is S3 suitable for data I reach for once a year?
Yes – and this is one of the best scenarios. An archive in object storage costs little, and the data remains instantly available, without restoring from tape.
Summary
Object storage is not "a better disk" – it is a different tool, designed for a different usage pattern: write once, read many times, from many places, at a scale you do not want to plan up front. When that pattern fits your data, S3 simplifies the architecture more than any additional file server. When it does not fit – it is better to know that before the application starts fighting the storage.
Do you have a specific case and do not know whether S3 is the right choice – or do you simply want to try it out? Write to us: we will go through it together, help you choose the configuration and issue access to a bucket.