Object Storage

Azion Object Storage is a scalable and secure storage service designed to integrate object storage with the Azion Web Platform using the S3 standard for object operations.

Object Storage allows you to create buckets, which can be used as origins for applications or as directories for real-time object upload. Alongside bucket creation, you possess complete control over storage allocation, bucket and object access management, as well as the ability to upload, change, and delete objects.

Why use Azion Object Storage?

• Minimal Latency: Distribute objects across our global Points of Presence (PoPs), ensuring data is as close as possible to end users.
• Zero Data Transfer Out (DTO) Cost: We don’t charge for data egress over Azion’s infrastructure. This eliminates billing surprises and enables massive scale without exorbitant transfer fees.
• Native S3 Compatibility: Use the S3 protocol for immediate compatibility with the ecosystem of tools, SDKs, and libraries you already use, such as AWS CLI and Boto3.

Best practices for object delivery

When using Object Storage, it’s essential to understand the correct approach for delivering objects to end users to avoid performance issues and throttling.

Use Applications with Connectors for object delivery

The recommended way to deliver objects stored in Object Storage to end users is through Applications connected to your buckets using Connectors. This approach:

• Routes traffic through Azion’s edge network, ensuring optimal performance and low latency
• Provides caching capabilities to reduce origin requests
• Avoids rate limiting on the management endpoint
• Scales properly for high-traffic scenarios

Avoid direct access to the S3 endpoint

The S3 endpoint (s3.us-east-005.azionstorage.net) is designed for object management operations, not for delivering content directly to end users. Direct access to this endpoint for content delivery can result in:

• Rate limiting and throttling (HTTP 429 errors): The endpoint has rate limits and isn’t built to handle the scale required for end-user content delivery
• Suboptimal performance: Requests bypass Azion’s edge network, resulting in higher latency
• No caching benefits: Direct requests don’t benefit from edge caching

Don’t use pre-signed URLs for end-user delivery

Generating pre-signed URLs that point directly to the azionstorage.net endpoint isn’t recommended for delivering objects to end users. When browsers access these URLs, they:

• Bypass Azion’s edge network entirely
• Hit the management endpoint directly, which may trigger rate limits
• Don’t benefit from edge caching and optimization features

Instead, use Applications with Connectors to serve objects through your domain, leveraging Azion’s global edge infrastructure for optimal performance and reliability.

Implementation

Buckets

Buckets are the system used to organize stored objects. Similar to folders, buckets are the top-level containers to store objects. Buckets can be created using the Azion API.

All buckets created with Azion Object Storage are stored in the us-east cloud server.

Bucket names are exclusive between all Azion accounts. Names must range between 6 and 63 characters and must not start with azion. Alphanumeric characters and hyphen (-) are accepted.

Best practices for naming buckets include specifying what types of objects are stored and the type of permissions for the objects. For example, a bucket for an application Banking App in read-only mode could be named banking-app-ro.

Objects

Objects, or files, can be uploaded, modified, downloaded, and removed from buckets using the Azion API, Azion Runtime, and the S3 protocol.

Object key

An object key is a string of characters that composes a unique identifier for objects stored in Object Storage buckets. Through the available tools, users can retrieve a file stored in a bucket by using its object key.

The object key isn’t required to match the original file path or name from the local storage from which it was retrieved, nor contain the original file extension. However, when uploading a local file to a bucket, it’s recommended to attribute the object key after the file to match local storage conventions. For example, for the local file folder/file.png, the object key should be the same.

The object key cannot be changed. Uploading a different object or modifying object contents using an existing key replaces the object. Once an object is replaced, earlier versions can’t be retrieved.

Prefix

A prefix is a combination of paths that simulate a folder hierarchy. Since buckets can’t be organized into folders, you can use the forward slash (/) when creating keys to categorize objects in your bucket into a prefix.

For instance, the list of keys below represents the simulated hierarchy of an application stored in a bucket with prefixes:

The object README.md is located at the root of the bucket. The src prefix corresponds to a folder and contains the objects index.js and index.html. Additionally, the src/assets prefix contains a styles.css object and the src/assets/images prefix, in turn, contains the image.png object.

When creating an Object Storage origin, you can set a prefix to serve to the edge. For instance, using the example above, you can create an origin that only serves the image.png object by setting the prefix to src/assets/images.

Origin

With Object Storage, you can use buckets as an origin in Azion Connectors to retrieve the content of an application.

You can determine if the content is retrieved from the root of the bucket or from a prefix within the bucket.

Operations

An operation refers to any data exchange between a client and Object Storage. All actions related to buckets and objects, such as create, delete, list, and update, are considered an operation. Each time one of these methods is used, an operation is logged through the API or the S3 protocol.

The current release of the Object Storage offers operations listed below, depending on whether the Azion API or the S3 protocol is used.

Operations and Billing

Our billing model is transparent and focused on actual usage, without penalizing your traffic growth:

Note: Storage is calculated per GB/hour, with no minimum retention.

Azion API operations

ListObjects

Retrieves a list of objects loaded into a bucket.

This operation returns details of all objects in the bucket, including the size in bytes and the timestamp of the last modification.

CreateBucket

Creates a new bucket for an account.

ListBuckets

Retrieves a list of buckets associated with an account.

UpdateBucket

Modifies bucket information.

Use this operation to change the access permissions to the objects in the bucket. Buckets cannot be renamed with this operation.

GetObject

Retrieves an object from a bucket.

PostObject

Uploads an object to a bucket. Objects are limited to a maximum size of 20 MB.

For the Azion API, you can specify the MIME type of the object being sent in the body using the Content-Type header. For example, objects with the .txt extension should contain the Content-Type: text/plain header. If the MIME type isn’t specified, Object Storage will attempt to interpret the file type based on the file extension. Alternatively, use the application/octet-stream MIME type to indicate that the data is a binary stream and the server should handle it as raw binary data.

Sending a new object with an object key already in the bucket will replace the previous object.

To upload a binary file such as an image:

PutObject

Uploads an object to a bucket.

Sending a new object with an object key that already existed in the bucket will replace the previous object.

DeleteObject

Removes an object from a bucket.

When you delete an object being served on the edge, it’ll immediately stop being served and will no longer be listed in the bucket.

DeleteBucket

Removes a bucket from an account.

Buckets that contain objects cannot be deleted. After removing the final object from a bucket, there is a 24-hour period before the bucket can be deleted.

S3 operations

If listBuckets is enabled, when attempting to retrieve files that aren’t in the bucket using an S3 credential, the proper 404 Not Found status response returns instead of a 403 Forbidden status. Find out more about S3 capabilities in S3 protocol compatibility.

Read more on Set up S3 credentials for Object Storage.

ListBuckets

Retrieves a list of buckets associated with an account.

S3cmd command: s3cmd ls

HeadBucket

Checks bucket existence and permissions, returning 200 OK if it does exist or 404 Not Found if it doesn’t.

S3cmd command: s3cmd info s3://BUCKET

ListMultipartUploads

Lists in progress multipart uploads in a bucket. It refers to a multipart upload that the S3 has initiated a Create Multipart Upload request but hasn’t yet been completed or aborted.

S3cmd command: s3cmd multipart s3://BUCKET

ListObjects

Returns a list of up to 1,000 objects in the bucket, sorted alphabetically by key. You can use the query parameters to filter the search.

S3cmd command: s3cmd ls s3://BUCKET

For more than 1,000 results, it’s recommended to use ListObjectsV2

ListObjectsV2

Returns a list of up to 1,000 objects in the bucket, sorted alphabetically by key. You can use the query parameters to filter the search.

This limit is a default setup. However, if the search results in more than the maximum result set size, then the first set is returned in the initial response, the <IsTruncated> response element contains the value true and the <NextContinuationToken> element contains a token to retrieve the next set of results.

Use this token as the continuation-token query parameter in another request to retrieve the next set of results.

S3cmd command: s3cmd ls s3://BUCKET

CopyObject

Creates a copy of an object that is already stored.

S3cmd command: s3cmd cp s3://BUCKET1/OBJECT1 s3://BUCKET2/OBJECT2

GetObject

Retrieves an object from a bucket.

S3cmd command: s3cmd get s3://BUCKET/OBJECT LOCAL_FILE

HeadObject

Retrieves metadata from an object without returning the object itself.

S3cmd command: s3cmd info s3://BUCKET/OBJECT

DeleteObject

Removes an object from a bucket entirely.

S3cmd command: s3cmd del s3://BUCKET/OBJECT

DeleteObjects

Deletes multiple objects from a bucket in a single request. In the XML body, provide the object keys and, optionally, version IDs if you want to delete a specific object version.

S3cmd command: s3cmd del s3://BUCKET/PREFIX --recursive

AbortMultipartUpload

Aborts a multipart upload.

S3cmd command: s3cmd abortmp s3://BUCKET/OBJECT Id

CompleteMultipartUpload

Completes a multipart upload by assembling previously uploaded parts.

This operation is part of the S3 multipart upload process, being handled automatically as part of the put process when uploading large files once all parts are successfully uploaded.

CreateMultipartUpload

Initiates a multipart upload and returns a uploadId. This uploadId is used to associate all of the parts in the specific multipart upload.

This operation is part of the S3 multipart upload process, being handled automatically as part of the put process when uploading large files when you use the put command with a large file. There’s no need to explicitly call this operation.

ListParts

Lists parts that have been uploaded for a given multipart upload.

This limit is a default setup. However, if the number of parts is more than the maximum result set size, the first set is returned in the initial response. In this case, the IsTruncated response element contains the value true and the NextPartNumberMarker element contains a token to retrieve the next set of results. Use this token as the part-number-marker query parameter in another request to retrieve the next set of results.

S3cmd command: s3cmd listmp s3://BUCKET/OBJECT Id

PutObject

Uploads an object to a bucket.

S3cmd command: s3cmd put FILE s3://BUCKET/OBJECT

UploadPart

Uploads a part of the file in a multipart upload. s3cmd automatically splits large files into parts and uploads them. You don’t need to manage the parts or call this operation directly manually.

Authentication and permissions

You can manage the following permissions for Object Storage:

• Azion Teams Permissions: operations involving buckets, such as uploading, listing, and deleting objects, always require authentication through the Azion account. Go to Teams Permissions to know more about the available Object Storage account permissions.
• Bucket permissions: manage access from the edge and other users to buckets and objects within buckets; related to the edge_access attribute.
• S3 credentials: manage access for Azion account users through capabilities and assign operation permissions exclusive to S3 protocol access.

Bucket permissions

In addition to the required authentication and necessary permissions, some API operations can be restricted by bucket permissions. The permissions available are:

• Read-only: objects in the bucket can be read but not modified by the Azion Web Platform.
• Read-write: objects in the bucket can be modified by the Azion Web Platform.
• Restricted: objects in the bucket can be modified and read but can’t be accessed by the Azion Web Platform. Restricted buckets can’t be modified using Azion Runtime and can’t be used as an origin for applications.

These permissions are related to the way the edge can access the bucket. For instance:

• If the bucket is set to read-only, the Azion Web Platform can retrieve objects from the bucket but cannot upload or modify them. However, authorized users can continue writing to Object Storage through the API or the S3 protocol.
• If the bucket is set to read-write, the Azion Web Platform and other users can both retrieve and modify objects within the bucket.
• If the bucket is set to restricted, the Azion Web Platform cannot access the bucket’s content. In this case, only authorized users can continue writing to Object Storage through the API or the S3 protocol.

For example, when an external user attempts to send a POST or PUT request to an application using a bucket configured with read-only or restricted permissions as its origin, the edge will deny access and return an error message.

S3 credentials

Object Storage offers compatibility with the S3 protocol through credentials.

Credentials can be created for any bucket that you own or for your account as a whole to manage all your buckets. With them, you can control permissions for operations associated with that credential. The permissions for the credential are exclusive to access through the S3 protocol.

To create a S3 credential, you must use an Azion personal token and run a POST request via API. However, after the credential is created, it works independently from your Azion token. This way, even if the token expires, the credential remains valid.

Once a credential is created, an access key and a secret key are generated, which can be used to set up access to the bucket through the S3 protocol. For security reasons, the secret key won’t be available after the credential is created. Existing credentials can’t be modified in any way.

Once a user’s access is verified, they’re allowed to make operations depending on the capabilities and permissions set for the credential.

Capabilities

You can assign the following capabilities to S3 credentials:

• listFiles: equivalent to ListObjects, returns a list of objects within the bucket.
• readFiles: equivalent to GetObject, returns an object from the bucket through the object key.
• writeFiles: equivalent to PutObject, allows modifying files in the bucket through the object key.
• deleteFiles: equivalent to DeleteObject, allows object deletion through the object key.
• listAllBucketNames: equivalent to ListBuckets, allows you to list all buckets associated with the account.
• listBuckets: if enabled, returns the proper 404 Not Found response when attempting to retrieve files that aren’t in the bucket using the credential.

Capability-to-operation mapping

The following table maps each S3 capability to its associated operations:

S3 protocol compatibility

After an S3 credential is created for a bucket, you can use the S3 protocol (s3://) to execute operations according to the list of capabilities.

The S3 protocol allows you to access buckets and objects using an Object Storage URL. This configuration facilitates file operations through command line interface (CLI) tools, such as s3cmd, database services, or functions.

You can use the access and secret keys provided by the S3 credentials API to set up a connection using the S3 protocol.

To do so, you’ll need the following information:

Limits

These are the default limits: