Uploading and downloading objects in Object Storage

In Object Storage, you work with files and folders called objects and place them in containers called buckets. This guide will help you upload files to buckets and download them.

Prerequisites

Web console

Make sure you are in a group that has at least the editor role within your tenant; for example, the default editors group. You can check this in the Administration → IAM section of the web console.

AWS CLI

Install and configure the AWS CLI. It provides simple commands for object management, like aws s3 cp to move objects between a bucket and a local computer and aws s3 rm to delete them.

For more details on installing and configuring, see How to get started with Object Storage: Create your first bucket and the guide on the AWS CLI.

Object naming requirements

Each Object Storage object has a key — object ID in a string format. This key can contain prefixes — they act similar to directories, organizing objects into groups.

To avoid issues with your objects, follow these requirements:

• Object keys can be up to 1024 bytes long, case sensitive.
• Use UTF-8 alphanumeric characters, slashes (/) and certain special characters.

For more information on character usage and which characters to avoid, see the AWS object naming guidelines.

How to upload

Uploading a single file

Web console

1. In the sidebar, go to Storage → Object Storage.

2. Select the bucket where you want to upload a file.

3. Upload a file:

   1. Click Add → Object.
   2. In the window that opens, select the file to upload.
   3. Click Upload.

4. Check that the Objects tab shows the new object.

AWS CLI

The AWS CLI does not support uploading objects to a storage class that differs from the bucket’s default. To do that, use s5cmd.

This command uploads a specified local file to your Object Storage bucket with a specified prefix:

aws s3 cp <source/path> s3://<bucket_name>/<object_key>

The object is uploaded to the storage class that is set as default for the bucket.

You can find more information on the cp command in the AWS CLI reference.

s5cmd

This command uploads a specified local file to your Object Storage bucket with a specified prefix:

s5cmd cp <source/path> s3://<bucket_name>/<object_key>

The object is uploaded to the storage class that is set as default for the bucket. If you need a different class, add the --storage-class STANDARD or --storage-class ENHANCED_THROUGHPUT parameter to this command.

Uploading a folder

You can use the web console to create a folder in a bucket and upload files to it. With the AWS CLI, you can upload an entire local folder including its contents.

Web console

To create a folder:

1. In the sidebar, go to Storage → Object Storage.
2. Select the bucket where you want to create the folder.
3. Click Add → Folder.
4. In the Create folder window, enter a name and click Create.

After you create the folder, you can add files to it.

AWS CLI

There are two ways to upload the contents of a specified local folder to your Object Storage bucket with a specified prefix:

• cp command copies all the files from the source:

  aws s3 cp --recursive \
      <source/path/> s3://<bucket_name>/[<prefix_for_object_keys>]/
  

• sync command only copies new and updated files compared to the target:

  aws s3 sync <local/destination/path/> s3://<bucket_name>/[<prefix_for_object_keys>]/
  

You can find more information on these commands in the AWS CLI reference:

• cp
• sync

If the name of your local folder contains spaces, put it in single quotation marks. For example: Documents/'My ML configurations'/.

How to download

You can download a single object from an Object Storage bucket with the web console or AWS CLI. The AWS CLI also lets you download all objects with a specified prefix.

Web console

1. In the sidebar, go to Storage → Object Storage.
2. Select the bucket that contains the object.
3. In the row of the object to download, click → Download.

AWS CLI

To download a single object with a specified prefix from your Object Storage bucket to a destination on your local machine:

aws s3 cp s3://<bucket_name>/<object_key> <local/destination/path/>

To download all the objects with a specified prefix, use one of the following commands:

• cp command:

  aws s3 cp --recursive \
      s3://<bucket_name>/[<prefix_for_object_keys>] <local/destination/path/>
  

• sync command:

  aws s3 sync s3://<bucket_name>/[<prefix_for_object_keys>] <local/destination/path/>
  

You can find more information on these commands in the AWS CLI reference:

• cp
• sync

If the name of your local folder contains spaces, put it in single quotation marks. For example: Documents/'My ML configurations'/.

Example with the AWS CLI

Assume that you already have an Object Storage bucket named quickstart-bucket. The example below shows how to upload objects to your bucket and download them using the AWS CLI.

1. Create a local folder from which you’ll upload files to your bucket:

   mkdir lorem-ipsum
   

2. Create the files to upload. Run the commands from the code block below:

   echo 'Lorem ipsum odor amet, consectetuer adipiscing elit. Posuere ullamcorper hendrerit faucibus pellentesque sociosqu montes. Tempus ut sit rutrum etiam sodales, porttitor congue condimentum nulla. Facilisis sodales habitant mi justo duis. Hendrerit praesent sit facilisi congue felis primis lobortis mattis nibh. Augue netus laoreet magna class enim fringilla molestie ipsum. Quisque natoque ligula lobortis accumsan potenti? Convallis pellentesque magna enim suscipit risus mauris molestie parturient quam.' > lorem-ipsum/lorem.txt
   echo 'Euismod porta phasellus bibendum urna dolor tincidunt dapibus eu maecenas. Aliquet eget gravida; tempus curae potenti class. Lectus nibh nam donec sagittis donec felis posuere libero. Suspendisse accumsan suscipit blandit orci platea iaculis ac. Suspendisse egestas aptent aliquam sapien ut maximus. Litora ridiculus augue mi, ut aliquam amet. Fusce dignissim nulla venenatis dis himenaeos habitasse. Blandit primis massa eros gravida rhoncus nascetur nulla' > lorem-ipsum/euismod.txt
   echo 'Litora integer iaculis libero dui pretium porta scelerisque. Dis laoreet ipsum porta viverra ipsum sem feugiat. Arcu ex natoque commodo faucibus facilisis vivamus sagittis; ultricies quis. Senectus vivamus nec cras porttitor penatibus. Ipsum vitae integer elementum; sem gravida etiam. Vivamus purus nunc nunc et aenean class. Tempor sagittis sociosqu erat class laoreet iaculis nec. Tempus ligula pellentesque a molestie fames elit vivamus.' > lorem-ipsum/litora.txt
   

   This creates three text files in your new lorem-ipsum/ folder:

   • lorem.txt
   • euismod.txt
   • litora.txt

3. Upload all the files from your lorem-ipsum/ folder to your Object Storage bucket with the lorem-ipsum prefix:

   aws s3 cp lorem-ipsum/ s3://quickstart-bucket/lorem-ipsum/ --recursive
   
   Output

   upload: lorem-ipsum/lorem.txt to s3://quickstart-bucket/lorem-ipsum/lorem.txt
   upload: lorem-ipsum/litora.txt to s3://quickstart-bucket/lorem-ipsum/litora.txt
   upload: lorem-ipsum/euismod.txt to s3://quickstart-bucket/lorem-ipsum/euismod.txt
   

4. List the objects with the lorem-ipsum prefix in your bucket:

   aws s3 ls s3://quickstart-bucket/lorem-ipsum --recursive --human-readable --summarize
   
   Output

   YYYY-MM-DD HH:MM:SS  458 Bytes lorem-ipsum/euismod.txt
   YYYY-MM-DD HH:MM:SS  442 Bytes lorem-ipsum/litora.txt
   YYYY-MM-DD HH:MM:SS  505 Bytes lorem-ipsum/lorem.txt
   
   Total Objects: 3
       Total Size: 1.4 KiB
   

5. Download the lorem.txt file from your bucket to the local lorem-ipsum/ folder as lorem-download.txt:

   aws s3 cp s3://quickstart-bucket/lorem-ipsum/lorem.txt lorem-ipsum/lorem-download.txt
   
   Output

   download: s3://quickstart-bucket/lorem-ipsum/lorem.txt to lorem-ipsum/lorem-download.txt
   

6. Check if the downloaded file is there:

   ls lorem-ipsum/
   
   Output

   euismod.txt        litora.txt         lorem-download.txt lorem.txt
   

7. Delete all the files with the lorem-ipsum prefix from your Object Storage bucket:

   aws s3 rm s3://quickstart-bucket/lorem-ipsum --recursive
   
   Output

   delete: s3://quickstart-bucket/lorem-ipsum/litora.txt
   delete: s3://quickstart-bucket/lorem-ipsum/euismod.txt
   delete: s3://quickstart-bucket/lorem-ipsum/lorem.txt
   

8. Remove the lorem-ipsum/ folder from your computer:

   rm -r lorem-ipsum/