Access to buckets for data transfers

When you create a data transfer, you provide credentials that Object Storage uses to access the buckets. These credentials belong to accounts in storage services, such as service accounts in Nebius AI Cloud. The accounts perform actions on buckets and objects during the data transfer, so these accounts must have sufficient permissions to perform these actions.

Required permissions

Before you launch a data transfer, make sure that the accounts at the source and destination have sufficient permissions to perform operations on buckets and objects. In Nebius AI Cloud, you can add a service account to a group and either assign this group roles with necessary permissions or set up bucket policies for the group.

The table below lists the actions that must be permitted and the minimum Nebius AI Cloud roles that contain the permissions:

Transfer side	Actions	Minimum Nebius AI Cloud roles
Source	ListObjects, HeadObject, GetObject	storage.object-lister + storage.object-viewer (combination) or storage.viewer (single role)
Destination	HeadObject, PutObject, CreateMultipartUpload, UploadPart, CompleteMultipartUpload	storage.object-viewer + storage.uploader (combination) or storage.object-editor (single role)

For more information about these actions, see Iterations.

Credentials

Provide credentials when you create a data transfer. The account (such as a service account) uses the credentials to access buckets and objects and operate with them.

Credentials for S3-compatible storage services

For a bucket in Object Storage or another S3-compatible storage service, the required credentials are a pair of keys called access and secret keys, also known as access key ID and secret access key. Add these credentials to the transfer configuration when you create or modify a data transfer:

Web console

In the web console, settings for credentials appear under Source and Destination when you create or modify a data transfer. To enter the access and secret keys, under Credentials, select Access and secret keys.

CLI

Use .{source,destination}.{nebius,s3_compatible}.access_key to add credentials. For example, when you’re transferring data from a Nebius AI Cloud bucket (.spec.source.nebius) to an S3-compatible bucket (.spec.destination.s3_compatible), add credentials like this:

{
  "spec": {
    "source": {
      "nebius": {
        "access_key": {
          "access_key_id": "<source_access_key_ID>",
          "secret_access_key": "<source_secret_access_key>"
        },
        // Other source parameters
      }
    },
    "destination": {
      "s3_compatible": {
        "access_key": {
          "access_key_id": "<destination_access_key_ID>",
          "secret_access_key": "<destination_secret_access_key>"
        },
        // Other destination parameters
      }
    },
    // Other parameters
  }
}

Set the values of .spec.{source,destination}.{nebius,s3_compatible}.access_key.{access_key_id,secret_access_key} securely, for example, by using environment variables.

Terraform

Use {source,destination}.{nebius,s3_compatible}.access_key to add credentials. For example, when you’re transferring data from a Nebius AI Cloud bucket (source.nebius) to an S3-compatible bucket (destination.s3_compatible), add credentials like this:

resource "nebius_storage_v1_transfer" "example" {
  source {
    nebius {
      access_key {
        access_key_id     = var.source_access_key_id
        secret_access_key = var.source_secret_access_key
      }
    }
    # Other source parameters
  }

  destination {
    s3_compatible {
      access_key {
        access_key_id     = var.destination_access_key_id
        secret_access_key = var.destination_secret_access_key
      }
    }
    # Other destination parameters
  }

  # Other parameters
}

Set the values of the var.source_access_key_id, var.source_secret_access_key, var.destination_access_key_id and var.destination_secret_access_key input variables securely, for example, by using environment variables or a secrets manager.

To get access and secret keys for an Object Storage bucket, create access keys for a service account that has the required permissions.

Credentials for source containers in Azure Blob Storage

If your source bucket (container) is in Azure Blob Storage, provide credentials of your storage account in Azure: the name and access key.

Do not confuse a storage account with an Azure account. The storage account is used for data objects, such as blobs or files.

Web console

In the web console, settings for credentials appear under Source when you create or modify a data transfer and select Azure as the source type. To enter the storage account name and access key, under Credentials, select Access key.

CLI

Use parameters in .source.azure_blob_storage:

{
  "spec": {
    "source": {
      "azure_blob_storage": {
        "endpoint": "<storage_account_endpoint>",
        "container_name": "<container_name>",
        "azure_storage_account": {
          "account_name": "<storage_account_name>",
          "access_key": "<access_key>"
        }
      }
    },
    // Destination and other parameters
  }
}

• .spec.source.azure_blob_storage.endpoint: Endpoint of the Azure storage account in the format https://<storage_account_name>.blob.core.windows.net.
• .spec.source.azure_blob_storage.container_name: Name of the source container in Azure Blob Storage.
• .spec.source.azure_blob_storage.azure_storage_account.account_name: Name of the storage account.
• .spec.source.azure_blob_storage.azure_storage_account.access_key: Access key from Azure that the storage account uses for authentication. Set this value securely, for example, by using environment variables.

Terraform

Use arguments in source.azure_blob_storage:

resource "nebius_storage_v1_transfer" "example" {
  source {
    azure_blob_storage {
      endpoint       = var.source_storage_account_endpoint
      container_name = var.source_container_name

      azure_storage_account {
        account_name = var.source_storage_account_name
        access_key   = var.source_storage_account_access_key
      }
    }
  }
  # Destination and other parameters
}

• source.azure_blob_storage.endpoint: Endpoint of the Azure storage account in the format https://<storage_account_name>.blob.core.windows.net.
• source.azure_blob_storage.container_name: Name of the source container in Azure Blob Storage.
• source.azure_blob_storage.azure_storage_account.account_name: Name of the storage account.
• source.azure_blob_storage.azure_storage_account.access_key: Access key from Azure that the storage account uses for authentication. Set this value (that uses an input variable in this example) securely, for example, by using environment variables.

Anonymous access

If a bucket has anonymous access enabled, meaning that all unauthenticated users have the required permissions in the bucket, you can skip providing credentials:

Web console

In the web console, settings for credentials appear under Source when you create or modify a data transfer. To set up anonymous access, under Credentials, select Anonymous.

CLI

Use {"anonymous": {}} in .spec.source.{nebius,s3_compatible,azure_blob_storage} or .spec.destination.s3_compatible. For example, when you’re transferring data from a Nebius AI Cloud bucket (.spec.source.nebius), set up anonymous access like this:

{
  "spec": {
    "source": {
      "nebius": {
        "anonymous": {},
        // Other source parameters
      }
    },
    // Destination and other parameters
  }
}

Terraform

Use anonymous {} in source.{nebius,s3_compatible,azure_blob_storage} or destination.s3_compatible. For example, when you’re transferring data from a Nebius AI Cloud bucket (source.nebius), set up anonymous access like this:

resource "nebius_storage_v1_transfer" "example" {
  source {
    nebius {
      anonymous {}
    }
  }
  # Destination and other parameters
}

Destination buckets in Nebius AI Cloud don’t support the anonymous access setting because Object Storage doesn’t allow anonymous write access.