Creating and launching data transfers in Object Storage

Data transfers allow you to move data between buckets when you want to migrate or replicate it. For more information, see How data transfers work.

Prerequisites

1. Make sure that you are in a group that has at least the editor role within your tenant; for example, the default editors group. You can check this in the Administration → IAM section of the web console.

2. Make sure that your account has access to the buckets. Prepare credentials for the buckets, for example, access and secret keys. For more information, see Access to buckets for data transfers.

3. Configure the web console or a developer tool to work in the correct project:

   Web console

   In the web console, select the project that contains the destination bucket (if it’s in Nebius AI Cloud) or the source bucket (if the destination is in an external storage service).

   CLI

   1. Install and configure the Nebius AI Cloud CLI.

   2. Get the ID of the project that contains the destination bucket (if it’s in Nebius AI Cloud) or the source bucket (if the destination is in an external storage service):

      nebius storage bucket get <bucket_ID> --format jsonpath='{.metadata.parent_id}'
      

   Terraform

   1. Install and configure the Nebius AI Cloud provider for Terraform.

   2. Save to a local variable the ID of the project that contains the destination bucket (if it’s in Nebius AI Cloud) or the source bucket (if the destination is in an external storage service):

      data "nebius_storage_v1_bucket" "nebius_bucket" {
        id = "<source_or_destination_bucket_ID>"
      }
      
      locals {
        transfer_project_id = data.nebius_storage_v1_bucket.nebius_bucket.parent_id
      }
      

How to launch a data transfer

Web console

1. In the sidebar, go to  Storage → Data Transfer and then click  Create data transfer.

2. Configure the data transfer:

   1. Specify its name.

   2. Select the data transfer type: one-time transfer or replication.

   3. Configure the source and destination buckets:

   4. Specify the path to the buckets.

      In the Bucket field, you can optionally add object key prefixes to the bucket name. In the source, you filter objects. In the destination, you overwrite prefixes of the source objects.

   5. Set credentials.

   6. Configure how the data transfer handles destination objects. See the settings under Overwriting destination objects and Unmanaged objects.

   7. (Optional) Configure limits on request rate and bandwidth.

3. Click Create transfer.

CLI

1. Create a transfer.json file with the data transfer configuration and replace values in the file. Use one of the files below, depending on the source and destination storage services:

   Nebius AI Cloud to Nebius AI Cloud

   transfer.json

   {
     "metadata": {
       "name": "<transfer_name>",
       "parent_id": "<project_ID>"
     },
     "spec": {
       "source": {
         "nebius": {
           "region": "<Nebius_AI_Cloud_region>",
           "bucket_name": "<bucket_name>",
   
           // Either:
           "access_key": {
             "access_key_id": "<access_key>",
             "secret_access_key": "<secret_key>"
           },
           // Or, for anonymous access:
           // "anonymous": {}
         },
         "prefix": "<source_object_key_prefix>" // Optional
       },
       "destination": {
         "nebius": {
           "region": "<Nebius_AI_Cloud_region>",
           "bucket_name": "<bucket_name>",
   
           "access_key": {
             "access_key_id": "<access_key>",
             "secret_access_key": "<secret_key>"
           }
         },
         "prefix": "<destination_object_key_prefix>" // Optional
       },
   
       // Either:
       "after_one_iteration": {},
       // Or:
       // "after_n_empty_iterations": {
       //   "empty_iterations_threshold": <number_of_empty_iterations>
       // },
       // Or:
       // "infinite": {},
       //
       // "inter_iteration_interval": "<HhMmSs>", // For example, "1h20m40s"
   
       "overwrite_strategy": "<NEVER|IF_NEWER>",
       "touch_unmanaged": true|false,  // Optional
       "limiters": {  // Optional
         "requests_per_second": <request_rate_limit>,
         "bandwidth_bytes_per_second": <bandwidth_limit>
       }
     }
   }
   

   Nebius AI Cloud to another storage service

   transfer.json

   {
     "metadata": {
       "name": "<transfer_name>",
       "parent_id": "<project_ID>"
     },
     "spec": {
       "source": {
         "nebius": {
           "region": "<Nebius_AI_Cloud_region>",
           "bucket_name": "<bucket_name>",
   
           // Either:
           "access_key": {
             "access_key_id": "<access_key>",
             "secret_access_key": "<secret_key>"
           },
           // Or, for anonymous access:
           // "anonymous": {}
         },
         "prefix": "<source_object_key_prefix>" // Optional
       },
       "destination": {
         "s3_compatible": {
           "endpoint": "<storage_service_endpoint>",
           "region": "<bucket_region>",
           "bucket_name": "<bucket_name>",
   
           // Either:
           "access_key": {
             "access_key_id": "<access_key>",
             "secret_access_key": "<secret_key>"
           },
           // Or, for anonymous access:
           // "anonymous": {}
         },
         "prefix": "<destination_object_key_prefix>" // Optional
       },
   
       // Either:
       "after_one_iteration": {},
       // Or:
       // "after_n_empty_iterations": {
       //   "empty_iterations_threshold": <number_of_empty_iterations>
       // },
       // Or:
       // "infinite": {},
       //
       // "inter_iteration_interval": "<HhMmSs>", // For example, "1h20m40s"
   
       "overwrite_strategy": "<NEVER|IF_NEWER>",
       "touch_unmanaged": true|false,  // Optional
       "limiters": {  // Optional
         "requests_per_second": <request_rate_limit>,
         "bandwidth_bytes_per_second": <bandwidth_limit>
       }
     }
   }
   

   Azure Blob Storage to Nebius AI Cloud

   transfer.json

   {
     "metadata": {
       "name": "<transfer_name>",
       "parent_id": "<project_ID>"
     },
     "spec": {
       "source": {
         "azure_blob_storage": {
           "endpoint": "<Azure_Blob_Storage_endpoint>",
           "container_name": "<container_name>",
   
           // Either:
           "azure_storage_account": {
             "account_name": "<Azure_storage_account_name>",
             "access_key": "<access_key>"
           },
           // Or, for anonymous access:
           // "anonymous": {}
         },
         "prefix": "<source_object_key_prefix>" // Optional
       },
       "destination": {
         "nebius": {
           "region": "<Nebius_AI_Cloud_region>",
           "bucket_name": "<bucket_name>",
   
           "access_key": {
             "access_key_id": "<access_key>",
             "secret_access_key": "<secret_key>"
           }
         },
         "prefix": "<destination_object_key_prefix>" // Optional
       },
   
       // Either:
       "after_one_iteration": {},
       // Or:
       // "after_n_empty_iterations": {
       //   "empty_iterations_threshold": <number_of_empty_iterations>
       // },
       // Or:
       // "infinite": {},
       //
       // "inter_iteration_interval": "<HhMmSs>", // For example, "1h20m40s"
   
       "overwrite_strategy": "<NEVER|IF_NEWER>",
       "touch_unmanaged": true|false,  // Optional
       "limiters": {  // Optional
         "requests_per_second": <request_rate_limit>,
         "bandwidth_bytes_per_second": <bandwidth_limit>
       }
     }
   }
   

   Another storage service to Nebius AI Cloud

   transfer.json

   {
     "metadata": {
       "name": "<transfer_name>",
       "parent_id": "<project_ID>"
     },
     "spec": {
       "source": {
         "s3_compatible_source": {
           "endpoint": "<storage_service_endpoint>",
           "region": "<bucket_region>",
           "bucket_name": "<bucket_name>",
   
           // Either:
           "access_key": {
             "access_key_id": "<access_key>",
             "secret_access_key": "<secret_key>"
           },
           // Or, for anonymous access:
           // "anonymous": {}
         },
         "prefix": "<source_object_key_prefix>" // Optional
       },
       "destination": {
         "nebius": {
           "region": "<Nebius_AI_Cloud_region>",
           "bucket_name": "<bucket_name>",
   
           "access_key": {
             "access_key_id": "<access_key>",
             "secret_access_key": "<secret_key>"
           }
         },
         "prefix": "<destination_object_key_prefix>" // Optional
       },
   
       // Either:
       "after_one_iteration": {},
       // Or:
       // "after_n_empty_iterations": {
       //   "empty_iterations_threshold": <number_of_empty_iterations>
       // },
       // Or:
       // "infinite": {},
       //
       // "inter_iteration_interval": "<HhMmSs>", // For example, "1h20m40s"
   
       "overwrite_strategy": "<NEVER|IF_NEWER>",
       "touch_unmanaged": true|false,  // Optional
       "limiters": {  // Optional
         "requests_per_second": <request_rate_limit>,
         "bandwidth_bytes_per_second": <bandwidth_limit>
       }
     }
   }
   

   For details, see:

   • Examples
   • Reference for the nebius storage transfer create CLI command
   • Transfer configuration

2. Launch the data transfer:

   nebius storage transfer create \
     --parent-id $PROJECT_ID \
     --file transfer.json
   

Terraform

1. Add one of the following data transfer blocks to your Terraform configuration, depending on the source and destination storage services, and replace values in the blocks:

   Nebius AI Cloud to Nebius AI Cloud

   resource "nebius_storage_v1_transfer" "my_transfer" {
     name      = "<transfer_name>"
     parent_id = local.transfer_project_id
   
     source {
       nebius {
         region      = "<Nebius_AI_Cloud_region>"
         bucket_name = "<bucket_name>"
   
         # Either:
         access_key {
           access_key_id     = "<access_key>"
           secret_access_key = "<secret_key>"
         }
         # Or, for anonymous access:
         # anonymous {}
       }
       # Optional:
       prefix = "<source_object_key_prefix>"
     }
   
     destination {
       nebius {
         region      = "<Nebius_AI_Cloud_region>"
         bucket_name = "<bucket_name>"
   
         access_key {
           access_key_id     = "<access_key>"
           secret_access_key = "<secret_key>"
         }
       }
       # Optional:
       prefix = "<destination_object_key_prefix>"
     }
   
     # Either:
     after_one_iteration {}
     # Or:
     # after_n_empty_iterations {
     #   empty_iterations_threshold = <number_of_empty_iterations>
     # }
     # Or:
     # infinite {}
     #
     # inter_iteration_interval       = "<HhMmSs>" # Optional. For example, "1h20m40s"
   
     overwrite_strategy             = "NEVER"    # Or "IF_NEWER"
     touch_unmanaged                = false      # Optional
   
     # Optional:
     limiters {
       requests_per_second      = <request_rate_limit>
       bandwidth_bytes_per_second = <bandwidth_limit>
     }
   }
   

   Nebius AI Cloud to another storage service

   resource "nebius_storage_v1_transfer" "my_transfer" {
     name      = "<transfer_name>"
     parent_id = local.transfer_project_id
   
     source {
       nebius {
         region      = "<Nebius_AI_Cloud_region>"
         bucket_name = "<bucket_name>"
   
         # Either:
         access_key {
           access_key_id     = "<access_key>"
           secret_access_key = "<secret_key>"
         }
         # Or, for anonymous access:
         # anonymous {}
       }
       # Optional:
       prefix = "<source_object_key_prefix>"
     }
   
     destination {
       s3_compatible {
         endpoint    = "<storage_service_endpoint>"
         region      = "<bucket_region>"
         bucket_name = "<bucket_name>"
   
         # Either:
         access_key {
           access_key_id     = "<access_key>"
           secret_access_key = "<secret_key>"
         }
         # Or, for anonymous access:
         # anonymous {}
       }
       # Optional:
       prefix = "<destination_object_key_prefix>"
     }
   
     # Either:
     after_one_iteration {}
     # Or:
     # after_n_empty_iterations {
     #   empty_iterations_threshold = <number_of_empty_iterations>
     # }
     # Or:
     # infinite {}
     #
     # inter_iteration_interval       = "<HhMmSs>" # Optional. For example, "1h20m40s"
   
     overwrite_strategy             = "NEVER"    # Or "IF_NEWER"
     touch_unmanaged                = false      # Optional
   
     # Optional:
     limiters {
       requests_per_second      = <request_rate_limit>
       bandwidth_bytes_per_second = <bandwidth_limit>
     }
   }
   

   Azure Blob Storage to Nebius AI Cloud

   resource "nebius_storage_v1_transfer" "my_transfer" {
     name      = "<transfer_name>"
     parent_id = local.transfer_project_id
   
     source {
       azure_blob_storage {
         endpoint       = "<Azure_Blob_Storage_endpoint>"
         container_name = "<container_name>"
   
         # Either:
         azure_storage_account {
           account_name = "<Azure_storage_account_name>"
           access_key   = "<access_key>"
         }
         # Or, for anonymous access:
         # anonymous {}
       }
       # Optional:
       prefix = "<source_object_key_prefix>"
     }
   
     destination {
       nebius {
         region      = "<Nebius_AI_Cloud_region>"
         bucket_name = "<bucket_name>"
   
         access_key {
           access_key_id     = "<access_key>"
           secret_access_key = "<secret_key>"
         }
       }
       # Optional:
       prefix = "<destination_object_key_prefix>"
     }
   
     # Either:
     after_one_iteration {}
     # Or:
     # after_n_empty_iterations {
     #   empty_iterations_threshold = <number_of_empty_iterations>
     # }
     # Or:
     # infinite {}
     #
     # inter_iteration_interval       = "<HhMmSs>"  # Optional. For example, "1h20m40s"
   
     overwrite_strategy             = "NEVER"     # Or "IF_NEWER"
     touch_unmanaged                = false       # Optional
   
     # Optional:
     limiters {
       requests_per_second      = <request_rate_limit>
       bandwidth_bytes_per_second = <bandwidth_limit>
     }
   }
   

   Another storage service to Nebius AI Cloud

   resource "nebius_storage_v1_transfer" "my_transfer" {
     name      = "<transfer_name>"
     parent_id = local.transfer_project_id
   
     source {
       s3_compatible_source {
         endpoint    = "<storage_service_endpoint>"
         region      = "<bucket_region>"
         bucket_name = "<bucket_name>"
   
         # Either:
         access_key {
           access_key_id     = "<access_key>"
           secret_access_key = "<secret_key>"
         }
         # Or, for anonymous access:
         # anonymous {}
       }
       # Optional:
       prefix = "<source_object_key_prefix>"
     }
   
     destination {
       nebius {
         region      = "<Nebius_AI_Cloud_region>"
         bucket_name = "<bucket_name>"
   
         access_key {
           access_key_id     = "<access_key>"
           secret_access_key = "<secret_key>"
         }
       }
       # Optional:
       prefix = "<destination_object_key_prefix>"
     }
   
     # Either:
     after_one_iteration {}
     # Or:
     # after_n_empty_iterations {
     #   empty_iterations_threshold = <number_of_empty_iterations>
     # }
     # Or:
     # infinite {}
     #
     # inter_iteration_interval       = "<HhMmSs>" # Optional. For example, "1h20m40s"
   
     overwrite_strategy             = "NEVER"    # Or "IF_NEWER"
     touch_unmanaged                = false      # Optional
   
     # Optional:
     limiters {
       requests_per_second      = <request_rate_limit>
       bandwidth_bytes_per_second = <bandwidth_limit>
     }
   }
   

   For details, see:

   • Examples
   • Reference for the nebius_storage_v1_transfer resource
   • Transfer configuration

2. Check that the configuration is correct:

   terraform validate
   

3. Apply the changes:

   terraform apply
   

Examples

One-time transfer between Nebius AI Cloud regions

The following configuration examples show how to create a one-time transfer (a transfer that stops after one iteration) from a bucket in the eu-north1 region to a bucket in the us-central1 region:

Web console

Create the data transfer in the project in us-central1 where the destination bucket is located:

• Type: One-time transfer

• Source: Nebius

  • Region: eu-north1
  • Bucket name (example): source_bucket
  • Credentials: Access and secret keys

• Destination: Nebius

  • Region: us-central1
  • Bucket name (example): destination_bucket
  • Credentials: Access and secret keys
  • Overwriting destination objects: Overwrite if source is newer

CLI

The .metadata.parent_id parameter should be the ID of the project in us-central1 that contains the destination bucket.

{
  "metadata": {
    "name": "eu-north1-to-us-central1-transfer",
    "parent_id": "project-***"
  },
  "spec": {
    "source": {
      "nebius": {
        "region": "eu-north1",
        "bucket_name": "source_bucket",
        "access_key": {
          "access_key_id": "****",
          "secret_access_key": "****"
        }
      },
    },
    "destination": {
      "nebius": {
        "region": "us-central1",
        "bucket_name": "destination_bucket",
        "access_key": {
          "access_key_id": "****",
          "secret_access_key": "****"
        }
      }
    },
    "after_one_iteration": {},
    "overwrite_strategy": "IF_NEWER"
  }
}

Terraform

The parent_id parameter should be the ID of the project in us-central1 that contains the destination bucket.

resource "nebius_storage_v1_transfer" "my_transfer" {
  name      = "eu-north1-to-us-central1-transfer"
  parent_id = "project-***"

  source {
    nebius {
      region      = "eu-north1"
      bucket_name = "source_bucket"
      access_key {
        access_key_id     = "****"
        secret_access_key = "****"
      }
    }
  }

  destination {
    nebius {
      region      = "us-central1"
      bucket_name = "destination_bucket"
      access_key {
        access_key_id     = "****"
        secret_access_key = "****"
      }
    }
  }

  after_one_iteration {}

  overwrite_strategy = "IF_NEWER"
}

Replication between Nebius AI Cloud regions

The following configuration examples show how to create a replication (a continuous transfer that you stop manually) from a bucket in the us-central1 region to a bucket in the eu-north1 region:

Web console

Create the data transfer in the project in eu-north1 where the destination bucket is located:

• Type: Replication

• Source: Nebius

  • Region: us-central1
  • Bucket name (example): primary_bucket
  • Credentials: Access and secret keys

• Destination: Nebius

  • Region: eu-north1
  • Bucket name (example): secondary_bucket
  • Credentials: Access and secret keys
  • Overwriting destination objects: Overwrite if source is newer

CLI

The .metadata.parent_id parameter should be the ID of the project in eu-north1 that contains the destination bucket.

{
  "metadata": {
    "name": "us-central1-to-eu-north1-backup",
    "parent_id": "project-***"
  },
  "spec": {
    "source": {
      "nebius": {
        "region": "us-central1",
        "bucket_name": "primary_bucket",
        "access_key": {
          "access_key_id": "****",
          "secret_access_key": "****"
        }
      }
    },
    "destination": {
      "nebius": {
        "region": "eu-north1",
        "bucket_name": "secondary_bucket",
        "access_key": {
          "access_key_id": "****",
          "secret_access_key": "****"
        }
      }
    },
    "infinite": {},
    "overwrite_strategy": "IF_NEWER"
  }
}

Terraform

The parent_id parameter should be the ID of the project in eu-north1 that contains the destination bucket.

resource "nebius_storage_v1_transfer" "my_transfer" {
  name      = "us-central1-to-eu-north1-backup"
  parent_id = "project-***"

  source {
    nebius {
      region      = "us-central1"
      bucket_name = "primary_bucket"
      access_key {
        access_key_id     = "****"
        secret_access_key = "****"
      }
    }
  }

  destination {
    nebius {
      region      = "eu-north1"
      bucket_name = "secondary_bucket"
      access_key {
        access_key_id     = "****"
        secret_access_key = "****"
      }
    }
  }

  infinite {}
  overwrite_strategy = "IF_NEWER"
}