Migrate from Backblaze to Storj

This article will discuss the migration process from Backblaze to Storj using the rclone tool, a command-line program to manage files on cloud storage.

Prerequisites

Before starting the migration process, you'll need to have the following:

1. Access to both your Backblaze and Storj accounts.

   • Navigate to https://storj.io/signup?partner=rclone to sign up or log in to an existing Storj account.

2. Installed and configured rclone on your machine.

Why Rclone?

Rclone is a command-line program written in Go language, which is designed to sync files and directories from different cloud storage providers. It allows for easy migration, syncs directories and files, checks file hashes, and even modifies drives. It works with a wide range of cloud storage providers, including Backblaze and Storj, which makes it an excellent tool for our use-case.

Alternative to Backblaze

Storj is a great alternative to Backblaze because it utilizes a distributed cloud model allowing for faster global data access and retrieval compared to Backblaze's centralized model. Storj is also more cost-effective with its lower price per gigabyte for storage and bandwidth.

Install rclone

Visit https://rclone.org/install/ for instructions on how to install rclone.

Create Access Credentials

Before using rclone, we'll need to create keyID and applicationKey for Backblaze and an access grant for Storj.

Backblaze Account and Key

Use your Backblaze account credentials to log in to the Backblaze Management Console.

1. Click Application Keys from the left side menu under Accounts

   

2. Click Add a New Application Key button

   

3. Populate the name of the key and Click the Create New Key button

   

4. Save the keyID and applicationKey as you'll need those later to configure rclone

   

Create Storj Access Grant

A Storj access grant is a serialized, self-contained credential that allows users to access a specific bucket, or object, within a Storj project. It encapsulates everything needed for authentication and authorization on the Storj network.

Create Access Grant in the Storj Console:

1. Navigate to Access on the left side menu.

2. Click Create Access Grant under the Access Grant block

   

3. When the Create Access screen comes up, set specifications according to the following guidelines:

   • Type: Access Grant

   • Name: The name of the credentials (e.g. my-access-grant)

   

4. Click Continue to provide permissions

   • Permissions: All

   • Buckets: Feel free to specify the bucket (e.g. my-bucket), or leave as “All”

   • End date: provide an expiration date for these credentials (optional)

   

5. Click Continue to provide Access encryption Information

   In order to see the data uploaded to your bucket in the web console, you must unlock the bucket with the same encryption passphrase as the credentials.

   • Use the current passphrase: this is default option

   • Advanced: you may provide a different encryption phrase either your own or generate a new one.

     • Enter a new passphrase: use this option, if you would like to provide your own new encryption phrase

     • Generate 12-word passphrase: use this option, if you would like to generate a new encryption phrase

   

6. Click Create Access to finish creation of your S3 credentials

7. Click Confirm the Confirm details pop-up message

   

8. Your Access Grant is created. Write them down and store them, or click the Download all button. You will need these credentials for the following steps.

   

Update Rclone config

After getting your access keys for Storj and Backblaze, you need to configure rclone.

We'll edit the rclone config file directly, you can find where it is stored by running the following:

rclone config file

rclone config file

Command output will look like this

Configuration file is stored at: /Users/dan/.config/rclone/rclone.conf

Edit ~/.config/rclone/rclone.conf with the access keys created above and the values below (see sample file).

Backblaze:

• [backblaze]: This is the section name for the configuration.

• type = b2: This refers to the type of storage you are interacting with, in this case 'b2' for Backblaze

• account: Use your Backblaze account ID or keyID. This is a unique identifier for your Backblaze account, used to ensure that rclone accesses the correct account.

• key: Use your Backblaze application key. The application key is a credential that grants rclone the necessary permissions to access and manipulate the data in your Backblaze B2 buckets.

Storj:

• [storj]: This is the section name for the configuration.

• type = storj: This indicates that the type of storage is Storj.

• access_grant: The access_grant is a serialized access grant string which encapsulates all necessary information to list or download objects. Replace with the actual access grant created previously.

~/.config/rclone/rclone.conf

[backblaze]
type = b2
account =  keyID # REPLACE ME
key = applicationKey # REPLACE ME

[storj]
type = storj
access_grant = access_grant # REPLACE ME

[backblaze]
type = b2
account =  keyID # REPLACE ME
key = applicationKey # REPLACE ME

[storj]
type = storj
access_grant = access_grant # REPLACE ME

Migrate data

Once the configuration is done, you can use the rclone sync command to migrate data from Backblaze to Storj. The syntax is as follows:

Replace my-backblaze-bucket with the name of your specific bucket. This command will sync the contents from your Backblaze bucket to your Storj bucket, effectively migrating the data.

rclone sync --progress backblaze:my-backblaze-bucket storj:my-backblaze-bucket

rclone sync --progress backblaze:my-backblaze-bucket storj:my-backblaze-bucket

Post-Migration Steps

After migration, validate the data integrity in your Storj bucket by running rclone check

rclone check backblaze:my-backblaze-bucket storj:my-backblaze-bucket

rclone check backblaze:my-backblaze-bucket storj:my-backblaze-bucket

This command will compare the source (Backblaze) and destination (Storj) and report any discrepancies.

You can also see the contents of your Backblaze bucket in the Storj Web Console.