Self-hosted S3 Compatible Gateway
A download can become a chargeable event for 2 times the actual file size if the gateway is running on another cloud provider. We recommend interfacing with the network directly through the Uplink Library or using our hosted Gateway MT.
For a complete list of the supported architectures and API calls for the S3 Gateway, see Concepts: S3 Compatibility.
✅ 1 CPU
✅ 2GB of RAM
Depending on the load and throughput, more resources may be required.
To save on costs and improve performance, please see this important note on multipart upload part sizes.
✅ Storj DCS Satellite Account
✅ Storj DCS Satellite token or Access grant
Download, unzip, and install the binary for your OS:
curl https://github.com/storj/gateway/releases/latest/download/gateway_windows_amd64.exe.zip -o gateway_windows_amd64.exe.zip; Expand-Archive gateway_windows_amd64.exe.zip -Destination . -Force
curl -L https://github.com/storj/gateway/releases/latest/download/gateway_linux_amd64.zip -O && unzip gateway_linux_amd64.zipchmod 755 gatewaysudo mv gateway /usr/local/bin/gateway
curl -L https://github.com/storj/gateway/releases/latest/download/gateway_linux_arm.zip -O && unzip gateway_linux_arm.zipchmod 755 gatewaysudo mv gateway /usr/local/bin/gateway
curl -L https://github.com/storj/gateway/releases/latest/download/gateway_darwin_amd64.zip -O && unzip gateway_darwin_amd64.zipchmod 755 gatewaysudo mv gateway /usr/local/bin/gateway
docker pull storjlabs/gateway
You have two ways to configure your Gateway ST:
Interactive Setup (only if it is your first setup)
1. Setup your S3 gateway by running the following command and following the instructions provided by the wizard:
gateway setup
gateway setup
docker run -it --rm --mount type=bind,source=/path/to/gateway-config-dir/,destination=/root/.local/share/storj/gateway/ --name gateway storjlabs/gateway setup
2. Enter the numeric choice or satellite address corresponding to the satellite you've created your account on.
The satellite address should be entered as <nodeid>@<address>:<port> for example: [email protected]
3. Choose an access name (this step may not yet be implemented in the version of S3 Gateway you are using - if you don't see this prompt, skip to step 5 below).
If you would like to choose your own access name, please be sure to only use lowercase letters. Including any uppercase letters will result in your access name not getting recognized when creating buckets.
4. Enter the Access token you generated:
5. Create and confirm an encryption passphrase, which is used to encrypt your files before they are uploaded:
Please note that Storj Labs does not know or store your encryption passphrase, so if you lose it, you will not be able to recover your files.
6. Your S3 gateway is configured and ready to use!
You can use two methods to obtain an Access Grant:
Now we got our access grant and can configure the gateway as follows:
./gateway setup --access 14aV.... --non-interactive
gateway setup --access 14aV.... --non-interactive
gateway setup --access 14aV.... --non-interactive
docker run -it --rm --mount type=bind,source=/path/to/gateway-config-dir/,destination=/root/.local/share/storj/gateway/ --name gateway storjlabs/gateway setup --access 14aV.... --non-interactive
This command will register the provided access as the default access in the gateway config file.
It is possible to have several access grants, see how here.
The gateway functions as a daemon. Start it and leave it running.
./gateway.exe run
gateway run
gateway run
docker run -it --rm -p 127.0.0.1:7777:7777 --mount type=bind,source=/path/to/gateway-config-dir/,destination=/root/.local/share/storj/gateway/ --name gateway storjlabs/gateway run
If you want to connect to your S3 Gateway via the network, then you should replace the -p 127.0.0.1:7777:7777 port mapping with -p 7777:7777
The gateway should output your S3-compatible endpoint, access key, and secret key.
If you are interested in more running options, checkout the Gateway ST Advanced Usage page where you can find how to:
Please make sure you have AWS S3 CLI installed.
Once you do, in a new terminal session, configure it with your Gateway's credentials:
$ aws configure---AWS Access Key ID: [Enter your Gateway's Access Key]AWS Secret Access Key: [Enter your Gateway's Secret Key]Default region name: [null]Default output format: [null]$ aws configure set default.s3.multipart_threshold 64MB
Then, test out some AWS S3 CLI commands!
aws s3 --endpoint=http://localhost:7777/ mb s3://bucket-name
aws s3 --endpoint=http://localhost:7777/ cp ~/Desktop/your-large-file.mp4 s3://bucket-name
aws s3 --endpoint=http://localhost:7777 ls s3://bucket-name/
aws s3 --endpoint=http://localhost:7777 cp s3://bucket-name/your-large-file.mp4 ~/Desktop/your-large-file.mp4
aws s3 --endpoint=http://localhost:7777 presign s3://bucket-name/your-large-file.mp4
(This URL will allow live video streaming through your browser or VLC)
aws s3 --endpoint=http://localhost:7777 rm s3://bucket-name/your-large-file.mp4
cp - Copies a local file or S3 object to another location locally or in S3
ls - List S3 objects and common prefixes under a prefix or all S3 buckets
mb - Creates an S3 bucket
mv - Moves a local file or S3 object to another location locally or in S3.
presign - Generate a pre-signed URL for an S3 object. This allows anyone who receives the pre-signed URL to retrieve the S3 object with an HTTP GET request.
rb - Deletes an empty S3 bucket
rm - Deletes an S3 object
sync - Syncs directories and S3 prefixes. Recursively copies new and updated files from the source directory to the destination. Only creates folders in the destination if they contain one or more files
And that's it! You've learned how to use our S3-compatible Gateway. Ideally, you'll see how easy it is to swap out AWS for the Uplink, going forward.
