The Storj Developer Hub

Welcome to the Storj developer hub. You'll find comprehensive guides and documentation to help you start working with Storj as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    Support

Storj Share Daemon (CLI)

Storjshare-Daemon-CLI setup
Step by Step Guide

If you are looking for a affordable board with low energy consumption please have a look at the Pine Rock64 4GB RAM board.

1. Introduction

Storjshare-daemon-CLI is a command line tool that allows the user to rent-out storage space from a variety of storage containers to the Storj network. It is intended for users which want the most light-weight and command line application for renting out storage space. Storjshare-daemon-CLI and Storjshare-GUI (+V.5.0) are compatible, meaning that both packages can run the same node, which removes the necessity of walking through migration steps between the two packages. Both the CLI and GUI versions of Storjshare-daemon can even run the same node simultaneously. With the CLI it is easier than ever to monitor your nodes as it has a variety of monitoring and status options integrated in its engine. Other large advantages include:

  • Allows the creation and execution of an unlimited number of nodes as long as the internet and hardware resources allow it (min 1-2 GB RAM per node).
  • Allows fine-tuning of each of the nodes for maximal performance.
  • Higher performance than the GUI as it does not require the rendering of graphics.
  • Allows the manipulation via a terminal over SSH.

In the next sections we will walk you through setting up three nodes using Storjshare-daemon-CLI. Although three nodes will be created in this guide, the user is free to run a different number of nodes, the setup procedure remains the same.

2. Installing the necessary dependencies

Before installing Storjshare-daemon-CLI it is necessary to get and install a few software packages that are necessary to successfully install and run Storjshare. The steps below should be followed in order.

Note: Please keep all the installed packaged up-to-date to ensure maximum compatibility.

2.1. GNU + Linux & MAC OSX

2.1.1. Installation of the required components

First open a terminal as a casual user and type in the following commands in a sequential order:

  1. wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash
  2. exit from bash shell
  3. run a new shell as casual user
  4. nvm install --lts
  5. restart shell
  6. sudo apt-get update && sudo apt-get dist-upgrade
  7. sudo apt-get install git python build-essential -y
  8. npm install --global storjshare-daemon

Red Hat / Centos

  • yum update
  • yum groupinstall 'Development Tools'
  • You might also find yourself lacking a C++11 compiler which can be found here.

Mac OSX

  • xcode-select --install

Info extracted from here.

2.1.2. NTP synchronization for GNU+Linux systems

Having a system correctly synchronized with NTP is essential to ensure optimal functionality of the Storj Share nodes. If the synchronization is off by more than 500 milliseconds, the nodes will start to fail as it does not keep the same time as all the other nodes on the network. As most messages have a time-stamp, it is essential to have a good synchronization for optimal performance.

Open up a terminal and type in the following commands:

  1. sudo apt-get install ntp ntpdate -y
  2. sudo service ntp stop
  3. sudo ntpdate -s time.nist.gov
  4. sudo service ntp start
  5. timedatectl status
  6. timedatectl list-timezones
  7. sudo timedatectl set-timezone <your timezone>

    e.g. "sudo timedatectl set-timezone CET"

Alternatively

Edit the ntp config file: vi /etc/ntp.conf

You’ll find a lot of lines in that file, but the important ones are the server lines. You can get a list of server addresses at pool.ntp.org, find the preferred ones for your area, and then add them to the file. For example if you are in the US:

  • server 0.north-america.pool.ntp.org
  • server 1.north-america.pool.ntp.org
  • server 2.north-america.pool.ntp.org
  • server 3.north-america.pool.ntp.org

Then you’ll need to restart or start the NTPD service:

  • /etc/init.d/ntpd restart

(you could also use nano instead of vi for editing)

2.2. Windows

The process of setting setting-up the CLI on windows is a bit more of a lengthy process however still very simple overall.

2.2.1. Installing Git

In some of the proceeding installation steps it will be necessary to install software packages using the command line terminal (CMD). Git allows us to easily install these packages using the command line. To install Git just go to the following link, figure 2.1. The download should start automatically and the file will be saved to the ‘downloads’ folder. After the download is completed move to the download directory and double click on the downloaded file. Click "next" on all installation windows (figure 2.2). After the installation is completed exit the program.

*Figure 2.1. Git [installation page](https://git-scm.com/download/win)*

Figure 2.1. Git installation page

*Figure 2.2. Git installation windows.*

Figure 2.2. Git installation windows.

2.2.2. Installing NodeJS (LTS)

After the installation of Git for windows we install NodeJS (LTS). NodeJS is available for both 32-bit (x86) and 64-bit (x64) systems. The correct version matching the system architecture should be downloaded. To check the correct version open the control panel and type in "system": (Control Panel\System and Security\System). This will open up a window that shows the architecture of our system (figure 2.3). The system in this guide is a 64-bit (x64) machine, indicating that the 64-bit version of NodeJS should be downloaded (figure 2.4).

*Figure 2.3. System architecture check.*

Figure 2.3. System architecture check.

After the system check, the correct version can now be downloaded from the NodeJS download page. Note that "LTS" must be selected (selected by default). Now click on the correct bit version and the executable should start downloading (figure 2.4). Clicking on "windows installer" will automatically detect the system architecture and download the correct executable. Note however that a manual check for the correct system bit version is always advisable as browsers such as Google chrome are notorious for downloading only 32-bit versions even if one is on a 64-bit machine. After the download completes double click on the executable to initiate the installation.

*Figure 2.4. NodeJS [download page](https://nodejs.org/en/download/)*

Figure 2.4. NodeJS download page

It normally takes a few seconds for NodeJS to calculate the necessary space required for the installation, when completed the "Next button" can be pressed. Press "Next" on all windows and finally click on "install". It should now install NodeJS. After the installation click on "finish", the node installation is now completed (figure 2.5).

*Figure 2.5. NodeJS (LTS) installation windows.*

Figure 2.5. NodeJS (LTS) installation windows.

2.2.3. Git and NodeJS installation check

We can now check if the previous packages installed correctly by opening a command line terminal (CMD) either by right-clicking on the windows start icon and selecting "command prompt", or by pressing the "windows + R" key and typing in cmd and then pressing enter. Both these methods should work. Now type in the following commands (figure 2.6):

  • node --version

A correct version should be returned. Now type in:

  • npm --version

A correct version should also be returned. Only if both these commands return a correct value can we proceed to the next step.

*Figure 2.6. Node and Git installation check.*

Figure 2.6. Node and Git installation check.

2.2.4. Installing the Windows Build Tools

We now proceed to the installation of the tools necessary to build Storjshare-daemon-CLI. More information on these tools can be found here. To be able to install the windows build tools it is necessary to use an elevated PowerShell window. To open this window execute the following steps:

  1. Click on the windows start icon.
  2. Type "PowerShell", this will start a windows search.
  3. Below "Best match" right-click on "Windows PowerShell (desktop App)" and "Run as Administrator" (figure 2.7). This should now launch a PowerShell window as Administrator.
*Figure 2.7. Start a PowerShell terminal as Administrator.*

Figure 2.7. Start a PowerShell terminal as Administrator.

Once the PowerShell terminal opens we can install the windows build tools by executing in the following command in the terminal:

  • npm install --global windows-build-tools

Press enter and the tools will start installing (figure 2.8). The installation can take some time due to the installation of the Visual Studio Build tools. During and after the installation no errors should appear and if both installations succeeded the command terminal should return the following green colored text "successfully installed Python 2.x" and "successfully installed Visual Studio Build Tools".

*Figure 2.8. Windows Build Tools installation (Note, this package was already installed in this system).*

Figure 2.8. Windows Build Tools installation (Note, this package was already installed in this system).

Note: If any errors appear (e.g. "weak modules not found") please join our community where we will help to resolve the installation issues.

2.2.5. NTP synchronization for windows

Windows machines are notorious for keeping a poor time synchronization with net time (not local machine time). If the synchronization is off by more than +-500 milliseconds the node will start to fail as it does not keep the same time synchronization as all the other nodes on the network. As most messages have a timestamp, it is essential to have a good synchronization for optimal performance. We will use a simple tool called NetTime which can be downloaded here.

  • We will download and install the latest stable version (Figure 2.9).
*Figure 2.9. NetTime download page.*

Figure 2.9. NetTime download page.

  • Once downloaded, open the download directory and double click on the executable to initiate the installation.
  • Once the installation window appears, click on "Next" on all the following installation windows (Figure 2.10).
*Figure 2.10. NetTime installation.*

Figure 2.10. NetTime installation.

  • Finally, click on "Finish".

To open the program, open the system tray and double click on the NetTime icon (Figure 2.11). This will open the main NetTime program.

*Figure 2.11. NetTime tray icon.*

Figure 2.11. NetTime tray icon.

  • Now click on “Setting” which will bring up the settings menu.
  • Next head over to pool.ntp.org and select the region you live in, for example Europe (pool.ntp.org/zone/europe).
  • Scroll down to the country you live in and click on it. This should open up a page with various time servers (e.g. Server 3.pt.pool.ntp.org), we only need to copy the last part of the string after “Server“ (e.g. 3.pt.pool.ntp.org), as shown in Figure 2.12.
*Figure 2.12. Time servers for a specific country (Portugal In this example).*

Figure 2.12. Time servers for a specific country (Portugal In this example).

  • Now paste the chosen server name into the first “Time servers” field in the NetTime Options (Figure 2.13).
  • Set the "update interval" to 15 minutes, leave all the other fields as default values, and click on “OK” (Figure 2.13).

Tip: The default setting "If time adjustment is greater than 2 min ..." is not always good enough (if 'Offset + Lag' deviates more than 500 ms), one can change that value to a few seconds-milliseconds if time sync is not successful with the steps above.

*Figure 2.13. NetTime settings with new parameters.*

Figure 2.13. NetTime settings with new parameters.

  • Finally, click on "Update Now" to synchronize NetTime with your own specified time server, which should lower the "Offset" and "lag" parameters (Figure 2.14).
*Figure 2.14. NetTime after successful configuration, **the closer to zero the offset and lag parameters are, the better**.*

Figure 2.14. NetTime after successful configuration, the closer to zero the offset and lag parameters are, the better.

After this step all the dependencies are successfully installed that are required to install and run Storjshare-Daemon-CLI. If any of the above installations failed please contact Storj support.

2.3. Installing Storjshare-daemon (CLI)

The following step is the most crucial step in this guide. We now install Storjshare-daemon-CLI by opening a command terminal and pasting in the following command (figure 2.15):

  • npm install -g storjshare-daemon

This will install the package globally using Node Package Manager.

If no red-colored text lines appear during the installation we can check if the installation of the CLI was successful. This can be done by typing in the following into the terminal:

  • storjshare --version

If this command returns a valid version number (figure 2.15) instead of an error, it indicates that the Storjshare-daemon-CLI installation was successful.

*Figure 2.15. Storjshare-daemon-CLI installation and installation check.*

Figure 2.15. Storjshare-daemon-CLI installation and installation check.

Note: Please update the CLI as soon as new versions are available here.

2.4. Text Editor

During the configuration of Storjshare it is preferable to have a text editor that structures the text in a user friendly manner.

2.4.1. GNU + Linux & Mac OSX

There are two main text editors, Vim and emacs that are included by default and thus no installation is required. To access a file with the text editors (which we will do later) one can CD to the folder containing the text file and execute the following command:

  • vi <name of text file>

    or
  • emacs <name of text file>

2.4.2. Windows

Notepad ++ is text editor of choice for windows. Notepad can be downloaded here (figure 2.16).

*Figure 2.16. Notepad++ [download page](https://notepad-plus-plus.org/download/).*

Figure 2.16. Notepad++ download page.

After the download is completed double-click on the executable in the download folder to launch the installation. Click "Next" on all the installation windows (figure 2.17).

*Figure 2.17. Notepad++ installation.*

Figure 2.17. Notepad++ installation.

3. Storjshare-Daemon (CLI) setup and configuration

3.1. Storage location

Before we can start the actual configuration of the farming nodes it is necessary to pre-determine the location where we want to store the data for each node. The storage container and location will be different for each individual, the following storage container devices are most commonly used with storjshare:

  1. Computer internal drives (e.g. HDD’s, SSD’s)
  2. External Hard drives.
  3. Network attached storage (e.g. NAS)
  4. Other storage devices (SD cards, USB sticks, Micro-USB cards...)

Each of the devices above have different characteristic and offer different levels of performance. NVME-SSD’s offer fast read-write speeds which has a large advantage over slower Solid state external storage drives. Although it should be noted that the impact of the read-write speed largely depends on the internet connection speeds, having a slow internet connection can completely nullify the read-write speed advantage of modern SSD’s as the limiting factor is network transfer speed. For gigabit networks however this is not the case, giving SSD’s the upper hand.

  • You can only run one node per logical CPU core (e.g. i7 has 4 cores and thus you can run 4 nodes).
  • The maximum storage size of a single storjshare-daemon node is 8.0 TB.
  • If one wants to rent out more storage space to the network it will be necessary add more nodes.

There is no minimum size for a node and thus even laptops with only a few MB-GB of free data can successfully run storjshare-daemon-CLI. Multiple nodes can be configured if one has many storage devices that are not agglomerated into one partition.
For example if one wants to rent out the extra space on an internal SSD drive of the computer and that of an external backup device, at least two nodes must be configured. In this guide 3x nodes will be configured to store the farming data on a 16GB external SD card. This is purely an example and the exact same configuration can be applied for other storage containers and location. The method remains the same. Firstly we have to create 3x folders for each node where the data will be stored.

3.1.1. GNU + Linux

First make sure the disk is mounted within your user account, if so, proceed.

Enter the storage container using ls/cd and type in the following commands:

  • mkdir Node_1 Node_2 Node_3

This will create three directories in which the data for each of the nodes will be stored. You can name the directories as you like, however it is advisable to remain consistent with the folder naming when one wants to add many nodes to prevent confusion.

3.1.2. Windows

Open the storage container folder and hold "CTRL + SHIFT + N" to create a new folder. You can name the folder as you like, however it is advisable to remain consistent with the folder naming when one wants to add many nodes to prevent confusion (figure 3.1).

*Figure 3.1. Three folder are created in the storage container. One folder for each node.*

Figure 3.1. Three folder are created in the storage container. One folder for each node.

3.2. Port Forwarding

To successfully connect to the outside world storjshare can either use NAT traversal strategies by accessing the network via UPnP or by using TCP forwarded ports. UPnP is configured by default and requires no configuration from the user unless the router does not have UPnP enabled, in that case the user has to manually enable UPnP in the router settings. Configuring TCP port forwarding is the preferred method for users seeking the most stable configuration possible.

If your router does not support UPnP and TCP port forwarding have a look at this guide

One of the main hurdles for setting up a good working node is the port forwarding step, which has been the cause for many bad performing farmers. In this section we will look how to configure TCP port forwarding for our 3x nodes.

3.2.1. Set the private IP address to static

Most consumer routers used DHCP which gives your computers on the local network a new IP address after a certain amount of time (normally 24 hours). If our computer is given a new private IP address, suddenly the specified Storj Share TCP port does not match the private IP address of our system anymore in the router configuration. When this happens our computer will not be able to access the port anymore and the port will thus be closed. To prevent this from happening you have to setup a static private IP address. There are plenty of tutorials on this topic on the internet, including:

3.2.1.1. Windows

3.2.1.2 Linux based systems

When done make sure to restart your computer and check if the private IP address is still static, if the IP address changed after a reboot it means that the IP is not static.

3.2.2. Ensure your public IP address is static

To be able to use TCP port forwarding in Storj Share it is necessary that the port you will forward is linked to a public IP or hostname. Although one can use his or her public IP, most internet providers (ISPs) do not give static public IPs, meaning that at some point in time your public IP address will change. The consequence of this is that the port that you forwarded ceases to match to the IP address specified in the Storj Share configuration file and thus the port will be inaccessible/closed. You can protect yourself against public IP address change by assigning a hostname to your own local network with a service like noip. If you are not sure if the public IP changes or not over time it is still a good idea to configure a hostname just in case. Below we will walk through using both methods. If you plan to use a hostname please skip the "Use a public IP address below" step.

3.2.2.1. Public IP address

If you are certain your public IP address is static this is the way to go.

Open a browser and head over to google, once I google type in “what’s my IP”, google will now return your public IP address. Keep this IP address at hand as we will need it later.

*Figure 3.2. Finding your public IP address.*

Figure 3.2. Finding your public IP address.

3.2.3. Hostname configuration

Note: You should only use one hostname for your entire local network, even if you plan to configure multiple drives/nodes or run Storj Share on multiple machines. This is true as long as all machines are connected to the same network and can thus be identified with the same public IP address. If you want to run one of the nodes behind a proxy or VPN on the same network, you would have to add another hostname. You also need a new hostname if you want to run Storj Share on another network.

When your public IP address is not static, your ISP will provide you with a new IP address after a certain amount of time. The consequence of this would be that when the IP address changes, Storj Share would lose connection to the network. Adding a hostname solves the issue of public IP change.
We will add a free hostname using NoIP (noip) which needs to be renewed for free every 30 days on a free account. On the NoIP website scroll down to where it says “Create Your Free Hostname now”, then do the following (Figure 3.3):

  1. In the hostname input field select a hostname of your liking (e.g. myhomestorjfarm), it can contain letters and numbers.
  2. Next select a hostname provider of your liking (e.g. “.ddns.net”) in thebox to the right.
  3. Click on “Sign Up
*Figure 3.3. Adding our own hostname.*

Figure 3.3. Adding our own hostname.

  1. On the sign-up page, enter your email, username and password. Make sure to write these details down, you will need them later (Figure 3.4).
*Figure 3.4. NoIP registration page.*

Figure 3.4. NoIP registration page.

  1. When done, click on “Create My Free Account”. NoIP will now send you a confirmation email with an activation link to your email address. Once you click on the activation link it should take you to the NoIP website and confirm that your account is now active.

  2. Now scroll down to where it says “How to remote access your device” and click on “get started with dynamic DNS” (Figure 3.5).

*Figure 3.5. The activation page: click on the large blue box to go to the hostname setup page.*

Figure 3.5. The activation page: click on the large blue box to go to the hostname setup page.

  1. Clicking on the link should take us to our NoIP dashboard.

  2. Now scroll down to “Dynamic Update Client for Windows” (DUC) and click on “Download” (Figure 3.6). This should take you to the download page where you can download the DUC tool. On the download page click on “Download Now”.

*Figure 3.6. Dynamic Update Client download*

Figure 3.6. Dynamic Update Client download

  1. After the file downloaded successfully, head over to the download folder and double click on the “DUCSetup” executable.

  2. On the resulting installation window, click on “Agree” -> “Install” -> “Finish”.

*Figure 3.7. DUC installation.*

Figure 3.7. DUC installation.

  1. The Dynamic Update Client should now open. Enter the sign-in details username and password from step (4) above and click on “Sign in” (Figure 3.8).
*Figure 3.8. Dynamic Update Client (DUC).*

Figure 3.8. Dynamic Update Client (DUC).

  1. Once logged in successfully, the “Edit groups/Hosts” menu should be displayed (Figure 3.9). If not already selected, choose the hostname box and click on “Save”.
*Figure 3.9. From the Edit groups/Hosts menu, select the hostname and click on save.*

Figure 3.9. From the Edit groups/Hosts menu, select the hostname and click on save.

  1. The DUC tool will now come to life (Figure 3.10). Next go to “File” -> “Preferences” and select “Start this application automatically when the user logs on”. In case your computer reboots, DUC will automatically start in the background. This is very handy because if Storj Share starts automatically, it will not run into a closed port as DUC is also already running.
*Figure 3.10. DUC once configured correctly.*

Figure 3.10. DUC once configured correctly.

You now have a hostname and a dynamic update tool that automatically tracks and assigns the IP address to your hostname. So if your public IP changes, Storj Share will not lose access to the TCP port. Please keep your hostname at hand as we will need it later.

Note: Some routers can also act as a DUC, in that case you can use the router directly instead of having to install the No-IP DUC. Search the router menus for "hostname" or "DDNS".

3.2.4. Router TCP port forwarding configuration

Now that we have our public IP address or hostname (rpcaddress), it is necessary to link the rpcaddress to a specific TCP port (rpcport) by forwarding that port in your router. All communication to and from your node will pass through this port.

Note: Each drive/node should have its own TCP port, so in case you want to add multiple drives it would look like this:

First, before we can start our port forwarding journey, we need to know the gateway (router) private IP address so that we can gain access to the router. This can be done in the following ways;

3.2.4.1. Windows

Finding the router IP address can be accomplished by typing in ipconfig into a CMD window (Figure 3.11). Then scroll down to “Default gateway” and copy-paste the router’s private IP address into
a browser window (Figure 3.12).

*Figure 3.11. Network settings.*

Figure 3.11. Network settings.

*Figure 3.12. Router Login page.*

Figure 3.12. Router Login page.

We can now log into our router and configure the TCP ports. The router manufacturer and model will vary from user to user and thus the port-forwarding appearance and menu location within the router GUI will also differ.

Doing a quick Google or YouTube search for:

  • Port forwarding with <Your router brand and model>

Should bring up enough information to successfully configure port forwarding for your router.

Note: The port forwarding menu in most routers is under the “Security” menu called “Virtual server”.

Now copy the IPv4 address from the command prompt (CMD) window in which we executed the ipconfig command. This IPv4 address is the private internal IP address of our computer and is required to set the port forwarding rule (Figure 3.13).

In the example below the ports for three nodes/drives were added to the
router:

  • Node 1: 4000 (rpcport)
  • Node 2: 4001 (rpcport)
  • Node 3: 4002 (rpcport)
*Figure 3.13. Port forwarding parameters for three nodes. Each row represents the forwarding parameters for a single node.*

Figure 3.13. Port forwarding parameters for three nodes. Each row represents the forwarding parameters for a single node.

3.2.4.2 Linux

The first thing we have to do is find our default gateway (router private IP address), this can be accomplished by typing in the following command into a terminal (Figure 3.14):

  • route -n

    or
  • ip route show
*Figure 3.14. Find your router gateway IP address.*

Figure 3.14. Find your router gateway IP address.

Now copy your “Gateway” address as shown in red in the figure above (Figure 3.14) and paste it into a browser window (Figure 3.15), this should bring up the router login page.

*Figure 3.15. Router Login page.*

Figure 3.15. Router Login page.

We can now log into our router and configure the TCP ports. The router manufacturer and model will vary from user to user and thus the port-forwarding menu location within the router GUI will also differ.

Doing a quick google or YouTube search for:

  • Port forwarding with <Your router brand and model>

Should bring up enough information to open the ports.

The port forwarding menu in most routers is under the “Security” menu
called “Virtual server”.

You now have to find the private IP address of the machine on which you want to run Storj Share, this can be done by executing the following command:

  • hostname -I

This should return your current static private IP address.

Now copy the IP address from the terminal window in which you executed the hostname -I command and use it to set the port forwarding rule in the router settings (Figure 3.16).

In the example below the ports for three nodes/drives were added to the router:

  • Node 1: 4000 (rpcport)
  • Node 2: 4001 (rpcport)
  • Node 3: 4002 (rpcport)
*Figure 3.16. Port forwarding parameters for three nodes. Each row represents the forwarding parameters for a single node.*

Figure 3.16. Port forwarding parameters for three nodes. Each row represents the forwarding parameters for a single node.

Note: The exact port numbers you want to forward are of your own choice, the ports in this guide are just suggestions.

After following all these steps you should have the following:

  1. A configured hostname or know your static Public IP address.
  2. A static private IP address.
  3. Configured port forwarding for every node/drive you want to add to Storj Share.

These are all the network tools and settings that have to be configured
to successfully run Storj Share drives/nodes via TCP.

3.3. Node configuration commands

The following steps involve the configuration of each of the CLI nodes.
The table below illustrates the configuration commands available for the
setup of the Storjshare-daemon nodes. Info taken from here.

Options Options info
--key <private key> specify the private key
-h, --help Output usage information
--storj <payout address> specify the STORJ payout address (required), e.g. ETH ERC20 address. Please do not use a exchange address
--storage <path to directory> specify the storage path
--size <maxsize> specify node storage size (e.g.: 10GB, 1TB)
--rpcport <port> specify the rpc port number (TCP or UPnP port that is open for only one specific node)
--rpcaddress <address> specify the rpc address (Hostname or Public IP address)
--maxtunnels <tunnels> Specify the number of tunnels to open through which other nodes can connect to the network.
--tunnelportmin <port> specify the minimum tunnel gateway port number
--tunnelportmax <port> specify maximum tunnel gateway port number
--manualforwarding Use TCP port forwarding, do not use NAT (UPnP) traversal strategies.
--logdir <path> specify the log directory
--noedit do not open generated config in editor
-o, --outfile <writepath> write config file to path
--verbosity <verbosity> specify the logger verbosity (0-4)

Table 1. Storjshare configuration commands. (Usage: storjshare-create [options])

3.4. Node Config-file generation

We can now generate a configuration file for each of the node instances. We want to save the config file in each of the node directories we created a few steps back. The configuration commands can be found in Table 1. The examples below are pasted one by one into a command prompt window under windows or a terminal as user (not root) under GNU+Linux. If all the parameters are correct, storjshare-daemon will create a configuration file in each of the node directories, it will then return
a success message and open the config file in the text editor (figure 3.17). If an error is returned instead of a success message, then either an invalid parameter was entered or the syntax is incorrect.

  • Node 1: storjshare-create --storj 0x8D7a2e3C16d029F838d1F6327449fd46B5daf881 --storage G:\Node_1 --size 3GB --outfile G:\Node_1\config.json --rpcaddress myhomestorjfarm.ddns.net --rpcport 4000 --manualforwarding true --logdir G:\Node_1
  • Node 2: storjshare-create --storj 0x8D7a2e3C16d029F838d1F6327449fd46B5daf881 --storage G:\Node_2 --size 3GB --outfile G:\Node_2\config.json --rpcaddress myhomestorjfarm.ddns.net --rpcport 4001 --manualforwarding true --logdir G:\Node_2
  • Node 3: storjshare-create --storj 0x8D7a2e3C16d029F838d1F6327449fd46B5daf881 --storage G:\Node_3 --size 3GB --outfile G:\Node_3\config.json -- rpcaddress myhomestorjfarm.ddns.net --rpcport 4002 --manualforwarding true --logdir G:\Node_3

VERY IMPORTANT: The parameters used above to create the config files are different from the ones YOU will use, substitute the information above with your own parameters (e.g. paths, payout address, public IP address, storage location storage size, ports…).

Note: Please backup the configuration file for each node. You can find the path to the configuration files in Table 4. You could also copy the entire directory called "configs" instead, since all the configuration files are contained within that directory. Backup the files to a safe location so in case your system crashes, you still will be able to recover the shards stored on each Storj drive.

*Figure 3.17. In the windows terminal above we successfully created a
configuration file for each of the nodes.*

Figure 3.17. In the windows terminal above we successfully created a
configuration file for each of the nodes.

To confirm that the “config.json” file was successfully created we can open the node folders and open the config file with a text editor, e.g. vi/nano or Notepad++ (right mouse click -> open with notepad ++), figure 3.18. We can now scroll through the config.json file and check if all the entered parameters are correct. If some of the parameters are incorrect they can be changed directly inside of the config file. You can edit the parameters in the config file without the necessity of running setup again.

*Figure 3.18. Config.json file opened in Notepad++.*

Figure 3.18. Config.json file opened in Notepad++.

3.4.1. Maximum Shard size

warning: Only change this parameter is you have under-performing hardware (e.g. Raspberry Pi).

One of the main reasons why Storj Share nodes in general crash (restart) is due to the fact that the transfer of large data chunks (shards) requires more CPU and RAM resources. One can define the maximum shard size in the config file of their node(s) so that only shards are downloaded that the hardware can handle. Finding the best shard size for each system requires some trial and error but Starting at 50-100MB and lowering it a few MB at a time is the best method to find the most stable shard size value.

The "maxShardSize": "100MB", parameter can be enabled in the config file by removing the double front slash as can be seen in the snippet below.

Tip: You can also set this parameter to a low value to prevent excessive network usage.

Note: Changing the maximum shard size will yield less shards on average because you are ignoring larger shard sizes.

 "storageAllocation": "2GB",
  // Max size of shards that will be accepted and stored
  // Use this and make this lower if you don't have a strong internet connection
  "maxShardSize": "100MB",
  // Periodically report your used and free capacity to Storj Labs to improve
  // the network - no personally identifiable information is sent
  "enableTelemetryReporting": true

Once you checked that all entered parameters are correct you can now proceed to the next step of actually running the nodes.

3.5. Running the nodes with the new configurations

Once storjshare-daemon-CLI is installed and correctly configured you can now run all three nodes. Below in Tables 2 and 3 the execution commands with coupled examples for storjshare-daemon-cli are presented.

Commands Command info
start start a farming node
stop stop a farming node
restart restart a farming node
status check status of node(s)
logs tail the logs for a node
create create a new configuration
destroy kills the farming node
killall kills all shares and stops the daemon
Daemon starts the daemon
Help [CMD] display help for [cmd]

Table 2. Storjshare-daemon commands

Execution command examples Execution command info
storjshare daemon Starts the daemon
storjshare start --config path/to/config.json Starts a specific farming node
storjshare start --config <path> --unsafe Run the node in unsafe mode, skip resources check.
storjshare stop --nodeid <nodeID> Stops a specific farming node
storjshare restart --nodeid <nodeID> Restarts a specific farming node
storjshare status Checks the status of the node(s)
storjshare logs --nodeid <nodeID> Tails the logs for a specific node
storjshare create --storj <address> … Create a new node configuration
storjshare destroy --nodeid <nodeID> Kills a specific farming node
storjshare killall Kills all running nodes and stops daemon
storjshare help logs Display help for the logs command
storjshare save Saves a snapshot of the running nodes
storjshare load Loads the saved snapshot

Table 3. Storjshare-daemon command execution examples.

First before we can start the nodes we have to start the daemon, this can be done by executing the following command in a terminal as user (not root) for GNU + Linux based systems or within a command prompt in windows systems:

  • storjshare daemon

    This should return the following message:

    starting daemon in background

We can now start each of the nodes by executing the following commands:

  • storjshare start --config <path to config.json of Node_1>

  • storjshare start --config <path to config.json of Node_2>

  • storjshare start --config <path to config.json of Node_3>

For example under windows:

  • storjshare start --config G:\Node_1\config.json

  • storjshare start --config G:\Node_2\config.json

  • storjshare start --config G:\Node_3\config.json

Note: Storjshare by default does not allow you to run more more nodes than the number of logical CPU cores, leading to a “insufficient resources error”. You can bypass this restriction by adding the --unsafe tag in the following way:

  • storjshare start --config G:\Node_1\config.json --unsafe

Warning: The limit is there for a reason!, only use this option if you (1) Have enough CPU power, (2) Enough RAM per node (1GB min) and (3) have a good enough internet connection. For example don’t ever think of using this command because you want to run 6x nodes on a Raspberry Pi,
or 6x nodes on a PC with 1GB of RAM. Use common sense!

Don't judge the computer resources that Storj is using during idle times!. At idle Storj can use
50MB but can spike to +2GB during intensive network usage.

Note: Make sure to include the config file name in the path. The path style will vary between different platforms, for GNU + Linux systems just make sure you point it at a mounted drive.

You can check if all nodes are running by executing the following command that will be discussed in the next chapter:

  • Storjshare status

If you are running windows, three NodeJS windows should open after starting each of the three nodes, which indicates that all three nodes are running. Double-check if three NodeJS windows are running (figure 3.19), if only 1-2 windows are open, execute the start command again for the node that did not start until all three NodeJS windows are present. Sometimes the first startup fails. Note, if you restart your computer you also have to restart storjshare.

*Figure 3.19. Three NodeJS windows are open, indicating that all three
nodes are running.*

Figure 3.19. Three NodeJS windows are open, indicating that all three
nodes are running.

3.5.1. Save a snapshot of the running instances

Storjshare has a build in functionality to save a snapshot of the running instances, meaning that storjshare will remember which nodes were running when the snapshot was created. First you have to assure that all three nodes are running, only then can a snapshot be saved by executing the
following command in a terminal:

  • storjshare save

If all goes well the following message should be returned for windows:

snapshot C:\Users\USER.config\storjshare\snapshot saved

Or for GNU + Linux:

snapshot ~/.config/storjshare/snapshot saved

The execution of this command writes a snapshot file to C:\Users\<user>\.config\storjshare for windows and ~/.config/storjshare/snapshot for Linux. The snapshot contains the config file path and nodeID for each of the nodes that were running when the snapshot was created.

Note: If one want to migrate or rename the config file to a different location the snapshot file can be edited with a text editor to accommodate the new parameters.

3.5.2. Load a snapshot of the running instances

The next time you need to start storjshare you can now load the snapshot by executing the following commands in order within a terminal:

  1. storjshare daemon

  2. storjshare load

Note: It is also possible to create a script with the daemon and load commands above. Executing the script would then first start the daemon and then start all nodes with the load command.

3.6. Confirm Port forwarding and monitor the node instances

Now that all three nodes are running you have to check if the nodes can successfully talk to the outside world. To do this we click on this link.
Now enter the “rpcport” ports (e.g. 4000, 4004 and 4008) into the “Port Number field” one at a time and click on "Check". Yougetsignal should automatically detect your public IP address, if
not, enter it manually. If you are using a hostname use that in the “Remote Address” field. We now should see the following for each of the ports (Figure 3.20):

*Figure 3.20. All nodes can successfully communicate to the outside
world.*

Figure 3.20. All nodes can successfully communicate to the outside
world.

We now know that all three nodes can successfully talk to the outside world. To be able to check the health of the running nodes open a terminal/CMD window and type in the following command:

  • storjshare status

This should now give us a good looking table with critical information about all the running nodes (figure 3.21).

*Figure 3.21. Check the status of the running node.*

Figure 3.21. Check the status of the running node.

We can see that the node is running and have a significant number of peers and the node did not restart, which is very good. Through time the “% shared” fields should increase towards 100%.

With the “Node ID’s” given in the table above we can also monitor the logs for each node. We can, for instance monitor the first node by typing in the following into a terminal/CMD window (figure 3.22):

  • storjshare logs --nodeid <nodeID>

e.g. “storjshare logs --nodeid 892f4424811f20d77160ffcce0785cc61ae1284f”

To select the node ID do a “right mouse click -> mark the text”. After marking do another right mouse click to copy the node ID.

Note: if you want to run all the logs at the same time, three CMD windows should be opened with each one running the tails for each node.

*Figure 3.22. Log monitoring of one of the nodes.*

Figure 3.22. Log monitoring of one of the nodes.

3.7 Updating storjshare-daemon-CLI

New updated versions of the storjshare-daemon-CLI package are released
very regularly. To check if any updates are available since storjshare was installed we can check the github releases section.

If new releases appear in the list we storjshare can be easily updates by executing the following commands in order:

  1. storjshare killall

  2. npm install -g storjshare-daemon

Note: Don’t forget to start storjshare again after the update is
complete.

4. Storj Share Troubleshooting

There are two ways we can check if the node is working correctly, we can upload our log to a website that checks the log or we can do a manual check if we want the highest amount of details.

Automatic log checker


We can check our log automatically by doing the following:

  1. Stop all nodes and exit storj Share.
  2. Open the log folder containing the Storj logs and delete all logs.
  3. Start Storj Share and wait for all nodes to start.
  4. Upload each log file one by one here.
  5. Once you click on “Process” it will check the logs for connectivity and other parameters.

Note: Windows firewall is notorious for blocking TCP ports and can thus also cause the ports to stay closed, please have a look at the windows firewall section described further down.

Automatic log checking by scripts


Please, read this: Logs checking tools

Manual log checking


With very little effort one can check if the configured drives are working correctly by combining the following sources of information:

  1. Storj Share logs.
  2. Storj API.
  3. Port checker.

If you are not sure if your node is working correctly or not, follow the
next few steps:

  1. Click on the gear next to a specific drive and click on "Logs".
  2. Search/grep for "Node created"

    • Windows -> CTRL + F -> "Node created"
    • GNU+Linux -> grep "node created" /path_to_your_log.log | tail

    This should return something like this:

    {"level":"info","message":"node created with nodeID
    54e7a82b6e3b99e33a06d32014b2f261a3271365","timestamp":"2017-04-14T16:36:37.521Z"}

  3. Now enter the nodeID (also the config/log file name) in the Storj API in a web-browser to check the status of your node as follows:

    e.g. api.storj.io/contacts/e24deba65d74263cd73a9a480d5c44d87139dd00

    This should return something like this:

{"lastSeen":"2017-04-15T17:15:11.930Z”,"port":4000,"address":"5.67.55.122","userAgent":"6.1.4","protocol":"1.1.0","responseTime":7953.184871461389,"lastTimeout":"2017-04-15T17:14:42.617Z”,"timeoutRate":0.000001423611111111111,"nodeID":"26dfd89c6c96df11cbcd4598e20117a7b72a97da"}
  1. Now copy the "address" and "port" and enter them into yougetsignal and click on "Check", It should return the following:

    • Port <port> is open on <IP address or hostname>.

    Note: Windows firewall is notorious for blocking TCP ports and can thus also cause the ports to stay closed, please have a look at the windows firewall section described further down.

    If the port is closed then most likely the drive or router configuration is incorrect. If you configured port forwarding, check the config file
    and router forwarding rule again.

    Note: Sometimes your internet provider (ISP) does not allow port forwarding, giving them a call to ask if TCP port forwarding is possible and asking them how to set it up if possible is a good idea.

    If you just followed the basic configuration, please try enabling "UPnP" in your router settings, reload Storj Share and try the port scan again.

  2. Next search/grep the same log for "delta", which is your clock synchronization.

    • Windows -> CTRL + F -> "delta".
    • GNU+Linux -> grep "delta" /path_to_your_log.log | tail

    This should return something like this:

    {"level":"info","message":"clock is synchronized with ntp, delta: 82 ms","timestamp":"2017-04-14T16:36:37.774Z"}

    We can see that in the example above, the delta value is 82ms which falls within the "-500ms < delta < 500ms" acceptable range. This
    indicates that our node is correctly synchronized. If the delta value falls outside of the range above, please synchronized your clock again
    (see chapters above for more details).

  3. Eventually if your node is working correctly you should start to see “OFFER” messages in your log.

  4. Sometimes a fatal error can occur such as database corruption, searching the logs for the following parameters allows you to check if any of these issues have occurred:

  • Windows -> CTRL + F -> "UsedSpace".
  • GNU+Linux -> grep "UsedSpace" /path/_to/_your/_log.log | tail

  • Windows -> CTRL + F -> "kfs".
  • GNU+Linux -> grep "kfs" /path/_to/_your/_log.log | tail

  • Windows -> CTRL + F -> "exception"
  • GNU+Linux -> grep "exception" /path/_to/_your/_log.log | tail

If you encounter any of these issues please join our community at
Storj community
where we will help to resolve this issue.

4.1. Windows Firewall

If you are having a hard-time to get storj Share to connect through mainly the forwarded TCP ports on a windows machine chances are that Storj Share is being blocked by the firewall.

To solve this do the following:

  1. Enter the "Control Panel".
  2. Search for "Windows Firewall with Advanced Security".
  3. Once in the Windows firewall with Advanced Security click on "Inbound Rules".
  4. The in the "Actions" list on the right-hand side click on "New Rule…".
  5. Once in the "New Inbound Rule Wizard" select the "Port" option and click on "Next".
  6. Next Select "TCP" and "All Local Ports" or just the Storj Share port range (e.g. 4000-4005) in "Specific Local ports" and click "Next".
  7. Select "Allow the connection" and hit "Next" again.
  8. Select all three rules (Domain, Private and Public) and click "Next".
  9. Now enter a description (e.g. "Storj Ports") and click finish.

Now the firewall inbound rules has been set, we now have to do the same thing for the outbound rule. Click on "Outbound Rule" and repeat the steps one through nine above.

4.2. Antivirus Recommendations

As Storj Share is a P2P program it communicates and transfers data to and from other nodes and renter on the network. Most common antivirus programs will either flag or completely block all or certain types of connections which in most cases prevents Storj Share from operating correctly. Normally the first observation is that is blocks Storj URL’s (other farmers). One can add an exception to the Storj Share program in the "Web Shied" settings and turn off "Block Malware URLS". The specific setting will be different for every antivirus.

4.3. Logger verbosity

You can instruct each node how detailed you want the logs to be, this can be done by clicking on the gear next to the specific node, selecting "Edit" and then in the config file change the "loggerVerbosity" value. The default value is three which shows "All information", you can set it to for example four which is the "Debug mode". If you just want to view warnings you can set it to two. Setting it to one only shows errors. If you don’t want Storj Share to dump any information into the logs set the logger verbosity to zero.

Logger verbosity Log information
0 No logs
1 Only show Errors
2 Only show warnings
3 Show all information (Info + warnings)
4 Debug mode

Table 6. Logger verbosity levels.

5. Some additional useful Storj Share monitoring tools

There are a number of tools which people have been using through the Storj ages to monitor Storj Share data usage. As Storj Share not only downloads data from renters but also uploads data back to them, there can be intensive network usage. If one wants to quantify the amount of bandwidth Storj uses, this can be easily achieved by installing a few third-party tools.

Node monitoring via a website

Windows

Using task manager to track CPU and RAM usage of Storj Share is also very handy.

GNU + Linux

6. Migrating from Storj Share GUI to Storjshare-daemon-CLI or vise-versa

6.1. Migrating from the GUI 5.0.0 to the CLI

The only thing you have to do is have to do is follow chapter two and install storjshare. When installed follow chapter 3.5 to start the nodes. As the config files are the same we can directly call the Storj Share GUI config file in the following way:

  • storjshare daemon

This should return the following message:

starting daemon in background

We can now start each of the GUI nodes by executing the following commands:

  • storjshare start --config <path to the GUI nodeID.json file>

Note: The table below illustrates the default paths to where the GUI
config files are saved.

Storj Share GUI v5+ Config file location
Windows %userprofile%\.config\storjshare\configs\nodeID.json
OSX ~/.config/storjshare/configs/nodeID.json
Linux ~/.config/storjshare/configs/nodeID.json

Table 5. Default configuration file location for Storj Share GUI v5+.

6.2. Migrating from the CLI to the GUI

You can migrate from the CLI to Storj Share GUI v5+ by in the following
way:

  1. Download the latest GUI release from here and install it.

  2. Once at the welcome screen, click on “I’m an experienced user, skip this step”, Figure 6.1.

*Figure 6.1. Now click on “Import Config” in the top-right corner.*

Figure 6.1. Now click on “Import Config” in the top-right corner.

  1. You can now import the configuration file of a single node from the previously running CLI.

    Note: If you did not specify a config file output name and location (-o) at the CLI setup (create command), then the config files are saved to the default directory which are illustrated below in table six:

Storj Share Daemon-CLI Config file location
Windows %userprofile%\.config\storjshare\configs\nodeID.json
OSX ~/.config/storjshare/configs/nodeID.json
Linux ~/.config/storjshare/configs/nodeID.json

Table 6. Default configuration file location for Storj Share Daemon-CLI.

  1. Once you found the correct configuration file location click on the config file and select “Open”, Figure 6.2.
*Figure 6.2. Click on “Import config”.*

Figure 6.2. Click on “Import config”.

That’s it! Storj Share will now automatically import the config file and start the drive(s).

Note: The GUI will now import the config file and create a new config file in the GUI default config file directory, see table seven below.

Storj Share GUI v5+ Config file location
Windows %userprofile%\.config\storjshare\configs\nodeID.json
OSX ~/.config/storjshare/configs/nodeID.json
Linux ~/.config/storjshare/configs/nodeID.json

Table 7. Default configuration file location for Storj Share GUI v5+.

6. Payments

Farmers are paid for the following resources:

  1. Storage ("gbHours" in the payout formula).
  2. Upload bandwidth ("downloadedBytes" in the payout formula).
  3. Sending telemetry reports ("telemReports" in the payout formula).
  4. Base payout

Farmers are paid for storing data over time. Just like is the case with electricity where you pay for kW/h, with Storj the unit is GB-hours. Every hour that you store one gigabyte is considered 1 GB/h, meaning that if you store 1GB during one month the calculation would go as follows:

GB/h = 1 GB x 24 hours/day x average number of days per month = 730GB/h for storing 1GB during the entire month.

The calculation above is actually the minimum requirement to be eligible for a payment.

Although you are not paid for downloading a shard, you are paid for uploading a shard back to the renter, thus you are paid for providing upload bandwidth.

You are also paid for telemetry reports which tell the Bridge how much free and used space you currently have for each node. If you accumulated the minimum GB/h requirement for the month, you should receive a base payout. This is an incentive for small farmers. However, if your node
goes offline for more than a week before the end of the current month, you will not qualify for payouts.

If you are tech savvy and interested in calculating how many GB/h you have accumulated, you can run this script made by Jens Heimbürge:

Storj_farmer_contracts

Note: Payouts are currently done once a month at the beginning of every month, a payout sheet is published in the #announcements channel of the Storj Share community chat for farmer review and discussion at the beginning of each month and payouts proceed when all reported problems have been attended.

9. Payout Formula

Below is the payout formula used to calculate Storj farmer payouts. The payout formula can be found here. This payout formula is subjected to changes through time.

paymentModelFunction = function(gbHours, telemReports, downloadedBytes) {
  ## 730 is the average number of hours in a month.
  HOURS_IN_MONTH = (24 * 365) / 12
  
  ## This is determined by value reported on https://coinmarketcap.com/ at time 
  ## of first preliminary payout calculation. 
  STORJ_USD_RATE = 0.756371 
  
  ## Both gbHoursScaled and downloadedBytesScaled can not be less than 0 to 
  ## ensure everyone gets at least the base payout amount. 
  gbHoursScaled =  sapply((gbHours - median(gbHours)) / sd(gbHours), 
                          function(x) ifelse(x < 0, 0, x))
  downloadedBytesScaled = sapply((downloadedBytes - median(downloadedBytes)) / sd(downloadedBytes),
                                 function(x) ifelse(x < 0, 0, x)) 
  
  ## At least one of the above criteria must be met to qualify for a payment.
  telemReportsFlag =  as.numeric(telemReports > 0)
  downloadedBytesFlag = as.numeric(downloadedBytes > 0)
  gbHoursFlag = as.numeric(gbHours >= 730)
  isQualifiedFlag = sapply(gbHoursFlag + telemReportsFlag + downloadedBytesFlag,
                           function(x) ifelse(x > 0, 1, 0)) 
                           
  ## The current base payout is set to $1.50 USD.
  basePayout = (1.50 / STORJ_USD_RATE) * isQualifiedFlag
  
  ghHourPayout = 12.2221 * gbHoursScaled * isQualifiedFlag
  downloadedBytesPayout = 12.6849 * downloadedBytesScaled * isQualifiedFlag
  
  payoutAmountSTORJ = ghHourPayout + downloadedBytesPayout + basePayout
  
  payoutAmountUsd = payoutAmountSTORJ * STORJ_USD_RATE
  
  cbind(payoutAmountSTORJ, payoutAmountUsd)
}

9.1. Details

  • Nodes that have not been seen in the past week (from the time first preliminary payouts are calculated) are not included in the metric totals.

  • Having telemetry reports helps qualify for the base payout but has no impact beyond that.

  • Each component (gbHours and downloaded bytes) is scaled so that each metric ends up being a measurement of how far away you are from the median.

  • Those values are multiplied by certain weights and summed to arrive at a final value

  • There are no rewards for storing less than 1 GB for the month.

10. Conclusion

In this guide we installed and configured Storjshare-daemon-CLI and all the necessary dependencies including network configuration. We looked at how to troubleshoot each of the nodes and how to fine-tune them for optimal performance. Finally the migration of nodes between the GUI and
the CLI was discussed.

If any of the chapters or steps is confusing or any error appears, feel free to join our awesome community on Rocketchat which is full of very friendly and helpful people that will be glad to help you on your way.

Happy farming!


What's Next

Storj Share and Privacy

Storj Share + PIA

Storj Share Daemon (CLI)

Storjshare-Daemon-CLI setup
Step by Step Guide