Rclone – synchronization with cloud storage services

R

Rclone is an open source tool that can sync Linux systems with the most important cloud storage services: Google Drive, Amazon Drive, S3, Dropbox, Backblaze B2, One Drive, Box, Hubic, Cloudfiles, Google Cloud Storage, Yandex and more others.

Its appearance was determined by the need of synchronizing a local directory with the space in the cloud owned by users, a space that most often has mounting problems. The best known command to synchronize two directories locally is rsync. The difficulty appears, as I said, when installing cloud space on the system. Rclone has solved this problem.

Installing rclone

Installing rclone on Linux is extremely simple. Running the command below:

curl https://rclone.org/install.sh | sudo bash

To install beta

curl https://rclone.org/install.sh | sudo bash -s beta

Configuring rclone

To start any configuration with rclone, the following command will be entered:

rclone config

Below, I’ll show you only the commands for Google Drive. I have marked the mandatory answers to be entered and I also commented with italic where I thought more details were needed.

$ rclone config
No remotes found – make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> GoogleDrive # or any name you choose

Type of storage to configure.
Choose a number from below, or type in your own value, such as Google Drive.
is on the 11th position, so we will type this number
11 / Google Drive
\ “drive”
Storage> 11
Google Application Client Id – leave blank normally.
client_id>
Google Application Client Secret – leave blank normal.
client_secret>

Scope that rclone should use when requesting access from drive.
Choose a number from below, or type in your own value
1 / Full access to all files, except Application Data Folder.
\ “drive”
2 / Read-only access to file metadata and file contents.
\ “drive.readonly”
/ Access to files created by rclone only.
3 | These are visible on the drive website.
| File authorization is revoked when the user deauthorizes the app.
\ “drive.file”
/ Allows read and write access to the Application Data folder.
4 | This is not visible in the drive website.
\ “drive.appfolder”
/ Allows read-only access to file metadata but
5 | does not allow any access to read or download file content.
\ “drive.metadata.readonly”

scope> 1
ID of the root folder – leave blank normal. Fill in to access “Computers” folders. (see docs).
root_folder_id>
Service Account Credentials JSON file path – leave blank normally.
Needed only if you want to use SA instead of interactive login.
service_account_file>

Remote config
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine or Y did not work
y) Yes
n) No
y/n> n # if the browser does not ask you to accept the application, copy the code below and paste it into the browser; it’s mandatory that when configuring rclone on a server without GUI
https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=xxxx.apps.googleusercontent.com&redirect_uri =urn%2ert%3Adf%3Aoauth%3A2.0%2lld&response_type=code&scope= https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive&state= state

Log in and authorize rclone for access
Enter verification code> 5/verification-code-example
Configure this as a team drive?
y) Yes
n) No
y/n> n

[GoogleDrive]
type = drive
client_id =
client_secret =
scope = drive
root_folder_id =
service_account_file =
token = {“access_token”:”token”,”token_type”:”Bearer”,”refresh_token”:”1/refresh_token”,”expiry”: “date”}

y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:

Name Type
==== ====
GoogleDrive drive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password # mandatory if configuring on a cloud server
q) Quit config
e/n/d/r/c/s/q> q

Note that if rclone is configured on a server without a graphical interface, the code provided at that step must be copied and paste in any browser that you access from your local computer, and then let rclone access your Google Drive space.

To show all configured cloud remote services:

rclone listremotes

GoogleDrive:

To display folders in your Google Drive account:

rclone lsd GoogleDrive

To show all files:

rclone ls GoogleDrive

To copy a local directory to Google Drive:

rclone copy /path/directory/local GoogleDrive:backup/directory_name

To sync a directory use ‘

rclone sync /path/directory/local GoogleDrive:backup/directory_name

Copy and sync can also be done from the remote account on your local system.

You can make a small script that runs copying or synchronizing the backup folder at the time you want.

If at the final step before leaving, choose s to set a password for the rclone configuration file (indicated if you set rclone on a cloud server or on a system where not only you have access to), you will need to assign a variable RCLONE_CONFIG_PASS environment containing that password (this environment variable will only be available for the session where the script is running). An example is below:

#!/bin/bash
#
RCLONE_CONFIG_PASS=your_rclone_password
export RCLONE_CONFIG_PASS
# instead of /home/backup input your source directory
# Instead of GoogleDrive:backup input the name of your destination directory
# to exclude all hidden files (which begin with “.”) we will use the –exclude directive
rclone sync / home / backup GoogleDrive: backup / server –exclude ‘. *’ –ask-password=false

About the author

Ilias spiros
By Ilias spiros

Recent Posts

Archives

Categories