Connect Using Rclone

Use Rclone to connect with Lyve Cloud Object Storage and manage your files from the command line or mount the cloud storage as a drive.

Prerequisites

You will need the access key and secret key for each account you’ll be using to connect with Rclone.

Connect to Lyve Cloud from Linux

Install Rclone

1. Download Rclone for Linux, then extract the rclone binary to your desired location.
2. To use Rclone, open a terminal window and navigate to the directory where you saved the executable.

Configure Rclone to connect to Lyve Cloud Object Storage

To configure a remote connection with Rclone:

1. Run rclone config to setup and select n for a new remote.

No remotes found - make a new one       
n) New remote        
s) Set configuration password        
q) Quit config       
n/s/q> n

2. Enter a name for the configuration.

name> <Name>

3. Select s3 storage.

Type the storage to configure.
Choose a number from below, or type in your own value        
1 /Fichier 
 \(fichier)  
2 /Akamai NetStorage
  \(netstorage) 
3 / Alias for an existing remote 
 \ (alias)        
4 / Amazon Drive 
 \ (amazon cloud drive)        
5 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS,  Lyve Cloud, Minio,  RackCorp, SeaweedFS, and Tencent COS  
 \ (s3)        
6 / Backblaze B2   
 \ (b2)  [snip]        
46 / seafilehttp Connection 
  \ (seafile)
 Storage> 5

4. Choose Lyve Cloud as the storage provider, or choose Other.

Choose the S3 provider.
Choose a number from below, or type in your own value
Press Enter for the default ("")
  1 / Amazon Web Services (AWS) S3
   \ (AWS)
 2 / Alibaba Cloud Object Storage System (OSS) formerly Aliyun
   \ (Alibaba)
 3 / Ceph Object Storage
   \ (Ceph)
 4 / Digital Ocean Spaces
   \ (DigitalOcean)
 5 / Dreamhost DreamObjects
   \ (Dreamhost)
 6 / IBM COS S3
   \ (IBMCOS)
 7 / Seagate Lyve Cloud
   \ (LyveCloud)
 8 / Minio Object Storage
   \ (Minio)
 9 / Netease Object Storage (NOS)
   \ (Netease)
10 / RackCorp Object Storage
   \ (RackCorp)
11 / Scaleway Object Storage
   \ (Scaleway)
12 / SeaweedFS S3
   \ (SeaweedFS)
13 / StackPath Object Storage
   \ (StackPath)
14 / Storj (S3 Compatible Gateway)
   \ (Storj)
15 / Tencent Cloud Object Storage (COS)
   \ (TencentCOS)
16 / Wasabi Object Storage
   \ (Wasabi)
17 / Any other S3 compatible provider
   \ (Other)          
Provider>7

5. Enter false to enter your credentials.

Get AWS credentials from the runtime (environment variables or EC2/ECS meta data if no env vars). 
Only applies if access_key_id and secret_access_key is blank.Enter a boolean value (true or false). 
Please Enter for the default ("false").Choose a number from below, or type in your own value     
   1 / Enter AWS credentials in the next step     
    \ "false"  
   2 / Get AWS credentials from the environment (env vars or IAM) 
     \ "true"
   env_auth>false

6. Enter your access key and secret key.

AWS Access Key ID.
Leave blank for anonymous access or runtime credentials.
Enter a string value. Press Enter for the default ("") 
    access_key_id> <access key>
AWS Secret Access Key (password)
Leave blank for anonymous access or runtime credentials.
Enter a string value. Press Enter for the default ("")
     secret_access_key> <secret key>

7. Leave the region blank.

Region to connect to.
Leave blank if you are using an S3 clone and you don't have a region.
Enter a string value. Press Enter for the default ("")
Choose a number from below, or type in your own value
1 / Use this if unsure. 
  | Will use v4 signatures and an empty region.
  \ ()
2 / Use this only if v4 signatures don't work. 
  | E.g. pre Jewel/v10 CEPH.
  \ "other-v2-signature"
region> <>

8. Specify the endpoint for Lyve Cloud Object Storage.
   • Lyve Cloud Object Storage currently supports region-specific S3 endpoints. To access buckets created in different regions in the S3 client, add an endpoint connection for each of the regions.
   • Specify the endpoint URL for your S3 service in the specific regions.

Endpoint for S3 API.
Required when using an S3 clone.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
 1 / Seagate Lyve Cloud US East 1 (Virginia)
   \ (s3.us-east-1.lyvecloud.seagate.com)
 2 / Seagate Lyve Cloud US West 1 (California)
   \ (s3.us-west-1.lyvecloud.seagate.com)
 3 / Seagate Lyve Cloud AP Southeast 1 (Singapore)
   \ (s3.ap-southeast-1.lyvecloud.seagate.com)
endpoint> 1

9. Press Enter to skip the location constraint as there is no location constraint

Location constraint - must be set to match the Region.
Leave blank if not sure. Used when creating buckets only.
Enter a string value. 
Press Enter for the default ("")location constraint>

10. Choose default ACL (private).

Canned ACL used when creating and or storing or copying objects.
This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
Note that this ACL is applied when server-side copying objects as S3
It doesn't copy the ACL from the source but rather writes a fresh one.
Enter a string value. Press Enter for the default ("")
Choose a number from below, or type in your own value   
1 / Owner gets FULL_CONTROL. 
  | No one else has access rights (default). 
   \(private)   
2 / Owner gets FULL_CONTROL. 
  | The ALLUsers group gets READ access. 
  \(public-read)    
3 /Owner gets FULL_CONTROL. 
  | The ALLUsers group gets READ and WRITE access.   
[snip]
acl>1

11. Select n to save the default advanced configuration.

Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n>n

12. Review the displayed configuration and accept to save the remote and then quit. The config file should look like this:

NAME]
    type = s3
    Provider = LyveCloud
    env_auth = false
    access_key_id = xxx
    secret_access_key = yyy
    region = us-west-1
    endpoint = s3.us-east-1.lyvecloud.seagate.com
    acl = private

13. Click y to confirm the configuration.

y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d>y

14. Type q to quit the configuration, else select any of the following to edit, delete, rename, copy, Set configuration password.

Current remotes:

Name                 Type
====                 ====
ashrcl               s3

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q>q

Mounting Lyve Cloud Object Storage as a drive

Prior to mounting Lyve Cloud Object Storage as a drive, connect and test the connection by viewing the contents of one or more buckets using the  rclone ls command.

To mount Lyve Cloud Object Storage as a drive, use this command where:

• remote is the name of the remote server
• path/to/files is the exact path to the bucket(s)
• path/to/local/mount is the local directory:

rclone mount remote:path/to/files /path/to/local/mount

For more information, see Rclone’s mount command documentation.

Manage data

Here are several of the more commonly-needed commands for viewing and managing your data from the command line. See the Rclone docs, including information on global flags, for additional information.

View information about your buckets and directories

There are three list commands with easily readable output available: ls, lsd, and lsl.

1. To list all data in a certain bucket, where remote is the name of the remote and path is the name of the bucket:

rclone ls remote:path [flags] 

2. To list the directories in a certain remote and see the total directory size, modification time, and number of objects in the directories:

rclone lsd remote:path [flags] 

Or

rclone lsd remote: [flags]

3. To list all objects in a certain remote and see modification time, size and path where path is the remote path beginning with the bucket name. Any of the filtering options can be applied to this command.

rclone lsl remote:path [flags] 

4. Learn more about ls, lsd, and lsl.

Video: Lyve Cloud - How to use Rclone list commands

Seagate on Vimeo: Lyve Cloud - How to use Rclone list commands

Upload data to a bucket

1. To upload data into a bucket, use either of these commands:

rclone copy C:/path/to/filename remote:path [flags] 
rclone copy filename remote:path 

2. You can also use rclone copy to copy a file or directory to a new location and rename the directory at the same time. Neither of these commands deletes the file from the source, and neither of these commands will copy unchanged files.
3. Learn more about copy and copyto.

Video: Lyve Cloud - How to use Rclone Copy-to and Copy-sync Commands

Seagate on Vimeo: Lyve Cloud - How to use Rclone Copy-to and Copy-sync Commands.

Download data to local storage

This is the same as copying, but the source path is something in the remote or in a bucket, while the destination path is on your local storage.

rclone copy remote:path C:/path/to/filename [flags] 

Delete data from a bucket

To delete files in a certain path from a certain bucket:

rclone delete remote:path [flags]

You can use flags to delete only files with certain characteristics:

For example, to delete files that are over 100MB:

rclone --min-size 100MB delete remote:path 

To delete only a specific file:

rclone deletefile remote:path [flags]

Learn more about rclone delete and deletefile.

Create a new folder

To create a new folder, for example, a file named 'blue' in the current location:

rclone mkdir blue

Create folders in other paths, or with other permissions, by setting the proper flags. Learn more about mkdir.

Delete a folder

To delete an empty folder, for example, a file named 'blue' in the current location:

rclone rmdir blue 

Add flags to delete folders in locations other than the current directory, for example, an empty folder named 'blue' that contains other empty folders:

rclone rmdirs blue 

The folders must be empty for rmdiror rmdirsto work.

Learn more about rmdir and rmdirs.

Copy data

To copy data:

1. First, connect rclone to Lyve Cloud. For step by step instructions, see Configure rclone to connect to Lyve Cloud Object Storage. Once the configuration is complete, the rclone.config file must be updated as:

[REMOTE NAME]type = s3provider = Otherenv_auth = falseaccess_key_id = XXXXXXXXXXsecret_access_key = YYYYYYYYYYYYYYYYYYYYYYYYYYYendpoint = https://s3.us-east-1.lyvecloud.seagate.comacl = privateregion = us-east-1

2. Start copying data from the existing cloud provider buckets to the buckets created in Lyve Cloud. To copy data:

$ rclone copy  SOURCE REMOTE:[SOURCE BUCKET] <TARGET REMOTE>[TARGET BUCKET>/<PREFIX>]

1. To copy all the data, including prefixes, from the source bucket to the target bucket:

$ rclone copy  SRT:[ SB ]   TRT:[TB]

2. To copy all objects with a prefix to the target:

$ rclone copy  SRT:[ SB ]/mypath1  TRT:[TB]/mypath1

Deleting a remote from Lyve Cloud

To delete a remote from Lyve Cloud, delete the remote’s name using this command, changing REMOTE_NAME to the name of the remote to disconnect:

rclone config delete REMOTE_NAME [flags] 

Learn more about config delete.

Migrate data

To migrate data:

1. Set the Source and the Target as remote. For more information, see Configure rclone to connect to Lyve Cloud Object Storage.
2. Sync the Source and Target remote using rclone sync command.

rclone sync <source remote name>:path <target remote name>:path

3. Once the source and the target are synced, all the data from the source is copied, removed or migrated to the target remotely.
4. You can use the following flags in the command to check the status of the sync/copy/migration.

• --progress Displays the real-time transfer progress.
• --interactive: Enables interactive mode and displays interactive for every action taken.

For more information on RClone, see https://rclone.org/s3/.

Video: Lyve Cloud - Use Rclone Delete and Purge commands

Seagate on Vimeo: Lyve Cloud - Use Rclone Delete and Purge commands.