使用Rclone链接对象存储

一、关于Rclone

介绍

Rclone 是一个命令行程序，用于管理云存储上的文件。它 是云供应商 Web 存储的功能丰富的替代方案 接口。超过 70 种云存储产品支持 rclone，包括S3对象存储，商业和消费者文件存储 服务，以及标准传输协议。

Rclone 具有强大的云等效于 unix 命令 rsync、cp、 MV、mount、LS、NCDU、tree、rm 和 cat。Rclone熟悉的语法 包括壳牌管道支持和保护。是的 在命令行、脚本中或通过其 API 使用。

功能

• 将文件备份（和加密）到云存储
• 从云存储还原（和解密）文件
• 将云数据镜像到其他云服务或本地
• 将数据迁移到云，或在云存储供应商之间迁移
• 将多个、加密、缓存或不同的云存储挂载为磁盘
• 使用 lsf、ljson、size、ncdu 分析和核算云存储上保存的数据
• 将文件系统合并在一起，将多个本地和/或云文件系统呈现为一个

二、安装配置Rclone

1、安装unzip

Centos、Redhat

yum install unzip -y

Ubuntu

apt-get install unzip 

2、安装rclone

官方提供了安装脚本，通过脚本安装即可

curl https://rclone.org/install.sh | sudo bash

也可以手动下载，选择版本安装，下载地址：https://rclone.org/downloads/

3、安装后验证

rclone version

4、配置rclone连接对象存储

配置有两种方式，一种直接写配置文件，一种是按照rclone引导配置

方式一：配置文件

[root@k8s-node2 logs]# vim /root/.config/rclone/rclone.conf

[qiniu_y]
type = s3
provider = Qiniu
access_key_id =  【七牛云AK】
secret_access_key = 【七牛云SK】
endpoint = 【七牛云bucket的S3地址】
location_constraint = 【七牛云bucket的S3地址】
acl = public-read 
region = cn-east-1
storage_class = STANDARD

[qiniu_y] ：rclone 配置的名称

type：连接类型，使用s3连接，七牛云可以使用s3连接

provider：运营商，这里选择七牛云，也可以使用Aliyun、Other等

acl：权限，private(私有)、public-read（公有读取）、public-read-write（公有读写）

配置文件写好之后，查看是否生效

rclone config
# 可以看到配置
Current remotes:

Name                 Type
====                 ====
qiniu_y              s3

方式二、按照引导配置

进入引导配置

# 进入引导配置
rclone config

按照提示新建配置,输入“n”确定

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q>n

输入配置名称，按enter确认

Enter name for new remote.
name> bbb

选择连接方式，连接方式比较多，可以直接输入s3

Storage> s3

选择运营商，可以查看列表有没有要连接运营商，没有则选择Other，我这里选择七牛

provider> Qiniu

是否开启验证，我这里不使用验证

Option env_auth.
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
Only applies if access_key_id and secret_access_key is blank.
Choose a number from below, or type in your own boolean value (true or false).
Press Enter for the default (false).
 1 / Enter AWS credentials in the next step.
   \ (false)
 2 / Get AWS credentials from the environment (env vars or IAM).
   \ (true)
env_auth> 1

输入平台的AK

Option access_key_id.
AWS Access Key ID.
Leave blank for anonymous access or runtime credentials.
Enter a value. Press Enter to leave empty.
access_key_id>

输入平台的SK

Option secret_access_key.
AWS Secret Access Key (password).
Leave blank for anonymous access or runtime credentials.
Enter a value. Press Enter to leave empty.
secret_access_key>

选择连接区域，可以看Bucker的s3的区域信息

Option region.
Region to connect to.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
   / The default endpoint - a good choice if you are unsure.
 1 | East China Region 1.
   | Needs location constraint cn-east-1.
   \ (cn-east-1)
   / East China Region 2.
 2 | Needs location constraint cn-east-2.
   \ (cn-east-2)
   / North China Region 1.
 3 | Needs location constraint cn-north-1.
   \ (cn-north-1)
   / South China Region 1.
 4 | Needs location constraint cn-south-1.
   \ (cn-south-1)
   / North America Region.
 5 | Needs location constraint us-north-1.
   \ (us-north-1)
   / Southeast Asia Region 1.
 6 | Needs location constraint ap-southeast-1.
   \ (ap-southeast-1)
   / Northeast Asia Region 1.
 7 | Needs location constraint ap-northeast-1.
   \ (ap-northeast-1)
region> 

选择对象存储节点，需要结合运营商节点选择

Option endpoint.
Endpoint for Qiniu Object Storage.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
 1 / East China Endpoint 1
   \ (s3-cn-east-1.qiniucs.com)
 2 / East China Endpoint 2
   \ (s3-cn-east-2.qiniucs.com)
 3 / North China Endpoint 1
   \ (s3-cn-north-1.qiniucs.com)
 4 / South China Endpoint 1
   \ (s3-cn-south-1.qiniucs.com)
 5 / North America Endpoint 1
   \ (s3-us-north-1.qiniucs.com)
 6 / Southeast Asia Endpoint 1
   \ (s3-ap-southeast-1.qiniucs.com)
 7 / Northeast Asia Endpoint 1
   \ (s3-ap-northeast-1.qiniucs.com)
endpoint> 

location_constraint也需要结合运营商选择

Option location_constraint.
Location constraint - must be set to match the Region.
Used when creating buckets only.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
 1 / East China Region 1
   \ (cn-east-1)
 2 / East China Region 2
   \ (cn-east-2)
 3 / North China Region 1
   \ (cn-north-1)
 4 / South China Region 1
   \ (cn-south-1)
 5 / North America Region 1
   \ (us-north-1)
 6 / Southeast Asia Region 1
   \ (ap-southeast-1)
 7 / Northeast Asia Region 1
   \ (ap-northeast-1)
location_constraint> 

acl权限，私有选择private，公读私写选择public-read，公有读写选择public-read-write

Option acl.
Canned ACL used when creating buckets and storing or copying objects.
This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.
For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl
Note that this ACL is applied when server-side copying objects as S3
doesn't copy the ACL from the source but rather writes a fresh one.
If the acl is an empty string then no X-Amz-Acl: header is added and
the default (private) will be used.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
   / Owner gets FULL_CONTROL.
 1 | No one else has access rights (default).
   \ (private)
   / Owner gets FULL_CONTROL.
 2 | The AllUsers group gets READ access.
   \ (public-read)
   / Owner gets FULL_CONTROL.
 3 | The AllUsers group gets READ and WRITE access.
   | Granting this on a bucket is generally not recommended.
   \ (public-read-write)
   / Owner gets FULL_CONTROL.
 4 | The AuthenticatedUsers group gets READ access.
   \ (authenticated-read)
   / Object owner gets FULL_CONTROL.
 5 | Bucket owner gets READ access.
   | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
   \ (bucket-owner-read)
   / Both the object owner and the bucket owner get FULL_CONTROL over the object.
 6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
   \ (bucket-owner-full-control)
acl> 

后面的都默认配置就可以了

验证对象存储是否可以连接

# 可以通过列举存储桶内的对象验证配置是否正确

rclone ls bbb:[bucker-name]