Rclone给dockek数据做整体备份
一、下载安装Rclone
官网地址:Rclone downloads
Linux debian安装命令
curl https://rclone.org/install.sh | sudo bash
运行rclone config命令开始配置:
Current remotes:
Name Type
==== ====
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n #输入n 建立新的
Enter name for new remote.
name> data # 输入一个名字
选择网盘序号
Option Storage.
Type of storage to configure.
Choose a number from below, or type in your own value.
1 / 1Fichier
\ (fichier)
....
18 / Google Drive
\ (drive)
Storage> 18 # 例如GoogleDrive这里可以看到是18号,就输入18
输入参数
Option client_id.
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a value. Press Enter to leave empty.
client_id> # 留空回车
Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a value. Press Enter to leave empty.
client_secret> # 留空回车
也可以配置 GoogleDriveAPI
Google API 控制台地址:https://console.developers.google.com/
给申请的网盘读写权限
Option scope.
Scope that rclone should use when requesting access from drive.
Choose a number from below, or type in your own value.
Press Enter to leave empty.
1 / Full access all files, excluding Application Data Folder.
\ (drive)
2 / Read-only access to file metadata and file contents.
\ (drive.readonly)
/ Access to files created by rclone only.
3 | These are visible in the drive website.
| File authorization is revoked when the user deauthorizes the app.
\ (drive.file)
/ Allows read and write access to the Application Data folder.
4 | This is not visible in the drive website.
\ (drive.appfolder)
/ Allows read-only access to file metadata but
5 | does not allow any access to read or download file content.
\ (drive.metadata.readonly)
scope> 1 # 输入1,给予读写权限
不使用高级配置
Option service_account_file.
Service Account Credentials JSON file path.
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.
Enter a value. Press Enter to leave empty.
service_account_file> # 直接回车
Edit advanced config?
y) Yes
n) No (default)
y/n> # 直接回车
使用自己的配置,不使用自动配置
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine
y) Yes (default)
n) No
y/n> # 输入 n
Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
rclone authorize "drive" "eyJzY29wZSI6ImRyaXZlIn0"
Then paste the result.
Enter a value.
config_token> # 回到本地 windows 上
进入:https://rclone.org/downloads/ 下载 windows 平台的 rclone ,并打开 rclone 文件夹
管理员命令提示符(cmd)输入下面的命令
rclone authorize "drive"
会自动打开浏览器让你登录你的google账户,然后登录你的账户。
点击允许后,浏览器出现Success!提示证明成功了。
然后返回cmd窗口
复制 access_token 注意:连同大括号一同复制,从大括号开始到大括号结束。
然后把复制的 code 粘贴到 SSH 终端上回车。
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No (default)
y/n> #输入 n 默认回车就是 n 了
Configuration complete.
Options:
- type: drive
- scope: drive
- token: {"access_token":"ya2PJrRmKORRz9dTlkq-on8XQQE65dr8CQEx-3PjmyPoFw6VxZxYYw0163","token_type":"Bearer","refresh_token":"1//0e7xWWDX2tRgPCgYIARAAGA4SNwF-L9Irvw-ZnCue7vgz6kb6jYstKGyZsBliCvNGbj9tWd1mZGEoegiCYxoPIU0z9j6nmFck41c","expiry":"2022-09-08T22:42:59.5596286+08:00"}
- team_drive:
Keep this "1" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> # 输入 y 完成。
二、将docker文件夹自动打包定时上传至GoogleDrive
1.在根目录新建备份文件夹和执行脚本
mkdir bak && nano beifen.sh
在脚本中填入
#!/bin/bash
#将root根目录下data文件夹按照当前日期时间打包到root目录下载bak文件夹下
tar czvf bak/"$(date "+%Y%m%d%H%M").tar.gz" data/
#利用rclone copy命令,将指定bak目录下新增加的文件,同步到Google网盘中。/bak是服务器上新建的备份目录 "data(rclone新建网盘名称):data(Google drive内的文件夹名称)"
rclone copy --update --verbose --transfers 30 --checkers 8 --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 --stats 1s "/bak" "data:data"
#输入日志即表示执行了一次备份命令
echo "BACKUP DATE:" $(date +"%Y-%m-%d %H:%M:%S") >> /var/log/backup.log
# 删除/bak文件夹下30天以前的备份文件
find bak/ -mtime +30 -name "*.tar.gz" -exec rm -rf {} \;
ctrl+x保存,然后赋予beifen.sh
执行权限:
chmod +x beifen.sh
2.测试脚本是否可以正常运行
./beifen.sh
然后查看bak目录和Google drive是否有更新文件
3.使用crontab添加定时运行脚本任务
输入:
crontab -e
选择1,默认nano打开配置文件
输入定时执行任务
0 1 * * * /root/beifen.sh #代表每个月每个星期每一天凌晨1点执行一次脚本 “分 时 天 月 星期”
sync同步备份
利用的rclone sync命令,将本地目录与网盘的文件夹保持一致。
rclone sync是双向操作,脚本如下:
#!/bin/bash
tar -czvf bak/"$(date "+%Y%m%d%H%M").tar.gz" data/
rclone sync -i /backup data:data --progress
find /bak -mtime +30 -name "*.zip" -exec rm -rf {} \;
echo "BACKUP DATE:" $(date +"%Y-%m-%d %H:%M:%S") >> /var/log/backup.log
mount挂载同步
利用的rclone mount命令,将谷歌网盘直接挂载在本地的目录夹,所有的操作都是直接同步到网盘当中的,同时网盘当中的操作也会反映到本地。好处是不占用本地磁盘空间。
rclone mount data:/data bak/ --no-check-certificate --allow-other --allow-non-empty --vfs-cache-mode full --vfs-cache-max-age 24h --vfs-cache-max-size 12G
以上命令是把名称为data的rclone,Google网盘中data文件夹,挂载到 /bak目录下。重新打开对话窗口,输入df -h就可以看到成功将谷歌100GB的网盘挂载到VPS上了。
卸载命令如下:
fusermount -qzu /bak