Amazon is offering UNLIMITED storage for only $59USD/year (with 3 months free trial).
For me – amazing thing for backups!
Unfortunately there’s only client for Windows/Mac which is pretty dumb…. It can just Upload/Download files. For manual backups from Win OK. But I wanted to do something that can do backups from my Linux machines.
Pretty nice tool that I’m using is acdcli.
It’s full documentation you can find at ReadTheDoc. So there’s no need to do full list of all features and howtos here…. I’m going to go through install and basic usage for working with the Amazon Drive.
INSTALL
Let’s start with requirements. The easiest way is through pip, and for pip we need python.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
root@hegesh:~# apt install python3-pip Reading package lists… Done Building dependency tree Reading state information… Done The following additional packages will be installed: binutils build-essential cpp cpp-5 dpkg-dev fakeroot g++ g++-5 gcc gcc-5 libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan2 libatomic1 libc-dev-bin libc6-dev libcc1-0 libcilkrts5 libdpkg-perl libexpat1-dev libfakeroot libfile-fcntllock-perl libgcc-5-dev libgomp1 libisl15 libitm1 liblsan0 libmpc3 libmpx0 libpython3-dev libpython3.5-dev libquadmath0 libstdc++-5-dev libtsan0 libubsan0 linux-libc-dev make manpages-dev python-pip-whl python3-dev python3-setuptools python3-wheel python3.5-dev Suggested packages: binutils-doc cpp-doc gcc-5-locales debian-keyring g++-multilib g++-5-multilib gcc-5-doc libstdc++6-5-dbg gcc-multilib autoconf automake libtool flex bison gdb gcc-doc gcc-5-multilib libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg glibc-doc libstdc++-5-doc make-doc python-setuptools-doc The following NEW packages will be installed: binutils build-essential cpp cpp-5 dpkg-dev fakeroot g++ g++-5 gcc gcc-5 libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan2 libatomic1 libc-dev-bin libc6-dev libcc1-0 libcilkrts5 libdpkg-perl libexpat1-dev libfakeroot libfile-fcntllock-perl libgcc-5-dev libgomp1 libisl15 libitm1 liblsan0 libmpc3 libmpx0 libpython3-dev libpython3.5-dev libquadmath0 libstdc++-5-dev libtsan0 libubsan0 linux-libc-dev make manpages-dev python-pip-whl python3-dev python3-pip python3-setuptools python3-wheel python3.5-dev 0 upgraded, 45 newly installed, 0 to remove and 0 not upgraded. Need to get 77.5 MB of archives. After this operation, 201 MB of additional disk space will be used. Do you want to continue? [Y/n] |
Quick hint (not sure if bug or feature of Ubuntu). I was receiving error:
1 2 |
You are using pip version 8.1.1, however version 9.0.1 is available. You should consider upgrading via the ‘pip install –upgrade pip’ command. |
The fix is easy – upgrade pip, but not through apt or the command that is recomended… But like that:
1 |
root@hegesh:~# pip3 install -U pip |
To do the initial “init” procedure we also need web browser to save the oauth file. I like w3m, but it really depends on your taste.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
root@hegesh:~# apt install w3m Reading package lists… Done Building dependency tree Reading state information… Done The following additional packages will be installed: libgc1c2 Suggested packages: w3m-img libsixel-bin xdg-utils w3m-el cmigemo The following NEW packages will be installed: libgc1c2 w3m 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded. Need to get 973 kB of archives. After this operation, 2,755 kB of additional disk space will be used. Do you want to continue? [Y/n] |
And finally install the acdcli with pip. I used the instalation from the git (this is great to obtain the latest version. Also we use this for upgrading 😉 )
1 |
pip3 install —upgrade git+https://github.com/yadayada/acd_cli.git |
Then we can create the initial init procedure (if you don’t have amazon account yet it’s better to start creating it from your workstation browser)
1 2 |
root@hegesh:~# acdcli init For the one-time authentication a browser (tab) will be opened at https://tensile-runway-92512.appspot.com/. Please accept the request and save the plaintext response data into a file called “oauth_data” in the directory “/root/.cache/acd_cli”. Press a key to open a browser. |
* Log in your Amazon account
* You receave the oauth_data
* save with <Shift><S> to /root/.cache/acd_cli/oauth_data
* little trick – it’s a bit crippled… so with ! you can exec shell and edit /root/.cache/acd_cli/oauth_data so it will contain something like:
{
“access_token”: “Atza|IwEBIMso……”,
“expires_in”: 3600,
“refresh_token”: “Atzr|IwEBIDeU……”,
“token_type”: “bearer”
}
And we can do the first sync
1 2 3 4 |
root@hegesh:~# acdcli sync Getting changes.. Inserting nodes. root@hegesh:~# |
From this moment we can use all acdcli features
1 2 3 4 5 6 7 |
root@hegesh:~# acdcli usage: acd_cli [–h] [–v | –d] [–nl] [–c {always,never,auto}] [–i {full,none,quick}] [–u] {version,v,sync,s,old–sync,psync,clear–cache,cc,tree,t,children,ls,dir,find,f,find–md5,fh,find–regex,fr,upload,ul,overwrite,ov,stream,st,download,dl,cat,create,c,mkdir,list–trash,lt,trash,rm,restore,re,move,mv,rename,rn,resolve,rs,add–child,ac,remove–child,rc,usage,u,quota,q,metadata,m,mount,umount,delete–everything,init,i,test} ... acd_cli: error: the following arguments are required: action root@hegesh:~# |
Next step is to create a directory where we want to save our data (this can be done with the web GUI. But really important is to do the sync to have the latest structure available.
1 2 3 4 5 |
root@hegesh:~# acdcli mkdir TESTbackup root@hegesh:~# acdcli sync Getting changes.. Inserting nodes. root@hegesh:~# |
And the last step is to finally backup what we want, where we want……
Thanks to Linux amazing feature to pipe one output to another one – we can make all needed in one step (tar/GPG/save)
tar what I want (in this case /etc )
-> GPG (configured to be possible to run from batch, taking password from file
-> saving directly to amazon
1 2 3 4 |
root@hegesh:~# tar -cz /etc | gpg –no-use-agent –passphrase-file ~/passwd –symmetric –batch –yes -c -o- | acdcli stream testing001.tgz.gpg /TESTbackup/ tar: Removing leading `/‘ from member names [#########################] 100.0% of 621KiB 1/1 211.6KB/s 0s root@hegesh:~# |
The reason why I use GPG in the middle is to have the feeling of encrypted data (and in case I lost the account, Amazon decided to sniff around etc. the stealing of this will be a bit more difficult)
And the final script that is run by cron is:
1 2 3 4 5 6 7 8 9 10 11 |
#! /bin/bash # script to create a tar backup file BACKUPDATE=`date +%Y%m%d` BACKUPNAME=$BACKUPDATE.full echo $BACKUPDATE echo $BACKUPNAME acdcli sync tar –cz /data/backup | gpg —no–use–agent —passphrase–file ~/passwd —symmetric —batch —yes –c –o– | /usr/local/bin/acdcli stream $BACKUPNAME.tgz.gpg /lnxBACKUPs/ |
this it the first version… maybe I’ll put some more notes later 😉
Leave a Reply