how to move tons of data using rsync


So: I bought a new 500 GB NAS from NewEgg to consolidate the 160 GB NAS + 250 GB USB2 drive Frankensteinian setup I’ve been using as a Media Dump — I love the linux-based SimpleShare as a data source for the XBox Media Center, but plugged in external USB drives tend not to want to spin down, generating heat and wasting electricity, so I got a large one to consolidate the big mess into something more manageable, quieter, and hopefully less of an electricity hog.

But here’s the problem — how does one move 400 GB of audio and video around from SMB share to SMB share, anyhow? First I (naively) tried Windows’ “copy” method — this actually got through quite a bit through before dying at a random point, leaving me no clue what was copied and what was left. Then, I tried Mac OS X, which also died midway through the transfer at a seemingly random and unknowable point. So that sucked.

So then I decided to pull out my Cruddy Ubuntu laptop(tm) and try rsync. After I installed the linux smb client:

sudo apt-get install smbfs smbclient

…and figured out how to map my SMB shares to local folders:

mkdir new_media
mkdir old_media
sudo mount -t smbfs //10.0.1.5/Media /home/jeff/old_media
sudo mount -t smbfs //10.0.1.10/Media /home/jeff/new_media

…all it took was one single command (and about 36 hours!) to move about 400 GB worth of data:

sudo rsync --progress -r /home/jeff/old_media/ /home/jeff/new_media/

I’m sure there’s a better way to do this (and if so, let me know!), but I was pretty psyched at how effectively it worked once everything was in place; rsync even moves things in alphabetical order, versus Mac and Windows, which both appear to copy files using some inscrutable, spindly method based on reasons long forgotten.

~Jeff

2 thoughts on “how to move tons of data using rsync”

Comments are closed.