It’s not that everyone should have two network storage devices but if you use your NAS as a primary storage device rather than as a backup of your PC, media center, etc. then perhaps you should consider how you would recover if it failed.

Unfortunate for me, the fundamental rule of backups reared its head while I was away on a recent business trip. I returned home to a NAS that had pretty lights on but nobody home. It would not power down and would not respond to any access. I had no choice but to crash it by unplugging the power. YIKES. It came back up and reported a brain aneurism and no memory of the 5.6TB volume that help everything! I assumed the worse but decided not to panic. After about 2 hours of research, I took the risk and did a firmware re-install. The ReadyNAS came back to life and it had all of my files. With the catastrophe averted, I set about establishing a backup plan.

The ReadyNAS 1100 is an "upgrade" to my first NAS, the ReadyNAS NV+. Technically, the NV+ is newer technology and has more RAM but the 1100 is nearly 3x the storage. So the 1100 is the primary on-line storage. That left me the NV+ which I could configure as a backup. So my plan was to setup things up that I could leave the NV+ unplugged 6 days a week. One day a week I would power it up, and press the "backup button" on the front and have it configured to backup all the changes from the 1100. When finished, I’d power it down and unplug it. If I get really concerned, I could even take the NV+ off site but I’m not there …. yet. Here is how to backup one NAS to another NAS. It assumes the volume has been organized into 4 "shares’ but it works fine with any number.

1. for convenience, create the same list of shares on the target NAS (the "secondary") as on the source NAS (the "primary")
2. on the target NAS, create a new backup job
3. the source for the backup is the primary NAS
4. use "NSF" as the transport
5. select one of your shares on the source
6. on the ReadyNAS you can test the connection
7. select the corresponding share on the target
8. do not establish a schedule for the backup
9. save the job
10. run the job
11. let the job complete
12. edit the job
13. change the source settings to use RSYNC (requires the admin user name and it’s password)
14. on the ReadyNAS you can test the connection
15. save the job
16. repeat for all of the shares

You want to complete the first backup using NFS because it will be much faster than starting with RSYNC. You then switch to RSYNC because it is is well suited to managing the incremental changes that occur over time.

You can choose to schedule the jobs or, as in my case, run them manually by adding them to the list of jobs triggered by the "backup button".

The only errors I received during the process were from some read only files on my target NAS so I corrected the attributes so they could be updated if/when the source files from the primary NAS were newer.

I picked up a used ReadyNAS 1100 to precede my ReadyNAS NV+. It won’t give me an extra bedroom or more kitchen space but it will keep track of my electronic document storage, email, music, videos, and photos with plenty of room to spare.

In reality, the 1100 is an older unit than my ReadyNAS NV+. However, it has two advantages for the farmhouse project. First, it is a 1U rack mount unit so it will go into the equipment room and completely out of sight and sound of the residence. Second, it is an 8 terabyte until ! Of course, 8TB with RAID5 redundancy means only 6TB of usable storage but that is a lot more than my current 2.1TB of usable storage.

My plan is to to make the 1100 the primary "on all the time" unit and then do weekly backups of primary data to the NV+. The NV+ will only power up once each week for a few hours to accomplish the backups (probably using rsync or something similar). In addition to my personal and work files, PC backups, and all of the documentation for the farmhouse project, the "primary data" will also include all of my photography – especially as I start to digitize nearly 15 years of 35mm photography from my time over seas in China, Japan, Singapore, and Borneo, as well as my travels around the USA visiting family and friends.

I was curious so I did some digging to understand what’s inside the 1100. I had to chuckle at what I found. The ReadyNAS units run a trimmed down embedded instance of Linux. In addition to the file storage operating system and some utilities, end users can install a few addons. All of this gets stored in "flash memory". The ReadyNAS 1100 specs say it has 256Mg of flash memory. When you see a picture of the system board, you may be surprised at how that "256 MB of flash storage is installed" – it’s a USB FLASH DRIVE !

The USB flash drive is clearly visible on the left of the photo. The 512MB of RAM is a standard laptop DIMM located near the middle of the photo.

I recently built a working Windows XP machine because Ubuntu was just not supported well enough for work. I used my old T40 so I could preserve my T60p until I was sure I had everything right.

Today I went to move the system over. I’ve done this many times over the years both as my computer was upgraded and when migrating demos to temporary machines. I’ve done lots of moves …. T20 -> T40, T40 -> T42, T42p -> T41, X60 -> T60p and even did an X60/T60p swappable drive once. So, I thought the T40 -> T60p would be just as quick as all the rest. ANGK!

I partitioned the new hard drive just like the old one – three partitions: 256MB, 32GB, ‘remainder’. I installed my SystemRescueCD files to the HDD and installed GRUB as well. Just for the record (since I’ve managed to loose my notes three times now), here are the instructions for installing GRUB to the 256mb partition along with SystemRescueCD …

1. boot from SystemRescueCD
2. create first partition on HDD is 256MB (yes, that little). It will be called “sda1″
3. format as ext2
4. copy SystemRescueCD files and “boot” directory to “sda1″ (already has my menu.lst customized)
5. start GRUB from the SystemRescueCD command prompt
6. GRUB> root (hd0,0)
7. GRUB> setup (hd0) -or- setup (hd0,0) hd0
8. GRUB> quit

So far, so good. I was on a roll. I then launched PARTIMAGE and restored the backup of the T40 to the 32GB partition.

I rebooted, saw the boot loader, the Windows splash screen and the BSOD …. hmmm. that’s odd. I’ve never had that happen before. …

A few more attempts hoping it was just a fluke and then it was off to Google for help. After about 30 minutes, I had my answer. the T60 has a SATA drive and WindowsXP is so “old school” (first released in 2001) it has not default drivers for it. Lots of attempts to get the drivers over and nothing worked. I went off and did other stuff. I cam back and Googled more and tried more stuff. I went off again …

I finally found out about nLite and created a Windows XP install disk with the SATA drivers. I figured I could just use the Recovery Console and add the missing driver and I’d be set. Well, I could not boot my Windows XP install CD and I could get to the Recovery Console and look around my restored drive but I could not find a way to actually install the missing driver. Turns out you can’t add a missing driver even if you have the driver and the Recovery Console lets you see your drive.

The magic (slow painful magic) came from an article discussing how to recover form replacing your PC’s system board. The writer clearly states; “do not use the option for “repair a Windows XP installation using the Recovery Console”. Instead, start the Windows install and then select “repair” once it has found your existing installation. I can only say “that’s far from intuitive!”

I had six or seven nasty looking “missing or corrupted” type pop-ups. I just clicked “OK”. I figured at this point, I’d been walking on thin ice for so long, I might as well dance the jig.

It took an hour for Windows to do what it thought it needed to do and I on the first reboot I saw that pretty blue screen again. It seems Windows never got around to actually installing the missing SATA driver !

… Oy vey !

Doing sufficient backups, building virtual machines, and storing my transient video collection “on-line” requires storage, storage, and more storage. With three computers now getting sufficient backups to give me moderate peace of mind, I needed more storage than my RAID1 NAS (network addressable storage) was ready to serve. I had 2 750GB drives but RAID1 used those as mirrors so I only got the real capacity of one drive (and in reality, you never get 100% of a disk drive and so it was more like 700GB).

Today I added 2 more 750 GB drives. The system reconfigured itself for RAID5 so now I have 2 TB (yes, TB = terabytes). So, it should be a while before I run out of storage and more important, with RAID5 I have hot swappable storage – if any one disk drive should ever fail.

the down side now is there is no way I will ever have a fallback for the NAS. It has become a single point of failure if – unlikely as it may seem – the entire thing should bite the dust. (I truly hope that writing that was not asking for trouble.)

BTW: for those with slow-speed internet, I apologize for the 360KB image. I tried to make it smaller but it started at 2.7MB so I feel I did “OK” when all things are considered. For those asking “how”, all I can say is; “GIMP is one of the most amazing photo manipulation tools I have ever used – and I’ve used Photoshop and Photoshop Elements for years … and GIMP is free !

I am trying to create an backup solution for a Windows machine. I have grown to really like the SimpleLinuxBackup solution I have on the rest of my machines. It is simple (obviously) and performs both full backups and incremental backups.

There are five characteristics I need …

1. “what to include” – directories and files to save; needs to support wildcards
2. “what to exclude” – directories and files to skip; needs to support wildcards
3. “since last backup” – support a timestamp (saved after the last successful backup) as a filter of what to backup on the next iteration
4. Create a single compressed file (using formats like ZIP, 7z, or GZip) containing all files selected for the backup
5. Free or reasonably price

On Linux TAR does all of this and it does it nicely. I just use the following:

tar -czvf outputfile.tar.gz timestamp -I file-with-include-list -X file-with-exclude-list -N=file-with-timestamp

All of the version of TAR I have found for Windows either has bugs related to the first three features or the documentation is so bad/void, there is no way to diagnose the generic error messages they spit out.

Most Windows solutions rely on the “archive bit” but that is not a reliable mark for what has been backuped with incremental backups.

So, what backup tools do you use ? What would you suggest?

At MacWorld 2007, Steve Jobs announced “Time Machine” which heralding in an easy to use, consumer focused solution for backing up your computer Mac with Leopard. However, it was not for everyone. It assumed the backup device – a USB connected hard drive was always plugged in which is fine for a desktop computer but not so good for a laptop and not great for a traveling laptop. At MacWorld 2008, Steve Jobs announced “Time Capsule” which addressed the USB connected issue by making Time Machine work over a high speed wireless connection to a small “network storage appliance”. Life got better.

But I can attest that neither Time Machine nor Time Capsule would have saved me a couple of weeks ago when I was traveling and I managed to hose (technical term) my laptop. What I needed was an “internet storage solution”. What I needed was Amazon’s S3 (Simple Storage Service).

So, will internet based storage be the next offering for iTunes ? Probably not – at least not with that name. But, Apple must be considering how to get some form of monthly reoccurring revenue from its consumer base. An Apple equivalent of S3 could appeal to owners of MacBooks, iPhones, and even iPods (specifically the iTouch).

My backup strategy already does periodic full system images and daily incremental backups to a network storage device. It could be easily adapted to a service like Amazon S3. I did a quick estimate and with my “unusually high volumne for a consumer” 25GB worth of backups (and assuming half of that changes every month) I only would pay $6.25/month. If you think 25GB is low, don’t forget that compression reduced most files other than your music and video. If you still think it is low, then assume no compression and your whole laptop will likely only run you about $12.50/month. (Here is a calculator to see what your backup storage would cost.)

It is only a matter of time before the notion of giving consumers a backup strategy that is easy to use, nearly seamless, and “out in the internet cloud” is too obvious to ignore and if it comes form Google or Apple, it may be too easy to avoid. The only barrier is marketing and Apple seems to have that in spades.

Related: There are already rumors of Google storage offerings. Amanda Networks has already delivered a backup solution that leverage Amazon S3 in the background. You can use JungleDisk to connect Amazon S3 as a network drive under OS X, Windows, and Linux.