Advice for backing up multi-OS home network?

BrianE · Post by **BrianE** » Fri May 11, 2007 1:04 pm

I've got a bit of a dilemma here. I have ready to go a new 320GB hard drive and a USB external enclosure for it, which I bought because I want to backup the files off one of the computers at home. Then I decided it would probably be a good idea to make it a backup repository for all the computers we have here.

The problem is that we're running multiple OS platforms, each with different file systems. The oldest computer is still running Windows 98SE using FAT32, a newer one is running XP Pro using NTFS, and I am planning to start up a Linux box (probably Puppy Linux) in the near future, which uses EXT2 or EXT3. I realized I had a problem on my hands when I sat down and got ready to format the new drive. I would like to be able to share all the files commonly with all computers if possible, so while I could partition the drive with different file systems, I would prefer not to if possible.

They will all be connected together on a home LAN.

Sticking points:

NTFS is backwards compatible to FAT32...
...but FAT32 is not backwards compatible to NTFS
FAT32 can be common to 98, XP, and Linux....
...but EXT isn't compatible with NTFS or FAT32
Also FAT32 is old and limiting (File fragmentation, 250GB partition limit(?), 4GB file size limit?)

GAH!

There is a project underway to give Linux some NTFS functionality, but I haven't been keeping up on how it's going and I'm sure there are still bugs to iron out. Besides, that still doesn't help the 98SE computer any. So what can I do?

A) Is there a backup utility program that can convert file formats on-the-fly? I tried looking for such an animal briefly but didn't come up with much (not that I expect to....).

B) Take the HD out of the enclosure, stick it in a old computer and convert it into a file server? I'd rather not do this because it seems wasteful (I'd be inclined to leave it on 24/7) and our usage doesn't seem to warrant a server, but if I were to go this route it would be a convenient way to share our MP3s....

C) Just format the backup drive as XXXXX and do (INSERT BRILLIANT IDEA HERE).

Any questions/comments/suggestions are welcome! Thanks.

IsaacKuo · Post by **IsaacKuo** » Fri May 11, 2007 1:21 pm

I'd go with the file server route. You don't need the computer to be JUST a file server. Any of your computers will be fine. Since you're probably most comfortable with NTFS, format the drive in NTFS and hook it up (by USB) to one of the Windows XP computers. Preferably, make it the WinXP computer which is least likely to be used for other thumbdrives or flash card readers and such (because those can change the drive letter of your USB drive when it gets plugged in).

Linux NTFS functionality is actually not too bad. It can read NTFS, and writes seem to be reliable. I'd still be wary of writing to NTFS in Linux, myself, but others report having no problems. Either way, you can read NTFS which is the important thing if you need to restore data and your XP computers are down.

Trekmeister · Post by **Trekmeister** » Fri May 11, 2007 2:22 pm

If the plan is to move the drive around and hook it up to the different machines I would stick with the lowest common denominator, beeing FAT32 in this case. Yes your linux files would lose the protection bits, user/group and links if you copy them over one by one, but if you plan to back stuff up I would put the files in a tar-ball and that way preserve all the stats. I can only assume there is some archiver for XP capable of the same thing, if needed.

Another way would be to create a file on the drive containing the EXT filesystem and mount it. Unfortunatley I think you would be stuck with a static size filesystem, but I would on the other hand not be suprised if there are tools to make the file expand and shrink automatically depending on the amount of stored data.

Third, if you are going to go the file server route I would say stick the disk in your linux machine (or connect it by USB) and format it to EXT3. Then share it using samba to your other machines. (and enjoy as you encounter the windows way of not following standards.

)

Gojira-X · Post by **Gojira-X** » Fri May 11, 2007 2:32 pm

if you are going to format to ext2, if you have the Ext 2 Installable File System You don't have to worry about your windows XP machine getting access.

There is explore2fs and ext2fsd that may or may not do the same thing in win98.

nzimmers · Post by **nzimmers** » Fri May 11, 2007 2:55 pm

well, you could try simply backing up the Hard disk partition in it's entirety.

I'm pretty sure Acronis (and a few others) would be able to handle it.....boot from a CD, and backup the entire internal disk to the external drive.

or, you could setup some kind of file sharing, backup all your data across the network to a windows PC with the external drive attached.

A home server these days is almost becoming a requirement when you have 2+ computers at home, just for convenience.

Devonavar · Post by **Devonavar** » Fri May 11, 2007 4:34 pm

Speaking from experience, I'd avoid the Ext 2 Installable File System if you use Unicode at all.

Reading the fine print, you'll find that the driver doesn't support file names with Unicode characters (i.e. any Asian character sets). When you try to copy files with Unicode names to the Ext 2 partition, you'll get an inaccessible, undeletable file with the Unicode characters recognized improperly.

I can only assume that they'd be accessible with a proper Linux driver, but for use in conjunction with Windows XP, it's a no go.

Of course, you may have similar issues with FAT32, but I can only assume that FAT32's response to Unicode File names doesn't create undeletable files...

IsaacKuo · Post by **IsaacKuo** » Fri May 11, 2007 6:02 pm

Interesting. I've never tried using a Windows program to access a (local) ext2/3 partition, so I haven't run into that problem. Accessing Unicode filename files locally on a Linux computer works fine, as well as accessing them remotely from a Windows computer via Windows Networking/SAMBA.

Strange, that you don't have any experience with using a "proper Linux driver" with ext2. Why were you messing with ext2 at all, without Linux?

floffe · Post by **floffe** » Sun May 13, 2007 2:36 pm

Tarballs of what you want to back up from the linux partitions make sense (created with -p to preserve permission/owner data), and can easily be compressed too. Then they can be copied over to a FAT (or NTFS, if it's on an XP computer) partition on the external drive, wherever it may be.

BrianE · Post by **BrianE** » Sun May 13, 2007 10:05 pm

Thanks for all the replies.

Just to clarify, the purpose of this drive is just to backup valuable data and various media files, rather than whole drive/OS images.

I'd also like to keep any additional costs minimal if possible.

No, no Unicode characters used so that's okay.

After looking into the various options of trying to get all 3 computers "speaking the same language" I think the server option is looking more and more attractive since a network is a great universal interface. FAT32 is conveniently common to all 3 file systems, but I think it's on its way out and then there's that pesky 4GB file size limit. Someday when that Win98 machine is retired, FAT32 will just be a legacy. I can no longer remember the huge hairball I ran into trying to figure out the various means of reading/writing Ext/NTFS, but I recall that there were limitations or costs involved and it all seemed a bit inconvenient, so this didn't look attractive. I could also do what IsaacKuo suggested and just share the drive from the XP computer, but I do find the MP3 server idea compelling and also this way we wouldn't have to power up the (gaming) XP computer just to back something up.

Soooooo.... I've just spent the last couple days pouring over the various options and subtleties of setting up a home NAS server. The free NAS programs out there look attractive because they are (supposedly) simpler to set up than trying to configure a full blown Linux distrubution as a server, and can use obsolete computers to run huge modern hard drives. This would be ideal because we have an old Pentium 166 in a nice heavy steel case collecting dust. It should also consume little power and be fairly easy to silence. I'm thinking of setting it up so that I have two drives in it - the 320GB drive for backup duty, and a smaller drive for serving MP3's and frequently accessed files. It would also serve as a buffer drive in between periodic backups to the larger drive. My plan is to run the large drive as infrequently as possible (within reason).

I looked at NASLite products for a while, and liked how simple and matured they seemed to be, but the fact that not even the paid versions offered power management options like HD spin down is a bit of a deal breaker. It seems that even BIOS ACPI settings are either ignored or rendered moot because the software does a SMART check on all the drives every 5 minutes. Since I don't need (or want) even the smaller HD to be active 24/7 I would have to either power on and off the NAS whenever I needed it, or at least connect the 320GB drive with USB, which would require me to buy a USB controller for this old machine (and hope it works...).

FreeNAS does almost everything I want to do and sounds great (other than buggy RAID which I don't need), BUT it is still under development and currently in Beta form. It also has the minor annoyance of adding a fourth file system to the mix - FreeBSD's UFS.

The upside to UFS is that it is less prone to file fragmentation than FAT/NTFS, and at least there are utilities to read UFS from Windows in a pinch. I suppose this may be my best option since it lets me leave the server running 24/7 with the drives inactive, and it seems there are a fair number of people who are using it successfully. I haven't really configured much of anything beyond Windows as of yet, but just as how I'll be testing out the Linux waters soon, I don't mind tinkering around with it and learning as I go... as long as the learning curve isn't a sheer cliff of course!

Anyone think I should reconsider this or have any comments/experience with FreeNAS? I know I have a tendency to fall in love with an idea and then I get tunnel vision until I realize it's beyond my means or ability, so I'd like second opinions.

EDIT: Any other suggestions for NAS software I should look into that are similar to NASLite/FreeNAS?

IsaacKuo · Post by **IsaacKuo** » Sun May 13, 2007 11:45 pm

Be aware that your Pentium 166 computer will need an expansion card in order to deal with a 320gig hard drive. The on board IDE controller most likely lacks LBA capability entirely, and certainly lacks double-pump LBA capability. If there's any USB at all, it will be the entirely too slow USB 1 standard.

I personally would rather spend a little more money on a cheap mATX Via motherboard and put together a more modern computer for a file server than make one out of a Pentium 166. The Pentium 166 can make for a nice little X terminal.

As for the operating system--I'd just put Debian 4.0 on it, if none of the "simple" options offer the features you want out-of-box. I do my backups with a simple little script which mounts the drive, rsyncs just the changes (MUCH faster than a full file backup!!!), and then unmounts/spins down the drive.

A slightly simpler option is to simply set the hard drive's spin down time and mount the drive in ext2 mode. This eliminates the journaling which periodically "touches" the hard drive (preventing spin down). But I prefer the method of mounting/unmounting the drive, because it keeps journaling and also prevents any random file browsing from spinning up the drive.

Devonavar · Post by **Devonavar** » Mon May 14, 2007 12:33 pm

IsaacKuo wrote:Strange, that you don't have any experience with using a "proper Linux driver" with ext2. Why were you messing with ext2 at all, without Linux?

Failed migration experiment.

BrianE · Post by **BrianE** » Mon May 14, 2007 12:37 pm

IsaacKuo wrote:Be aware that your Pentium 166 computer will need an expansion card in order to deal with a 320gig hard drive. The on board IDE controller most likely lacks LBA capability entirely, and certainly lacks double-pump LBA capability. If there's any USB at all, it will be the entirely too slow USB 1 standard.

That's what I thought too, but it seems that both these programs can bypass any BIOS LBA limitations and access the disk directly. This is more explicitly stated on the NASLite site (link) and forum, where they have staff on hand, and with some digging around it seems that FreeNAS will also do this. I'm not sure how the Linux-based NASLite does it, but from what I gather FreeNAS behaves this way because this is the way FreeBSD works.

I spent a lot of time yesterday trying to scour forums and the internet for confirmation and more info on this, and there seems that there is a fair amount of anecdotal evidence that 28-bit LBA machines (ATA33/66) can work around the "137GB barrier" when running Linux/FreeBSD. I seem to recall reading something about installing the kernel in the first 1024 cylinders of the boot drive sometimes being a factor too....

I know I will be limited to ATA/33 transfer speeds, but I won't be having heavy traffic on this thing and doesn't 33MB/s translate into just under 300Mb/s anyway?

No USB on this MB by the way.... it's a 1997 vintage Asus TX97.

Checking some prices though, I can get a USB PCI card for about $10 new so if I need one it wouldn't be a big deal beyond hoping it works.

IsaacKuo · Post by **IsaacKuo** » Mon May 14, 2007 1:02 pm

The problem is that the hardware won't be able to do any sort of LBA. My experience with my P120 is that it simply can't deal with any hard drive bigger than 2gigs in size, period. Linux or Windows, it doesn't matter. It just can't do it.

Neither Linux nor *BSD use the BIOS to communicate with the IDE controller, so they can bypass artifical BIOS limitations. However, they can't magically make the IDE controller capable of LBA or double-pumping if the hardware simply lacks the capability.

Maybe there's something that I just don't know about; some sort of "trick" to get things to work. For example, many hard drives have a jumper which limits the apparent size to 32gigs to the BIOS. This apparently will often work to get the motherboard to boot up and then Linux just ignores the BIOS and accesses the full size of the hard drive anyway. However, a 320gig hard drive isn't going to have a jumper to limit the apparent size to 2gigs.

BrianE · Post by **BrianE** » Mon May 14, 2007 10:59 pm

Hmm, I went to double check and actually this MB does have USB.... 1.x [sarcasm]Yay.[/sarcasm]

IsaacKuo wrote:The problem is that the hardware won't be able to do any sort of LBA. My experience with my P120 is that it simply can't deal with any hard drive bigger than 2gigs in size, period. Linux or Windows, it doesn't matter. It just can't do it.

Neither Linux nor *BSD use the BIOS to communicate with the IDE controller, so they can bypass artifical BIOS limitations. However, they can't magically make the IDE controller capable of LBA or double-pumping if the hardware simply lacks the capability.

Okay I just got done doing some heavy duty research (again) on this issue just to confirm that I wasn't having delusions or something since it was pretty late when I looked into this. The more I read up on this the more I am convinced that, barring some software incompatability or the HD's compatability with ATA/33, this might actually work....

First I read up on LBA and the issue that arises between hard drive addressing limitations and how LBA is involved. It turns out that any hard drive larger than 528MB will require LBA already. Cracking open my MB manual shows that it does support LBA functionality. Checking out some of the various internet sources seemed to indicate that LBA support is mainly a BIOS and OS issue, not hardware (other than the HD itself). While they tend to be slightly Windows-centric these pages all mention this and don't seem to say IDE controllers are limiting:
Link
48bitlba.com

These Samsung pages make for interesting reading too. This one goes over a few of the various HD size recognition limits.. This page goes over the concept of LBA and a dynamic drive overlay (DDO) a little: Link. This file 137gb.pdf from Seagate seems telling in the way it words its description of the BIOS's role and how a DDO gets around the size limit:

....The system BIOS auto-detects the hard disc drive and issues an inquiry to get the drive's number of LBAs. The largest number it is prepared to receive may be 137GB. In this case you might check with your system manufacturer for a BIOS upgrade. Many namebrand systems have the ability to "flash" the BIOS with new firmware. This process varies in difficulty and is optional.

Alternatively, BIOS support for ATA>137GB can also be accomplished by putting the necessary boot code on the disc drive itself. This small bit of programming emulates the BIOS support so that when the higher-level operating system drivers begin to launch they can "see" the full potential capacity of the drive. Seagate provides hard disc installation software called DiscWizard to help with this task. DiscWizard writes boot code called a DDO on discs that do not have native BIOS support for the full capacity of the drive. DiscWizard only writes the DDO when required.

Note: It is a common misconception that the DDO is some kind of memory resident program or that it has the potential to slow disc access by being in memory. Like all BIOS routines, the DDO is discarded from memory after about 5 seconds into the launch of the operating system when the 32-bit direct access device drivers take over. The DDO exists only to detect and then present the full capacity of the disc drive to these operating system device drivers.

I failed to find much in the way of authoratative point-blank statements that these NAS programs will definitely mesh ancient hardware and large capacity hard drives, but there is this claim from the NASLite product page I linked earlier:

For example, with NASLite v1.x you can take a retired Pentium 200MHz computer, remove the old fixed disk drives, CD-ROMs, etc., install four 160GB fixed disk drives and have a fast, reliable 640GB NASLite v1.x file server for the cost of the fixed disk drives.

A 200MHz P1 system has got to be pre-48-bit LBA and about the same hardware vintage as my computer is.

Since 48-bit LBA didn't become standard until ATA/100 (ATA-6) did, I tried looking for some examples of people using NASLite and big HDs on old ATA33/66 systems that should also be limited to <137GB by their hardware or BIOSes. No one seems to mention using IDE controller cards either:
Some people's hardware. The guy in the second post uses my exact chipset.
Some more (first page only)
Pg 2 of another thread
Terabyte server (Translated from German) Running off a 440BX (ATA/66) MB.

Oh yes, I almost forgot to mention I came across a (modded?) BIOS update for my ancient MB floating around the internet that reportedly allows it to recognize drives up to the 137GB limit. I take this as a good sign that over 137GB may not be out of reach because surely 137GB must be much higher than what the MB was originally designed for....

nzimmers · Post by **nzimmers** » Mon May 14, 2007 11:20 pm

OpenFiler is in the same vein as FreeNAS

I have heard that Openfiler might support some kind of Raid 5 that allows expanding the array with additional disks (un confirmed)

linky: http://www.openfiler.com/

oh oh oh, I almost forgot............ OpenFiler supports iSCSI and I saw a video on youtube where a guy added a folder on an openfiler machine and it was mapped as a *local disk* on a windows machine, very cool

mellon · Post by **mellon** » Tue May 15, 2007 12:09 am

Yeah, Openfiler does have iSCSI support (via http://iscsitarget.sourceforge.net/) but it is imperative to remember that iSCSI is a SAN solution, not NAS. In practise, you can only have one initiator connect to a target LUN at a time or you will get file system corruption on the target. Havng >1 initiators connect to the same target LUN would be the same as having >1 hard drive controllers running the same HDD simultaneously.

I've read somewhere that iSCSI target implementations with file locks do exist but they're not readily available. Overall, it is best to stick to a NAS system (Samba-based) unless you have a single computer/server that has need for a fast flexible storage unit. For those situations iSCSI over 1Gb ethernet is blazingly fast compared to Samba.

EDIT: Not to mention that Openfiler is really geared towards corporate use, a couple of versions ago you couldn't even get the thing working without having a separate authentication server running. I've understood that in the most recent release it contains an LDAP server but it is probably still non-trivial to set up, especially with varied Windows clients.

IsaacKuo · Post by **IsaacKuo** » Tue May 15, 2007 6:38 am

BrianE wrote:Okay I just got done doing some heavy duty research (again) on this issue just to confirm that I wasn't having delusions or something since it was pretty late when I looked into this. The more I read up on this the more I am convinced that, barring some software incompatability or the HD's compatability with ATA/33, this might actually work....

I did similar research when I was hoping to use my old equipment for a file server, and came to the same hopeful conclusions. But I couldn't ever actually get it to work. I'd attach the drive, and the motherboard wouldn't be able to POST.

I never found a nice little how-to on what to do about that.

Of course, one problem I have is that I have no documentation on my old equipment, so usually I don't know what model motherboards are involved. On my old Gateways, though, I was able to flash the motherboards to the latest BIOS. No luck, though...still wouldn't work with larger hard drives.

So, I think there must be something that I'm "missing". Maybe it's some step which everyone thinks is so obvious it's not worth mentioning.

Let me know if you have success! There are a lot of things I'd like to do if I can get my older hardware to work with larger hard drives!

flyingsherpa · Post by **flyingsherpa** » Tue May 15, 2007 8:13 am

I actually have a TX97 system (see sig) running a 120GB HD through a Promise controller card. Works great in Linux and Win98SE. I don't know if this card can do >137GB or not, but I imagine a little Googling would answer that. I think it probably can because it's advertised as an ATA/133 interface. These controller cards are pretty cheap and have never given me a problem. As long as you have a spare PCI slot you're good to go.

BrianE · Post by **BrianE** » Tue May 15, 2007 4:19 pm

Hi everyone,

I forgot to mention that I briefly looked into Openfiler already (Wikipedia links

) but I wasn't too thrilled about its higher system requirements (500mHz, 1-4GB for the OS). It would require me to use one of the newer machines I have as the server rather than just recycling a really old, unused one. Just seemed too.... "heavy" sounding for my tastes.

Isaac, thanks for the encouragement! I'll let you know how it goes. I just realized that I could just go ahead and give it a shot anyway. The only thing it might cost me is some time and patience, since everything I should need is lying around already. I vaguely recall the TX97 being a fairly popular board at the time... here's hoping that it was "mainstream" and advanced enough to make this work.

...and if it's not, yes it's true I could get a controller card for ~$30, but I think that would be my last resort since I'd like to use something I have on-hand first (like a bunch of PII components).