Hardware RAID, software RAID, 3ware 9550SX, level 1 vs. level 5

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Hardware RAID, software RAID, 3ware 9550SX, level 1 vs. level 5

David Gaudine
I have to set up a system that is totally reliable w.r.t. data
integrity.  That is, if a disk (or anything else) fails, it's OK if the
system is down for a few hours, but when it comes back up it has to be
exactly as it was, i.e. I can't restore from the previous day's backup.  
The obvious solution is to use RAID level 1.  Questions;

- Is level 1 as reliable as level 5?  My understanding is that level 5
has better performance than level 1, but comparable reliability (maybe a
bit less, since disaster occurs if 2 out of 3 disks go bad at the same
time.)

- I need to use RAID for everything, not just some partitions, so the
root has to use RAID.  I've found lots of websites that describe doing
that with software (mdadm), but very little about hardware.  Using
software is cheap and seems to be simple, but I assume hardware gives
better performance.  Is the performance difference significant?  If not,
I guess my further questions don't matter.  But I'd rather spend money
than give up performance.

- Many/most modern motherboards claim to support RAID.  Are any of them
useful, or do they all just provide a BIOS that fakes it well enough for
Windows but not for Linux?

- The 3ware 9550SX looks good, and there's even a Sarge install:
      http://www.3ware.com/KB/article.aspx?id=14860
Has anyone tried the installer?  I can't unless I buy the card (and a
computer to put it in) first.  I don't know whether it will do the whole
job of allowing me to properly set up everyting on RAID at installation
time, although that seems like the point of having it in an ISO instead
of just having a driver module.

- Any other ideas about a RAID board that's supported by a Sarge install
or for which there are step-by-step instructions how now to install
Sarge and then move it to RAID as is frequently done for mdadm?

David


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Hardware RAID, software RAID, 3ware 9550SX, level 1 vs. level 5

Johannes Wiedersich
David Gaudine wrote:
> I have to set up a system that is totally reliable w.r.t. data
> integrity.  That is, if a disk (or anything else) fails, it's OK if the
> system is down for a few hours, but when it comes back up it has to be
> exactly as it was, i.e. I can't restore from the previous day's backup.  
> The obvious solution is to use RAID level 1.  Questions;
>

I have no other experience of raid apart that I run a server with
hardware raid and another one with software raid.

My personal experience (YMMV):

Hardware raid:
- not all hardware raids are well supported by linux.
- when I upgraded the hardware raid from kernel 2.4 to 2.6, the names of
my raid were changed; so I couldn't boot any more
- a few weeks before warranty expired, our hardware raid had a hardware
failure (had it happend a few weeks later, it would have been expensive
to repair: of course that particular hardware is out of production by now)
- when your hardware fails, you have to replace your special raid controller

Software raid (mdadm)
- software raid works well, never had problems
- installation and administration is straightforward
- if some other part of your hardware fails you can take the disks and
carry them to your next computer ;-) -- or just replace the motherboard,
etc.
- should work on any standard motherboard/hardware that supports the
number of disks your require (please correct, if I'm wrong)

Maybe that's biased by my personal distribution of good luck and bad
luck, so take it with a grain of salt.

Johannes



--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Hardware RAID, software RAID, 3ware 9550SX, level 1 vs. level 5

Joseph H. Fry
In reply to this post by David Gaudine
I am partial to software raid for one important reason.... longevity.  One
great thing about linux is that it rarely makes something entirely
obsolete... and even if it does, you can always download previous versions of
your favorite distro... an array created by mdadm today will likely be
readable by most linux distros for many years to come.

this is unlike hardware which can become almost impossible to replace in a few
months sometimes.  I've heard stories about hardware controllers that
wouldn't read data from arrays created by previous versions of the same model
controller due to an updated bios, and it was almost impossible to get the
previous version from the manufacturer.

As far as the best redundancy... I like raid 50.  Raid 50 is essentially a
raid 5 array that is mirrored to another raid 5 array... this protects from
multi-drive failures as long as one of the raid 5 arrays only suffers a
single drive failure... if both have multiple failures you may be able to
recover if you can swap drives to build one array with one or zero failed
disks.   Of course, raid 50 is very wasteful... as not only do you waste 50%
capacity for the mirror, but also 1/n% (n is number of drives in each raid 5
array) for the parity.  Essentially, an array equivalent to the capacity of 2
disks would require 6 disks in raid 50 configuration.

As far as performance... yes, a true respectable hardware controller will
outperform software while also reducing the CPU load,  This will make the
machine much faster overall... for some things.  If your building a file
server for example... will it matter if the IO consumes more CPU cycles, and
if your pulling data through a 100Mb link will the network be the bottleneck
or the controller?  This is all very dependant upon your needs... I have a
couple of servers that are nothing but old workstations with PCI SATA
controllers, 4 drives, and software raid that act as file servers for a few
users... they tell me that they are faster than the Dell (w/ a PERCII SCSI HW
RAID) that they were on before.  Probably due to running a minimal
installation of Debian instead of W2K.

That's my 2 cents!

Joe.



On Friday 27 January 2006 2:57 pm, David Gaudine wrote:

> I have to set up a system that is totally reliable w.r.t. data
> integrity.  That is, if a disk (or anything else) fails, it's OK if the
> system is down for a few hours, but when it comes back up it has to be
> exactly as it was, i.e. I can't restore from the previous day's backup.
> The obvious solution is to use RAID level 1.  Questions;
>
> - Is level 1 as reliable as level 5?  My understanding is that level 5
> has better performance than level 1, but comparable reliability (maybe a
> bit less, since disaster occurs if 2 out of 3 disks go bad at the same
> time.)
>
> - I need to use RAID for everything, not just some partitions, so the
> root has to use RAID.  I've found lots of websites that describe doing
> that with software (mdadm), but very little about hardware.  Using
> software is cheap and seems to be simple, but I assume hardware gives
> better performance.  Is the performance difference significant?  If not,
> I guess my further questions don't matter.  But I'd rather spend money
> than give up performance.
>
> - Many/most modern motherboards claim to support RAID.  Are any of them
> useful, or do they all just provide a BIOS that fakes it well enough for
> Windows but not for Linux?
>
> - The 3ware 9550SX looks good, and there's even a Sarge install:
>       http://www.3ware.com/KB/article.aspx?id=14860
> Has anyone tried the installer?  I can't unless I buy the card (and a
> computer to put it in) first.  I don't know whether it will do the whole
> job of allowing me to properly set up everyting on RAID at installation
> time, although that seems like the point of having it in an ISO instead
> of just having a driver module.
>
> - Any other ideas about a RAID board that's supported by a Sarge install
> or for which there are step-by-step instructions how now to install
> Sarge and then move it to RAID as is frequently done for mdadm?
>
> David

--
Joseph H. Fry
Network Administrator
School Of Architecture
University of Detroit Mercy
(313) 993-1507
[hidden email]


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Hardware RAID, software RAID, 3ware 9550SX, level 1 vs. level 5

Stephan Seitz
On Fri, Jan 27, 2006 at 04:50:47PM -0500, Joseph H. Fry wrote:
>versions of your favorite distro... an array created by mdadm today will
>likely be readable by most linux distros for many years to come.

Yes, that's certainly true. And I think, you can boot from a software
RAID1 without much trouble, even if / or /boot are RAID1.

Hardware RAID has the advantage, that you can put your whole system on a
RAID5 array without thinking about complex initrds. But you should have a
replacement controler from the same type as backup.

I never had problems with 3ware controlers.

>As far as the best redundancy... I like raid 50.  Raid 50 is essentially
>a raid 5 array that is mirrored to another raid 5 array... this protects

Hm, are you sure? Mirroring is RAID1. RAID10 means, that you have one
RAID0 array mirrored at another one. Since you need three disks for a
RAID5, I thought RAID50 would mean a RAID5 array over at least three
RAID0 arrays (each with at least two disks, so RAID50 should need at
least 6 disks).

Shade and sweet water!

        Stephan

--
| Stephan Seitz                    E-Mail: [hidden email] |
|          WWW: http://fsing.rootsland.net/~stse/            |
| PGP Public Keys: http://fsing.rootsland.net/~stse/pgp.html |

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hardware RAID, software RAID, 3ware 9550SX, level 1 vs. level 5

Oliver Lupton
On Fri, 27 Jan 2006 23:33:54 +0100
Stephan Seitz <[hidden email]> wrote:
> Hm, are you sure? Mirroring is RAID1. RAID10 means, that you have one
> RAID0 array mirrored at another one. Since you need three disks for a
> RAID5, I thought RAID50 would mean a RAID5 array over at least three
> RAID0 arrays (each with at least two disks, so RAID50 should need at
> least 6 disks).

As I understand it, and wikipedia agrees with me, RAID 50 is the same as RAID 5+0. Or a RAID 0 array of RAID 5 arrays. I *think* that the term specifically means a RAID 0 or RAID 5's rather than vice versa.

Cheers,

-ol

--
I will live for never, or live failing.

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hardware RAID, software RAID, 3ware 9550SX, level 1 vs. level 5

Noah Dain
In reply to this post by David Gaudine
On 1/27/06, David Gaudine <[hidden email]> wrote:
> I have to set up a system that is totally reliable w.r.t. data
> integrity.  That is, if a disk (or anything else) fails, it's OK if the
> system is down for a few hours, but when it comes back up it has to be
> exactly as it was, i.e. I can't restore from the previous day's backup.
> The obvious solution is to use RAID level 1.

Hardly.  RAID levels do not guarantee filesystem integrity.  In fact,
raid has _nothing_ to do with it.  RAID is just one aspect of
high-availability subsystems, and the most common.  If you lose a
harddrive, the system keeps on going.  This is all raid gets you with
respect to reliability.

Now, if the machine were to crash from some form of hardware failure
(cpu, ram, cosmic radiation, etc.) or software bug (ie. filesystem),
raid has nothing to do with maintaining the validity of the data
written to and stored on the drives.  If a cpu overheats and starts
writing crap to the filesystems, the raid subsystem is just going to
write the same crap to all drives, parity drives included.

RAID is also NOT a substitue for regular, proper backups.  Don't even
think about implementing a mission critical system without some form
of backup system.  A thorough and tested disaster recovery plan is
much preferable.

What you are looking for is, at the minimum, some form of fail-over
clustering system.  Two or more machines essentially working in
parallel.  If one dies, corrupts itself, etc., it can be removed from
the cluster without loss of data or service.

--
Noah Dain

"Single failures can occur for a variety of reasons that have nothing
to do with a hardware defect, such as cosmic radiation ..." - IBM
Thinkpad R40 maintenance manual, page 25

Reply | Threaded
Open this post in threaded view
|

Re: Hardware RAID, software RAID, 3ware 9550SX, level 1 vs. level 5

Tony Godshall
In reply to this post by Joseph H. Fry

According to Joseph H. Fry,

> I am partial to software raid for one important reason.... longevity.  One
> great thing about linux is that it rarely makes something entirely
> obsolete... and even if it does, you can always download previous versions of
> your favorite distro... an array created by mdadm today will likely be
> readable by most linux distros for many years to come.
>
> this is unlike hardware which can become almost impossible to replace in a few
> months sometimes.  I've heard stories about hardware controllers that
> wouldn't read data from arrays created by previous versions of the same model
> controller due to an updated bios, and it was almost impossible to get the
> previous version from the manufacturer.
>
> As far as the best redundancy... I like raid 50.  Raid 50 is essentially a
> raid 5 array that is mirrored to another raid 5 array... this protects from
> multi-drive failures as long as one of the raid 5 arrays only suffers a
> single drive failure... if both have multiple failures you may be able to ...

Personally, I prefer Google's approach (I was at an event on
this very topic event at their Mountain View campus yesterday).  
They don't try to achieve fault-tolerance in a single system-
They achieve their incredible reliability and performance by
spreading load redundantly over many hosts.  It's RAIH, not RAID.  

That's my 2 cents!

Tony


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Hardware RAID, software RAID, 3ware 9550SX, level 1 vs. level 5

Neil Dugan-3
In reply to this post by David Gaudine
David Gaudine wrote:

> I have to set up a system that is totally reliable w.r.t. data
> integrity.  That is, if a disk (or anything else) fails, it's OK if the
> system is down for a few hours, but when it comes back up it has to be
> exactly as it was, i.e. I can't restore from the previous day's backup.  
> The obvious solution is to use RAID level 1.  Questions;
>
> - Is level 1 as reliable as level 5?  My understanding is that level 5
> has better performance than level 1, but comparable reliability (maybe a
> bit less, since disaster occurs if 2 out of 3 disks go bad at the same
> time.)

That's my understanding, RAID1 is more wasteful though (you loose 1/2 of
your storage).

>
> - I need to use RAID for everything, not just some partitions, so the
> root has to use RAID.  I've found lots of websites that describe doing
> that with software (mdadm), but very little about hardware.  Using
> software is cheap and seems to be simple, but I assume hardware gives
> better performance.  Is the performance difference significant?  If not,
> I guess my further questions don't matter.  But I'd rather spend money
> than give up performance.

To use software RAID comes down to expected load on the computer (i.e.
do you have spare CPU power to use on it).

I haven't setup a hardware RAID, but my understanding of it is that as
it is done by hardware the software (i.e. Linux) doesn't know or care if
it is a RAID system it just uses it as if it is an ordinary drive.


>
> - Many/most modern motherboards claim to support RAID.  Are any of them
> useful, or do they all just provide a BIOS that fakes it well enough for
> Windows but not for Linux?

Most of these are a software raid implemented in the BOIS, and from my
reading not recommended to be used, mdadm is much more sophisticated.

>
> - The 3ware 9550SX looks good, and there's even a Sarge install:
>      http://www.3ware.com/KB/article.aspx?id=14860
> Has anyone tried the installer?  I can't unless I buy the card (and a
> computer to put it in) first.  I don't know whether it will do the whole
> job of allowing me to properly set up everyting on RAID at installation
> time, although that seems like the point of having it in an ISO instead
> of just having a driver module.
>
> - Any other ideas about a RAID board that's supported by a Sarge install
> or for which there are step-by-step instructions how now to install
> Sarge and then move it to RAID as is frequently done for mdadm?
>
> David
>
>

I am using a software RAID with 4 drives, 2 PATA drives and 2 SATA
drives, all 200GB, all partitioned with 3 partitions, 1 small, 1 medium,
1 large.

The PATA drive are each on a separate controller, so if on of those fail
it can't stop the other PATA drive, which is possible if both where on
the same cable.

I used a software raid1 over 2 small partitions of the drives for the
/boot partition, as my Linux can only boot from a RAID1 partition, and
used the other two small partitions as a parallel swap partitions.

I used a software raid5 on all the medium partitions for the Linux
install, and a software raid5 on the large partitions as a network wide
storage.

This has worked fine for me.




--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Hardware RAID, software RAID, 3ware 9550SX, level 1 vs. level 5

Joseph H. Fry
In reply to this post by Oliver Lupton
On Friday 27 January 2006 5:44 pm, Oliver Lupton wrote:

> On Fri, 27 Jan 2006 23:33:54 +0100
>
> Stephan Seitz <[hidden email]> wrote:
> > Hm, are you sure? Mirroring is RAID1. RAID10 means, that you have one
> > RAID0 array mirrored at another one. Since you need three disks for a
> > RAID5, I thought RAID50 would mean a RAID5 array over at least three
> > RAID0 arrays (each with at least two disks, so RAID50 should need at
> > least 6 disks).
>
> As I understand it, and wikipedia agrees with me, RAID 50 is the same as
> RAID 5+0. Or a RAID 0 array of RAID 5 arrays. I *think* that the term
> specifically means a RAID 0 or RAID 5's rather than vice versa.
>
> Cheers,
>
> -ol

I stand corrected... brain fart.

The raid level I meant to suggest is called 51... it is essentially two raid 5
arrays combined in raid 1 form.  I don't know if this is achievable in pure
software or if it would require two raid5 hardware controllers and software
raid1.  There is also raid 6 that will protect from multiple drive failures
at a higher utilization... unfortunately this is unachievable in software as
of yet due to the massive number of XOR operations used to achieve the
redundancy... and the controllers that support it are not cheap.
--
Joe


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]