I want to build a 1.5TB storage array for MythTV

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

Recently ran into the account of a guy who built his own 1.2TB
RAID50-based storage array for $1600 (see
<URL:http://www.finnie.org/terabyte/>). I really like the idea and
have been thinking about following suit.

Like Finnie, I want to be able to store huge amounts of DivX/Xvid
files online. In addition to the storage array, I also plan to build a
separate MythTV (<URL:http://www.mythtv.org>) box, which among other
things will let me play them at will. My 200GB Series 1 TiVo's been
serving me well for more than four years, but I really like the idea
of being able to seamlessly integrate my AVI collection with TV
recordings, and from what I gather MythTV has finally matured enough
to be a realistic TiVo alternative.

I have been 100% Linux at home for almost a decade and am quite
comfortable with most of the technical aspects of the project.

I'm planning on making the following changes to Finnie's build
configuration:

* Instead of 200GB ATA, use 250GB SATA drives for a total of
1.5TB. Outpost.com offers a Western Digital 250GB SATA drive for
$170 (<URL:http://shop1.outpost.com/product/3868597>). I just missed
the chance to get a $30 rebate off each drive, but I'm sure
Fatwallet will alert me to a similar opportunity sooner or later.
* Accordingly, get a HighPoint SATA RAID card instead of the specified
RocketRAID 454 ATA RAID card. I think the RocketRAID 1640
(<URL:http://www.newegg.com/app/SearchProductResult.asp?Submit=Go&DEPA=0>)
is the way to go.
* Instead of ext3, use XFS as the file system.

My questions:
* If I connect the storage array to my Linksys WRT54G router, will
100Mbps Ethernet be fast enough to pump the AVI files to the MythTV
box without dropping frames?
* Conversely, will 100Mbps Ethernet be sufficient to let me use the
storage array as the primary storage medium for MythTV's recordings?
What about HDTV encodings (using the pcHDTV Linux-only card)? Or do
I have to upgrade to a Gigabit Ethernet router? Or would the encoder
card and MythTV software have to run on the storage array itself in
order to achieve acceptable performance? (Actually, I'm not opposed
to doing so, if one box can simultaneously handle both storage and
MythTV tasks.)
* Anything else that I'm missing or should keep in mind?

--
Read my Deep Thoughts @ <URL:http://www.ylee.org/blog/> PERTH ----> *
Cpu(s): 8.9% us, 5.3% sy, 85.4% ni, 0.0% id, 0.0% wa, 0.3% hi, 0.0% si
Mem: 516960k total, 514056k used, 2904k free, 9212k buffers
Swap: 2101032k total, 989384k used, 1111648k free, 135456k cached

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

Yeechang Lee wrote:

> Recently ran into the account of a guy who built his own 1.2TB
> RAID50-based storage array for $1600 (see
> <URL:http://www.finnie.org/terabyte/>). I really like the idea and
> have been thinking about following suit.
>
> Like Finnie, I want to be able to store huge amounts of DivX/Xvid
> files online. In addition to the storage array, I also plan to build a
> separate MythTV (<URL:http://www.mythtv.org>) box, which among other
> things will let me play them at will. My 200GB Series 1 TiVo's been
> serving me well for more than four years, but I really like the idea
> of being able to seamlessly integrate my AVI collection with TV
> recordings, and from what I gather MythTV has finally matured enough
> to be a realistic TiVo alternative.
>
> I have been 100% Linux at home for almost a decade and am quite
> comfortable with most of the technical aspects of the project.
>
> I'm planning on making the following changes to Finnie's build
> configuration:
>
> * Instead of 200GB ATA, use 250GB SATA drives for a total of
> 1.5TB. Outpost.com offers a Western Digital 250GB SATA drive for
> $170 (<URL:http://shop1.outpost.com/product/3868597>). I just missed
> the chance to get a $30 rebate off each drive, but I'm sure
> Fatwallet will alert me to a similar opportunity sooner or later.

You might want to check the CompUSA site--they were having a deal for 200
gig drives for about 90 bucks counting a mail-in rebate, but I don't know
if it's still on.

> * Accordingly, get a HighPoint SATA RAID card instead of the specified
> RocketRAID 454 ATA RAID card. I think the RocketRAID 1640

Personally for a RAID that size I'd go for a 3Ware or LSI Logic. No
particular reason, just that I'm used to terabytes being mainframe
territory and I get nervous with consumer RAID controllers trying to handle
that much data.

> (<URL:http://www.newegg.com/app/SearchProductResult.asp?Submit=Go&DEPA=0>)
> is the way to go.
> * Instead of ext3, use XFS as the file system.
>
> My questions:
> * If I connect the storage array to my Linksys WRT54G router, will
> 100Mbps Ethernet be fast enough to pump the AVI files to the MythTV
> box without dropping frames?

Yes.

> * Conversely, will 100Mbps Ethernet be sufficient to let me use the
> storage array as the primary storage medium for MythTV's recordings?

Yes.

> What about HDTV encodings (using the pcHDTV Linux-only card)?

Yes.

> Or do
> I have to upgrade to a Gigabit Ethernet router?

No.

> Or would the encoder
> card and MythTV software have to run on the storage array itself in
> order to achieve acceptable performance? (Actually, I'm not opposed
> to doing so, if one box can simultaneously handle both storage and
> MythTV tasks.)
> * Anything else that I'm missing or should keep in mind?

HDTV is carried on two adjacent 6 MHz channels, for an aggregate of 12
analog MHz. I don't recall the actual bit rate but it's not very high.
There's no reason that any other kind of TV would need more than that
unless you're storing files in an uncompressed or lossless-compressed form.

--
--John
Reply to jclarke at ae tee tee global dot net
(was jclarke at eye bee em dot net)

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

Yeechang Lee wrote:
> Recently ran into the account of a guy who built his own 1.2TB
> RAID50-based storage array for $1600 (see
> <URL:http://www.finnie.org/terabyte/>). I really like the idea and
> have been thinking about following suit.
>
> Like Finnie, I want to be able to store huge amounts of DivX/Xvid
> files online. In addition to the storage array, I also plan to build a
> separate MythTV (<URL:http://www.mythtv.org>) box, which among other
> things will let me play them at will. My 200GB Series 1 TiVo's been
> serving me well for more than four years, but I really like the idea
> of being able to seamlessly integrate my AVI collection with TV
> recordings, and from what I gather MythTV has finally matured enough
> to be a realistic TiVo alternative.
>
> I have been 100% Linux at home for almost a decade and am quite
> comfortable with most of the technical aspects of the project.
>
> I'm planning on making the following changes to Finnie's build
> configuration:
>
> * Instead of 200GB ATA, use 250GB SATA drives for a total of
> 1.5TB. Outpost.com offers a Western Digital 250GB SATA drive for
> $170 (<URL:http://shop1.outpost.com/product/3868597>). I just missed
> the chance to get a $30 rebate off each drive, but I'm sure
> Fatwallet will alert me to a similar opportunity sooner or later.
> * Accordingly, get a HighPoint SATA RAID card instead of the specified
> RocketRAID 454 ATA RAID card. I think the RocketRAID 1640
> (<URL:http://www.newegg.com/app/SearchProductResult.asp?Submit=Go&DEPA=0>)
> is the way to go.
> * Instead of ext3, use XFS as the file system.
>
> My questions:
> * If I connect the storage array to my Linksys WRT54G router, will
> 100Mbps Ethernet be fast enough to pump the AVI files to the MythTV
> box without dropping frames?
> * Conversely, will 100Mbps Ethernet be sufficient to let me use the
> storage array as the primary storage medium for MythTV's recordings?
> What about HDTV encodings (using the pcHDTV Linux-only card)? Or do
> I have to upgrade to a Gigabit Ethernet router? Or would the encoder
> card and MythTV software have to run on the storage array itself in
> order to achieve acceptable performance? (Actually, I'm not opposed
> to doing so, if one box can simultaneously handle both storage and
> MythTV tasks.)
> * Anything else that I'm missing or should keep in mind?
>

I don't know much about MythTV, but I did some testing with Gigabit
Ethernet. The results are here:

http://somacon.com/docs/gigabitnas.html

Highlights:------------------------
TEST: Repeated disk-to-disk 150 MB file transfer via FTP.
Connected via Gigabit Switch to Athlon XP 2500+.

P3 @ 500 Mhz, tg3 driver = 27 MB/sec

P3 @ 850 Mhz, tg3 driver = 33 MB/sec

P3 @ 850 Mhz, smc95x2 driver = 36 MB/sec

P3 @ 1000 Mhz, smc95x2 driver = 36 MB/sec

---------------------------------------

With Gigabit Ethernet, maximum transfer speed is affected by the CPU
speed. The CPU burden is likely introduced by the TCP stack and
network driver. By comparison, current hardware will easily top out a
100Mbit network, with a rate of 11 MB/sec.

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

Yeechang Lee wrote:


> * Instead of 200GB ATA, use 250GB SATA drives for a total of
> 1.5TB. Outpost.com offers a Western Digital 250GB SATA drive for
> $170 (<URL:http://shop1.outpost.com/product/3868597>). I just missed
> the chance to get a $30 rebate off each drive, but I'm sure
> Fatwallet will alert me to a similar opportunity sooner or later.

Generally rebates are limited to one per SKU per household. Be sure to
check the details.


> * Anything else that I'm missing or should keep in mind?

I'm doing a cheap-o scaled down version of what you describe. As soon
as my drives come in I'll let you know. I have a feeling that the
latency will be increased when seeking through a recording, but other
than that might be OK.


-WD

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

On 4 Aug 2004 17:48:15 GMT, Yeechang Lee <ylee@pobox.com> wrote:

>* If I connect the storage array to my Linksys WRT54G router, will
> 100Mbps Ethernet be fast enough to pump the AVI files to the MythTV
> box without dropping frames?
>* Conversely, will 100Mbps Ethernet be sufficient to let me use the
> storage array as the primary storage medium for MythTV's recordings?
> What about HDTV encodings (using the pcHDTV Linux-only card)? Or do
> I have to upgrade to a Gigabit Ethernet router? Or would the encoder
> card and MythTV software have to run on the storage array itself in
> order to achieve acceptable performance? (Actually, I'm not opposed
> to doing so, if one box can simultaneously handle both storage and
> MythTV tasks.)

Seems to me your architecture has just one primary client system (the
MythTV box). So why not stuff an extra gigabit NIC in the MythTV box
and the storage box and connect them with a cross-over cable? A quick
peak suggests that gigabit NICs will cost from about $30...

(Keep the regular network for everything else, of course).

When/if you want to expand the gigabit network, just add a switch...

Malc.

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

J. Clarke (jclarke@nospam.invalid) wrote in alt.video.ptv.tivo:
> HDTV is carried on two adjacent 6 MHz channels, for an aggregate of 12
> analog MHz.

No, HDTV (or any form of ATSC digital TV) in the US uses just one 6MHz
channel.

> I don't recall the actual bit rate but it's not very high.

19.3Mbps at most. That's 2.5MB/sec. It's easily carried in real time
over 100Mbps Ethernet.

--
Jeff Rife |
SPAM bait: | http://www.nabs.net/Cartoons/RhymesWithOrange/ReadyForADog.jpg
AskDOJ@usdoj.gov |
spam@ftc.gov |

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

Will Dormann wrote:
> I have a feeling that the latency will be increased when seeking
> through a recording, but other than that might be OK.
>

That's something I hadn't even considered. On my TiVo seeks are
lightining-fast; I can't believe any network-based system will be
quite so quick. (On the other hand, even over a wireless link and
Samba I've found seeks using mplayer to be more or less acceptable, so
a storage array on much newer equipment might not be so bad at all.) I
wonder if MythTV has the ability to locally cache recordings?

--
Read my Deep Thoughts @ <URL:http://www.ylee.org/blog/> PERTH ----> *
Cpu(s): 8.9% us, 5.3% sy, 85.5% ni, 0.0% id, 0.0% wa, 0.3% hi, 0.0% si
Mem: 516960k total, 513536k used, 3424k free, 12744k buffers
Swap: 2101032k total, 989264k used, 1111768k free, 118932k cached

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

Yeechang Lee wrote:

> I
> wonder if MythTV has the ability to locally cache recordings?

Not that I'm aware of. I haven't looked into it too much, but I think
it would be nice to be able to record to and watch from the local disk
but after a certain amount of time the recordings would get offloaded to
the NAS for storage.

I know there is a built-in option to transcode files after an amount of
time, but I wonder if it would be possible to have that same function
just do a simple transfer without re-encoding.

I'm sure once I get my NAS set up, I'll be looking deeper into it!


-WD

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

Jeff Rife wrote:

> J. Clarke (jclarke@nospam.invalid) wrote in alt.video.ptv.tivo:
>> HDTV is carried on two adjacent 6 MHz channels, for an aggregate of 12
>> analog MHz.
>
> No, HDTV (or any form of ATSC digital TV) in the US uses just one 6MHz
> channel.

You're right--I don't know where I got the two adjacent channels--maybe that
was something proposed early on.

>> I don't recall the actual bit rate but it's not very high.
>
> 19.3Mbps at most. That's 2.5MB/sec. It's easily carried in real time
> over 100Mbps Ethernet.
>

--
--John
Reply to jclarke at ae tee tee global dot net
(was jclarke at eye bee em dot net)

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

Malcolm Weir wrote:

> On 4 Aug 2004 17:48:15 GMT, Yeechang Lee <ylee@pobox.com> wrote:
>
>>* If I connect the storage array to my Linksys WRT54G router, will
>> 100Mbps Ethernet be fast enough to pump the AVI files to the MythTV
>> box without dropping frames?
>>* Conversely, will 100Mbps Ethernet be sufficient to let me use the
>> storage array as the primary storage medium for MythTV's recordings?
>> What about HDTV encodings (using the pcHDTV Linux-only card)? Or do
>> I have to upgrade to a Gigabit Ethernet router? Or would the encoder
>> card and MythTV software have to run on the storage array itself in
>> order to achieve acceptable performance? (Actually, I'm not opposed
>> to doing so, if one box can simultaneously handle both storage and
>> MythTV tasks.)
>
> Seems to me your architecture has just one primary client system (the
> MythTV box). So why not stuff an extra gigabit NIC in the MythTV box
> and the storage box and connect them with a cross-over cable? A quick
> peak suggests that gigabit NICs will cost from about $30...

Why bother?

> (Keep the regular network for everything else, of course).
>
> When/if you want to expand the gigabit network, just add a switch...
>
> Malc.

--
--John
Reply to jclarke at ae tee tee global dot net
(was jclarke at eye bee em dot net)

 

[Wolf]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

"Shailesh Humbad" <noreply@nowhere.com> wrote in message
news:OR9Qc.115875$fv.4318@fe2.columbus.rr.com...
> Yeechang Lee wrote:
>
> I don't know much about MythTV, but I did some testing with Gigabit
> Ethernet. The results are here:
>
> http://somacon.com/docs/gigabitnas.html
>
> Highlights:------------------------
> TEST: Repeated disk-to-disk 150 MB file transfer via FTP.
> Connected via Gigabit Switch to Athlon XP 2500+.
>
> P3 @ 500 Mhz, tg3 driver = 27 MB/sec
>
> P3 @ 850 Mhz, tg3 driver = 33 MB/sec
>
> P3 @ 850 Mhz, smc95x2 driver = 36 MB/sec
>
> P3 @ 1000 Mhz, smc95x2 driver = 36 MB/sec
>
> ---------------------------------------
>
> With Gigabit Ethernet, maximum transfer speed is affected by the CPU
> speed. The CPU burden is likely introduced by the TCP stack and
> network driver. By comparison, current hardware will easily top out a
> 100Mbit network, with a rate of 11 MB/sec.

Depending on your workload, it is affected by so many things: CPU,
memory, backplane, drivers, even the file system and how NFS (in
our tests) is configured.

Using dual 2.4 Xeons to drive 2 3ware cards and two disk arrays
of 8 in a 6-1-1 raid 5 config and raid 1 the two arrays (~2.8TB)
with the 2.6.7 linux kernel using XFS I am up to ~55MB/sec in
our prelim testing. We got a little better read speed with JFS but
XFS wrote faster.

We have done thinks from changing NFS parameters to
tweaking the intel ethernet drivers:
http://support.intel.com/support/network/sb/CS-009209.htm
to changing memory buffers. I am doing some research now
on tweaking flush buffers.

You can get rather obsesses with this.

--
Angel R. Rivera aka Wolf
----------------------------------------------------------------
Please post all reponses to UseNet. All email cheerfully and automagically
routed to Dave Null

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

J. Clarke wrote:

> HDTV is carried on two adjacent 6 MHz channels, for an aggregate of 12
> analog MHz. I don't recall the actual bit rate but it's not very high.

It's one channel, not two. http://www.atsc.org/document_map/video.htm
-Joe

ATSC Standard A/53: Digital Television Standard
5. SYSTEM OVERVIEW

The Digital Television Standard describes a system designed to transmit
high quality video and audio and ancillary data over a single 6 MHz
channel. The system can deliver reliably about 19 Mbps of throughput in
a 6 MHz terrestrial broadcasting channel (8VSB encoding) and about 38 Mbps
of throughput in a 6 MHz cable television channel.

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

On Thu, 05 Aug 2004 02:09:45 -0400, "J. Clarke"
<jclarke@nospam.invalid> wrote:

>Malcolm Weir wrote:

>>>* If I connect the storage array to my Linksys WRT54G router, will
>>> 100Mbps Ethernet be fast enough to pump the AVI files to the MythTV
>>> box without dropping frames?
>>>* Conversely, will 100Mbps Ethernet be sufficient to let me use the
>>> storage array as the primary storage medium for MythTV's recordings?
>>> What about HDTV encodings (using the pcHDTV Linux-only card)? Or do
>>> I have to upgrade to a Gigabit Ethernet router? Or would the encoder
>>> card and MythTV software have to run on the storage array itself in
>>> order to achieve acceptable performance? (Actually, I'm not opposed
>>> to doing so, if one box can simultaneously handle both storage and
>>> MythTV tasks.)
>>
>> Seems to me your architecture has just one primary client system (the
>> MythTV box). So why not stuff an extra gigabit NIC in the MythTV box
>> and the storage box and connect them with a cross-over cable? A quick
>> peak suggests that gigabit NICs will cost from about $30...
>
>Why bother?

'cos that way you don't *need* to know what else is going on.

The recommendation was, and remains, one that will address a stated
concern. It is, likely, overkill, but at $30 per card, who cares?

Now, if you want to claim that you *know* the amount of traffic on the
existing 100BaseT network, and therefore you *know* that there is
sufficient excess bandwidth for the new application, be my guest...
but I'd be curious *how* you know that!

Malc.

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

In article <MPG.1b7b6b5214d22d8998976e@news.nabs.net>,
wevsr@nabs.net says...
> J. Clarke (jclarke@nospam.invalid) wrote in alt.video.ptv.tivo:
> > HDTV is carried on two adjacent 6 MHz channels, for an aggregate of 12
> > analog MHz.
>
> No, HDTV (or any form of ATSC digital TV) in the US uses just one 6MHz
> channel.
>
> > I don't recall the actual bit rate but it's not very high.
>
> 19.3Mbps at most. That's 2.5MB/sec. It's easily carried in real time
> over 100Mbps Ethernet.

And if my math skills haven't failed me... around 8.5 or
8.75 GB/hr. (Call it 9 for planning purposes.)

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage (More info?)

Malcolm Weir wrote:

> On Thu, 05 Aug 2004 02:09:45 -0400, "J. Clarke"
> <jclarke@nospam.invalid> wrote:
>
>>Malcolm Weir wrote:
>
>>>>* If I connect the storage array to my Linksys WRT54G router, will
>>>> 100Mbps Ethernet be fast enough to pump the AVI files to the MythTV
>>>> box without dropping frames?
>>>>* Conversely, will 100Mbps Ethernet be sufficient to let me use the
>>>> storage array as the primary storage medium for MythTV's recordings?
>>>> What about HDTV encodings (using the pcHDTV Linux-only card)? Or do
>>>> I have to upgrade to a Gigabit Ethernet router? Or would the encoder
>>>> card and MythTV software have to run on the storage array itself in
>>>> order to achieve acceptable performance? (Actually, I'm not opposed
>>>> to doing so, if one box can simultaneously handle both storage and
>>>> MythTV tasks.)
>>>
>>> Seems to me your architecture has just one primary client system (the
>>> MythTV box). So why not stuff an extra gigabit NIC in the MythTV box
>>> and the storage box and connect them with a cross-over cable? A quick
>>> peak suggests that gigabit NICs will cost from about $30...
>>
>>Why bother?
>
> 'cos that way you don't *need* to know what else is going on.
>
> The recommendation was, and remains, one that will address a stated
> concern. It is, likely, overkill, but at $30 per card, who cares?
>
> Now, if you want to claim that you *know* the amount of traffic on the
> existing 100BaseT network, and therefore you *know* that there is
> sufficient excess bandwidth for the new application, be my guest...
> but I'd be curious *how* you know that!

So how much traffic does a network that somebody has in his house usually
have?

> Malc.

--
--John
Reply to jclarke at ae tee tee global dot net
(was jclarke at eye bee em dot net)

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

Shailesh Humbad <noreply@nowhere.com> writes:

>> * Anything else that I'm missing or should keep in mind?
>>
>
>I don't know much about MythTV, but I did some testing with Gigabit
>Ethernet. The results are here:
>
>http://somacon.com/docs/gigabitnas.html
>

>
>With Gigabit Ethernet, maximum transfer speed is affected by the CPU

Not exactly. Consider that maximum full-duplex throughput of a gigabit ethernet
card is 200 mega_bytes_ per second. Put this on a standard 33Mhz PCI
bus (132 megabytes/second peak), and you can see that the speed of the
CPU doesn't matter from the standpoint of getting maximum throughput,
especially considering you may be using some of that available PCI
bandwidth for disk I/O.

FTP is hardly a useful benchmark for determining the peak ethernet
performance anyway. Try with something lower overhead like rcp.

>speed. The CPU burden is likely introduced by the TCP stack and
>network driver. By comparison, current hardware will easily top out a
>100Mbit network, with a rate of 11 MB/sec.

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

Scott Lurndal wrote:

> FTP is hardly a useful benchmark for determining the peak ethernet
> performance anyway. Try with something lower overhead like rcp.

Huh? Once you've gotten past the dialog that starts the transfer,
ftp has no level 3 overhead: it sends just data bytes until EOF.

-Joe

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

"J. Clarke" <jclarke@nospam.invalid> wrote in message news:<cerdg502d9g@news4.newsguy.com>...
> You might want to check the CompUSA site--they were having a deal for 200
> gig drives for about 90 bucks counting a mail-in rebate, but I don't know
> if it's still on.

The last time I saw one of these rebates, it was limited to one per
household unfortunately.

> > * Accordingly, get a HighPoint SATA RAID card instead of the specified
> > RocketRAID 454 ATA RAID card. I think the RocketRAID 1640
>
> Personally for a RAID that size I'd go for a 3Ware or LSI Logic. No
> particular reason, just that I'm used to terabytes being mainframe
> territory and I get nervous with consumer RAID controllers trying to handle
> that much data.

I think, from the link he mentioned, that he intends to use the Linux
md driver. (software RAID). Most of those cheap RAID cards are not
hardware RAID at all. They advertise it falsely and give you a Windows
driver that does software RAID.

Nothing is worse than cheap RAID. If you're not using something solid,
even if it does really do hardware RAID, your best bet is software
RAID. The Linux software RAID is better-tested and more widely used
(by people who will notice if something went wrong and get it fixed or
at least complain) than any of the cheapo RAID or fakeraid cards.

Bigger issues than the size of the RAID group are things like, what
happens if a drive fails? This can be very hard to test without
special HD firmware made for this purpose. I've seen cheap hardware
RAIDs where yanking the drive out seems to fail it fine, but when a
drive fails for real, the errors get passed through to the OS.

Or how does the RAID behave when running into bad blocks on a "good"
drive, while resyncing onto a hot-spare or a replaced disk. I've seen
even hugely expensive "enterprise" solutions fail miserably here.

Or how does the RAID behave when it sees a double-disk failure? Even
some high-end solutions give you no way to recover. Obviously if both
drives have really failed, you're screwed. But what if a power
connection gets knocked loose?

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

(carloschoenberg@yahoo.com) wrote in alt.video.ptv.tivo:
> Or how does the RAID behave when it sees a double-disk failure? Even
> some high-end solutions give you no way to recover. Obviously if both
> drives have really failed, you're screwed. But what if a power
> connection gets knocked loose?

I had this exact situation with Linux software RAID-5 (2.6 kernel), and
a simple remove and re-add of the drives caused the re-build to start, and
I lost no data.

I'm trying to figure out how to get a spare onto the array to avoid even
this sort of problem, but although the software can do it, I don't have any
more room in the case.

--
Jeff Rife |
SPAM bait: | http://www.nabs.net/Cartoons/UserFriendly/GeekCommunication.gif
AskDOJ@usdoj.gov |
spam@ftc.gov |

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

carloschoenberg@yahoo.com wrote:
> The last time I saw one of these rebates, it was limited to one per
> household unfortunately.

You may be right; hopefully I will be able to find a bona fide price
cut soon. As mentioned, even $170 (the pre-rebate price) isn't bad.

> I think, from the link he mentioned, that he intends to use the Linux
> md driver. (software RAID). Most of those cheap RAID cards are not
> hardware RAID at all.

I definitely get the sense through research that modern Linux software
RAID is superior to the consumer-grade RAID cards. That said, if I
follow the approach outlined at <URL:http://www.finnie.org/terabyte/>,
I will be using both the RcoketRAID card's RAID 5 *and* software RAID
0. But if Finnie is mistaken and just software RAID is sufficient and
more reliable, I'd go that way.

> Or how does the RAID behave when it sees a double-disk failure? Even
> some high-end solutions give you no way to recover. Obviously if
> both drives have really failed, you're screwed. But what if a power
> connection gets knocked loose?

Reliability is important, of course, but my desire has limits. Since
the array would be used to hold video files, of course I want some
degree of redundancy (as a RAID 50 arrangement apparently
provides). That said, video files aren't *so* important as to
necessitate in my mind extraordinary redundancy; that's why I'm
willing to take the risk of two disks going bad at once, and why I'm
not even going to bother trying to backup a 1.5TB array.

--
Read my Deep Thoughts @ <URL:http://www.ylee.org/blog/> PERTH ----> *
Cpu(s): 8.7% us, 5.2% sy, 85.7% ni, 0.0% id, 0.0% wa, 0.3% hi, 0.0% si
Mem: 516960k total, 512916k used, 4044k free, 14452k buffers
Swap: 2101032k total, 1070004k used, 1031028k free, 135152k cached

 

[Wolf]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

"Jeff Rife" <wevsr@nabs.net> wrote in message
news:MPG.1b7f1307e221ad7698977e@news.nabs.net...
> (carloschoenberg@yahoo.com) wrote in alt.video.ptv.tivo:
> > Or how does the RAID behave when it sees a double-disk failure? Even
> > some high-end solutions give you no way to recover. Obviously if both
> > drives have really failed, you're screwed. But what if a power
> > connection gets knocked loose?
>
> I had this exact situation with Linux software RAID-5 (2.6 kernel), and
> a simple remove and re-add of the drives caused the re-build to start, and
> I lost no data.
>
> I'm trying to figure out how to get a spare onto the array to avoid even
> this sort of problem, but although the software can do it, I don't have
any
> more room in the case.

If you are going to do software RAID on the 2.6 kernel anyway, why not
check into RAID6(?) that is supposed to have the equiv of two parity drive?
This is something I am going to look into soon.

--
Angel R. Rivera aka Wolf
----------------------------------------------------------------
Please post all reponses to UseNet. All email cheerfully and automagically
routed to Dave Null

 

[Guest]

Archived from groups: comp.sys.ibm.pc.hardware.storage,comp.arch.storage,alt.video.ptv.tivo (More info?)

Will Dormann wrote:
> I'm currently copying
> over my MythTV recordings (very large files) via NFS and the CPU usage
> is around 60%. Almost 10% of this is from the md3_raid5 process, and
> the rest is nfsd. When copying over files from my windows machine via
> Samba, the CPU usage was around 40% I believe. I haven't done any
> special tuning or benchmarking, so I'm not sure if NFS is less efficient
> than Samba, or if it's just moving the data faster.

I'm doing some copying of similar files over Samba now, and the CPU
usage is between 60% and 70%. So it appears that NFS and Samba are
pretty close, CPU-wise.


-WD