Sign in with
Sign up | Sign in
Your question

RAID 0 for Video Editing

Last response: in Storage
Share
March 16, 2008 7:53:44 PM

For reference, I'm working with Sony Vegas, Final Cut Pro, Adobe After Effects, and Autodesk Maya. Q6600 (3.4Ghz), 8GB RAM, deciding on a motherboard with RAID (Gigabyte or ASUS most likely, nothing much above $150).

So I think I've situated myself for a RAID 0 setup down the road. I'm looking at doing 7200.11 32MB drives for all the drives. 250GB for the OS and apps, 2x250GB for raw video files, 250GB for rendered video, and 500GB for raw backup and rendered backup. I will not be utilizing all the space so 500GB is plenty. If I ever do get into HD, I'll be re-organizing the system.

The other idea I had was 500GB for OS and rendered video, 3x250GB for video, and 500GB for backup. Or 250GB for OS, 4x250GB (Raid 0+1) files. I'm thinking the last setup is overkill for a RAID that only contains source material. I like the idea of combining a RAID setup with a (read drive --> write drive) setup. I'm not sure as to how much of an increase I will get with a 3x250 setup vs a 2x250 setup? I work almost exclusively with uncompressed footage. I do keep all of the analog source material, but it takes days to recapture. I obviously want the easiest to rebuild array but seeing as how I have my OS on a separate drive, I won't run in to much of a fuss. I'm also planning on getting a 16GB flash drive so I can carry some of my work and backup, most important to me, my project files.

Let me know what you guys think! I'm looking to maximize my video editing as well as some UT3 modding.

More about : raid video editing

March 16, 2008 9:29:00 PM

Ok first off I understand you are capturing uncompressed analog footage... which depending on what codec you're using it could be in a range of 13MB per sec upto a theoretical maximum of 26MB per sec. I don't know why anyone would want to capture uncompressed analog these days but to each their own.

Capturing DV would put you at a rate of 3.5 MB per second and cut out alot of serious overhead on your editing rig. In video editing I'm sure you're well aware of GIGO... but for those who don't know it stands for Garbage In Garbage Out. The cleaner you keep the footage or less compressed if you will... will give you a better picture.

However this day in age most people, like 99% of them wouldn't know the difference from analog to dv to vhs to beta max etc. Capturing uncompressed analog in todays world is really old school like by a good 12 yrs at least. Not to pick on you, just saying most people would not notice the difference from DV to Analog provided all the other factors are accounted for primarily lighting. If you capture uncompressed with poor lighting it won't matter in the edit bay because there's only so much you can do with a poorly lit clip.

You could get an even cleaner picture by buying a pro level camera that has a 3 chip ccd capture in DV and no one would be the wiser. This just goes back to the GIGO argument above. A 1 chip ccd camera can only get so much information on tape. If you're moving to Uncompressed HD then you're talking an entirely different arena and I think a raid system would be of better use.

An average hard drive these days can easily sustain around 60MB per sec is an estimated guess. So capturing one stream uncompressed would be no problem typically. Where you could run into trouble is in editing and trying to run multiple streams uncompressed. If you're not doing this then a raid setup would really be a waste IMHO.

Every little bit would not hurt, but there are many more things to consider than just raw speed of the Raid set up. What are you going to do to keep heat to a minimum? If you neglect this aspect then you'll have hard drives failing on you or the system will simply crash all the time and you'll get nothing done other than pull your hair out.

The more complexity you add to your system in dealing with video editing the more chances you will have trouble. You could be dollars ahead to go to a 3 chip ccd DV camera and capture to DV @ 3.5 MB per sec. With HD on the scene these cameras should be alot less expensive now. To the naked eye this footage could rival HD in certain aspects if all your other ducks are in a row... lighting, audio, effects, etc.

MB = Mega Byte Mb = Megabit 8 Mb's = 1 MB so don't get these confused... as alot of disk manufacturer's use the Mb term to make their drive look faster. :pt1cable:  Same goes for GB and Gb.


Good Luck :sol: 


March 16, 2008 9:48:19 PM

When I said analog, I actually meant to say physical source, as in DV tapes. I wanted to separate the word source, from the hard drive source, from the DV tapes source, but definitely used the wrong word :whistle:  I see myself getting a Canon HV20 down the road. What I'm working with 99% of the time now, is digitally created material. I'm specifically working with a Halo 3 montage at the moment. I mix High Def and SD material in these projects since there is a lot of digital effects going into the footage. There is a chance, down the road, that I may get an HD capture card for capturing live HD video (which cannot be captured any other way).

My biggest concern when working with video editing is getting as close to real-time as possible. I know it is impossible to get real-time in every aspect of video editing. When I work with my project the only thing I want slowing down my work flow, is the ideas generated by my brain. In the regards that this has to RAID, I am simply trying to open up another bottle neck to my editing experience. I guess I haven't completely got my head around the impact RAID has on editing, but in which parts of editing does RAID reduce the time it takes to do something? Instant drag and drop of material would be nice, along with uninterrupted playback of source material. Basically I want the computer to run like my brain, where everything just seems to flow great, with no waiting, and complete control :) 

As for cooling, I do have a 120mm intake fan pulling in air, directly in front of the drives. It is a Cooler Master RC690, which has tons of room for fans. I was considering a full tower, but $100 more for a larger case isn't something I exactly wanted. If the 120mm is not enough, I will definitely consider going the full tower route.
Related resources
March 17, 2008 3:40:33 PM

ok I'm not sure what you mean by physical source. DV is more or less already compressed. It is a compressed format. To compress it further would be to encode it to any variety of other formats IE: mpeg2, mpeg1, wmv, mov etc. Each one has different varieties of compression.

If you are capturing digital sources then I am not sure what you mean by capturing uncompressed analog. There are a couple of flavors of this at least and the capture rate can vary between 13MB/sec to around 26MB/sec. To me capturing uncompressed analog is really a waste of computer resources these days.

Actually HD footage can be captured via firewire through some sony camera I can't remember the name of it off hand but it came out a few years ago and it uses proprietory codec of sony. It also can record DV format as well. Adobe Premiere Pro actually came out with support for the codec. I think part of this idea was to cut out the overhead associated with such high capture rates and it might have been an "offline" codec now that I'm thinking about it.

Realtime [RT] is a misnomer more or less. I can understand why you would want that. The best way is a 3rd party hardware card that speeds up encoding and rendering of effects/ transitions or whatever the case may be. The reason I say Realtime is a misnomer is that I have a RealTime Dv hardware card. I can do about 4 streams in RT provided I don't stack many effects. If you stack alot of effects and such then you'll start dropping frames and have to render and this is with DV @ 3.5 MB/sec.

There are better cards now where I could probably get this up to 6 streams RT DV with a new computer and just basic hard drive system SATA2 harddrives etc. There are HD cards as well but about the best you can expect is 2 RT streams with light effects...more or less where we were with DV pre 2000 time frame.

Your idea of Raid isn't correct. If you have no trouble capturing video now the time it takes to do this will not improve with raid. Back in the day that I had raid it was to make one large disk [80 GB] I put 4 20 giggers together.
It was somewhat about speed because when the 20GB disk would fill to around 80% sustained data rates would start to drop and you'd get stuttering. Capturing DV was never a problem just the length of the capture more or less.

Being that size of disk is no longer an issue speed is really the only thing that raid can offer. But the speed is only incrementally faster unless you go to fibre channel which is prohibitively expensive. Here in the next 10yrs we should see flash hard drives take over... when that happens. That will be the cat's meow for video editors.

So to put this into perspective the only time Raid is really going to have an effect on what you do is when you are stacking streams. Like say you have 4 graphic layers and 2 video layers that last maybe 10 sec. or whatever. The raid will help deliver this bandwidth. But if you are using DV this bandwidth is only 21MB/sec [3.5x6] which is easily sustainable by any modern hard drive today. So all the excess bandwidth Raid delivers is just sitting there idle and wasted. But the heat generation of the 4 disk Raid array is still there.

Now if you go to HD the equation changes because the data rate is so much higher depending on the format you use for editing [ IE... native or offline files]. If cost was no object I would tell you to go SCSI Raid and build a seperate Raid box to keep the heat out of the main machine. If cost is an object and you still want the raid go with SATA2 disks in a seperate box from the computer. Keeping that heat out of the main computer will keep it much more stable. I really don't like the idea of fans... you put so many fans in and then next all you're doing is replacing burnt up fans and the noise is deafening. If you ever tried to tape a voice over you'd see what I mean. :heink: 

But realize Raid is only incrementally faster.

I am not familar with that camera you mention... so I'll search that in the meantime.
March 17, 2008 5:12:37 PM

That Canon HV20 is a consumer/prosumer HD camcorder. It's definitely low on my priority list.

I'll try to be as specific as I can, about what I am editing.
http://www.youtube.com/watch?v=FvObKbh6l7c

Just an example of a Halo 2 montage. The HD capture comes in with Halo 3 and a Blackmagic Intesity Pro. This allows uncompressed capture of a component analog video source (720p in my case). This, as well, is low on my priority list. I have no problems in areas of capturing, so all things aside, I just want to focus on the computer itself.

Typically I have, on my computer, 30GB of footage at 3.5 MBps along with some HD sounds, graphics, and motions. In all, I'll have a good 70GB of uncompressed source material that would go on the RAID setup.

BTW, those 3CCD camcorders I no good unless the chip size is greater than 1/4.7", and even that is pushing it. I would limit it to 1/4", but those tend to get expensive. Panasonic were really the first to introduce 3CCD chips, but only their high end consumer models seemed to benefit (G400 and G500).

That HV20 uses a 1/2.7" CMOS chip, and still gets realistic colors, and as always, the low light performance is great. Considering that it is such a cheap camcorder, the low light performance is great for the crowd that doesn't have the resources or know-how in establishing good lighting themselves. I'm just interested in seeing how cheap these will get. Hopefully, manufactures won't accidentally trip into overrated pool of hard drive based camcorders. They still need a lot of work, seeing as their compression hasn't gotten better since they first came out. I guess they are OK for sub $500 camcorders because the quality is bad enough that you can't tell the difference.[/OnMyHorse]
March 18, 2008 12:15:16 PM

Ok I think you're using the word uncompressed a bit loosely here. In my mind uncompressed analog is at a minimum 13MB/s as said before this will depend on the codec you are using. All things being equal it depends on the codec.

Here's the link to a little chart that shows compressed and uncompressed formats. You will notice DV and even DV50 as well as HD-DVCPRO are all compressed formats. All data rates are in mega bits so you need to divide the number by 8 to get the MegaByte data rate. You will see that the lowest uncompressed data rate they show is just above 26MB/s. [These are 10bit HD formats]
http://www.microsoft.com/windows/windowsmedia/howto/articles/UnderstandingHDFormats.aspx#storageneedsforhdcapture

I think what you really need to do is figure out the data rate of what you're using to give you a guideline as to what you really need. You could capture source footage for 30 sec or whatever and take the size of that file and divide it by number of sec to figure your data rate, but use a fast action sequence for the clip you use for the calculation. That way you'll know without a doubt whether raid is overkill or not.

Ok I took a look at several cameras the canon as well as several sony's and I see what they are doing now. The canon looks pretty good because it records to tape the codec should work with your edit software. On the Sony side they have on the low end sub $1000 recording to DVD and Hard-Drive. The sony is unique on the hard drive front as you wouldn't have to spend money on tapes, worry about the heads getting dirty etc.

Also capturing is painless as you just transfer the footage via USB. However, on these models the codecs used aren't that good on the lower end models. You would need to spend at least $1400 to get a decent recording codec. [Sony HDR-SR12] Additionally this has the capability of taking 10.2 MP stills as well.

The sony that I referred to in an earlier post is the HDR-FX1 and has the 3 ccd image sensors... but I'm sure is well out of your price range at $3700 clams.

Anyway it seems for the video editor you're going to have to be really choosy going down the road. I'm not sure how many different formats Premiere and Vegas will support as this seems to be the differentiating factor from the upper end cameras is the codec and the cmos image sensor.

As far as the argument about 3 ccd cameras it's in the eye of who's looking at it. Everyone has an opinion of this and it's usually because this option cost more money. In the end game these are all just tools and don't edit for you or find the perfect shot. You could use the most expensive equipment money can buy and if you can't edit your way out of a paper bag it does you no good anyway.

I just refer to them because it's what the big boys use... and we all want to be big boys one day. lol.
November 6, 2008 2:18:45 PM

Hi there I just caught this thread because it's pretty relevant to where I am now. I myself am looking for a good external hard drive to store my HD footage that I caught from the Canon HV20 which someone above is considering getting. Well it's pretty relevant because Ive edited with the Canon HV20 already at my university and we used a very basic setup. It's just a normal IMac like this one but actually the previous version: http://store.apple.com/uk/browse/home/shop_mac/family/i.... And the external hardrive we use is the old Lacie D2 about 500GB. In all intents and purposes the university didnt set up the system for HD editing yet it still works. To capture the footage into the IMac I just used a Firewire and connected it to the Camera, it captured fine. I also used 25fps/24fps Cine and yes its pushing the system quite a bit since rendering the footage can take quite long especially if lots of effects are used. It can get pretty slow, but just so that its known, it does work and Ive been working with it for the last 3months. Without giving a too subjective opinion of the Canon HV20, it has standard options to record in 50fps and 25 fps, but with a little trickery you can make it capture in 25fps progressive, which gives some quite amazing images. I didnt have a big budget so at the time it was all I could afford. However the manual controls on it are pretty bare.

Which leads to my question for anyone who might have any answers, Im looking for a good realiable external hard drive to edit from with the type of footage I described above, but do I really need a RAID harddrive, its so far worked fine without one...


thanks.
!