Sign in with
Sign up | Sign in
Your question

Compression Program with SMP support

Last response: in CPUs
Share
March 13, 2006 9:32:09 AM

Hey, in midst of all the Conroe-babble i'd like to know if anyone knows of an archive/compression (winace/7-zip, etc) program that supports SMP/Dual Core CPUs?
March 13, 2006 10:56:42 PM

it's pretty sad if there isn't one, it would be obvious to recode one for SMP

I need to compress 13710 files in a total of 90gb, so it would be handy :p 
March 13, 2006 11:25:04 PM

You know, I don't think there is a mainstream one. What many consider to be the best, WinRAR, doesn't even do SMP.

I'm not sure why. The advantages would be obvious. From what I've seen high clock speed makes the most difference in WinRAR. Dual cores seem to have no effect on compression/decompression times.

I agree... it is sad.

-mpjesse
Related resources
March 25, 2006 3:51:43 PM

Thanx for your reply, i hope they'll address this issue. it is something that everyone would love to see, it's every single day that that a person extracts or compresses something, well basically anyway.

Since i didn't want to wait for 24hr+, i ended up selecting 'store' method, which means basically no compression at all using winace which is my preferred program. Since it was mp3's the compression factor would have been 98-99% so the it was rather pointless to give up the extra time.

take care, Mopey
March 25, 2006 6:30:52 PM

You could have separated the data into 2 directories and ran a zip or tar + gzip or bzip2 on each.

On Linux at least one tar + bzip2 would use CPU0 and the other CPU1.
March 28, 2006 9:38:39 PM

knoppix wouldn't even install on my pretty standard dothan 1.733ghz w. 1gb of ram from september with the recent version of knoppix of that time, so i've had my share of linux for now. though it looks great and work fast, but seem complex with a fully edged commandline interface in midst of the gui, and i hate the fact that you have to have a seperate partition for the swapfile alone and that it can't write to ntfs

let's not even get into the lack of gamesupport and full hardware acceleration on the gui.

i have a lot of respect for linux, but it just isn't my game
March 28, 2006 9:43:04 PM

Quote:
knoppix wouldn't even install on my pretty standard dothan 1.733ghz w. 1gb of ram from september with the recent version of knoppix of that time, so i've had my share of linux for now. though it looks great and work fast, but seem complex with a fully edged commandline interface in midst of the gui, and i hate the fact that you have to have a seperate partition for the swapfile alone and that it can't write to ntfs

let's not even get into the lack of gamesupport and full hardware acceleration on the gui.

i have a lot of respect for linux, but it just isn't my game


It can write to NTFS. I loaded Suse on an old box and the NTFS partition on the second drive I had in the system came up as a windows folder in root. Games are about the only thing keeping me from switching. Maybe OSX will go mainstream someday.
March 29, 2006 1:48:42 AM

Quote:
knoppix wouldn't even install on my pretty standard dothan 1.733ghz w. 1gb of ram from september with the recent version of knoppix of that time, so i've had my share of linux for now. though it looks great and work fast, but seem complex with a fully edged commandline interface in midst of the gui, and i hate the fact that you have to have a seperate partition for the swapfile alone and that it can't write to ntfs

let's not even get into the lack of gamesupport and full hardware acceleration on the gui.

i have a lot of respect for linux, but it just isn't my game



Knoppix is a live CD it doesn't need to be installed. You do not have to use a swap partition Linux can swap on the filesystem or you can turn swap off completely if you want.

Knoppix has NTFS read and write support out of the box.

nVidia and ATI both distribute accelerated drivers for Linux.
March 29, 2006 11:04:28 PM

I wonder how well that would work decompressing? I think I'll try it when I get home.

It's kind of a waste of time though- it would take longer to tell the frigin OS which instance of WinRAR to use than just decompressing something...

Unless it were really big... like a 4.7GB DVD.
March 29, 2006 11:21:30 PM

Quote:
I wonder how well that would work decompressing? I think I'll try it when I get home.

It's kind of a waste of time though- it would take longer to tell the frigin OS which instance of WinRAR to use than just decompressing something...

Unless it were really big... like a 4.7GB DVD.




On Linux the kernel does that for you automatically.

If you have 2 or more CPUs one compress or decompress will be assigned to CPU0 and the other to CPU1, etc.

If you have 4 CPUs you can run 4 bzip2's or gzip's at the same time.

:-D
March 29, 2006 11:52:26 PM

I hope Vista figures that out 'cause you've got to set affinity in WinXP.

Pain in the arse...
March 29, 2006 11:58:14 PM

Quote:
I hope Vista figures that out 'cause you've got to set affinity in WinXP.

Pain in the arse...



Who knows... they are going to re-write 60% of it...
March 30, 2006 10:51:00 AM

must be an Zip or rar program that supports SMP

the setting affinity thing it make No dif if its set to 1 cpu or 2 cpus as its single treaded so it will only used 50% allways (if 2 cpus) (it use 100% on one or 50% on both)

90gb of files i just copy that to a other disk insted of useing an compresson program what if the zip rar file gets broken
March 30, 2006 4:41:43 PM

thats the one some one talked about before

i test it on my X2 3800+ duley later on
March 30, 2006 6:25:28 PM

There's more to it than that. Compression (compress or decompress operations) acts on the entire contents of a serial stream (i.e. file) which must be managed by a single thread of execution. The act of generating compression on a file depends on the entire content of that file. Effectively, you can't break a file down into multiple threaded pieces and recombine at the end of the operation. A program that would start multiple threads for multiple file operations could benefit, but you would need more intimate konwledge of the system. The ability to force starting thread 1 on a CPU, and then thread 2 on a different CPU (you can't trust the OS to do this for you) which isn't as easy as it sounds. There are challenges to which the solution would really provide less bang for the buck than optimizing the algorithms would. The 7zip algorithim is similar to above, so the claim might just be marketing gimmickry (they don't _not_ support SMT or SMP), or they are doing something with multiple files. I would bet it's not a 2x (or even 1.5x) improvement in time though.
March 30, 2006 10:15:23 PM

Blah.. i was tired and miswrote. I meant to say that i tried Ubuntu!! not knoppix, don't really see the point of running my os on a 4-7mb/s device
March 30, 2006 10:17:59 PM

7-zip addded that now? Sweet, i'll def. check it out.. but for some reason i think i will be solely disappointed, let's see how it goes

EDIT: okay, i checked it out, they surely did improve the program. The load was roughly 70-90% as well as extremely high dictionary support

seems to be working pretty damn fast, tnx
March 31, 2006 12:27:06 AM

Quote:
Blah.. i was tired and miswrote. I meant to say that i tried Ubuntu!! not knoppix, don't really see the point of running my os on a 4-7mb/s device



Knoppix works amazingly well.

Installable distros like FC5 and ubuntu work even better :-D
April 4, 2006 1:37:18 PM

lol

i did however, like that the made a linux os that would fit a floppy.. and one that fit a 128mb memorystick, that's just showoff ;) 
!