Compression Program with SMP support

mopeygoth · Mar 13, 2006

Hey, in midst of all the Conroe-babble i'd like to know if anyone knows of an archive/compression (winace/7-zip, etc) program that supports SMP/Dual Core CPUs?

mopeygoth · Mar 14, 2006

it's pretty sad if there isn't one, it would be obvious to recode one for SMP

I need to compress 13710 files in a total of 90gb, so it would be handy

mpjesse · Mar 14, 2006

You know, I don't think there is a mainstream one. What many consider to be the best, WinRAR, doesn't even do SMP.

I'm not sure why. The advantages would be obvious. From what I've seen high clock speed makes the most difference in WinRAR. Dual cores seem to have no effect on compression/decompression times.

I agree... it is sad.

-mpjesse

mopeygoth · Mar 25, 2006

Thanx for your reply, i hope they'll address this issue. it is something that everyone would love to see, it's every single day that that a person extracts or compresses something, well basically anyway.

Since i didn't want to wait for 24hr+, i ended up selecting 'store' method, which means basically no compression at all using winace which is my preferred program. Since it was mp3's the compression factor would have been 98-99% so the it was rather pointless to give up the extra time.

take care, Mopey

linux_0 · Mar 25, 2006

You could have separated the data into 2 directories and ran a zip or tar + gzip or bzip2 on each.

On Linux at least one tar + bzip2 would use CPU0 and the other CPU1.

mopeygoth · Mar 29, 2006

knoppix wouldn't even install on my pretty standard dothan 1.733ghz w. 1gb of ram from september with the recent version of knoppix of that time, so i've had my share of linux for now. though it looks great and work fast, but seem complex with a fully edged commandline interface in midst of the gui, and i hate the fact that you have to have a seperate partition for the swapfile alone and that it can't write to ntfs

let's not even get into the lack of gamesupport and full hardware acceleration on the gui.

i have a lot of respect for linux, but it just isn't my game

gpfear · Mar 29, 2006

knoppix wouldn't even install on my pretty standard dothan 1.733ghz w. 1gb of ram from september with the recent version of knoppix of that time, so i've had my share of linux for now. though it looks great and work fast, but seem complex with a fully edged commandline interface in midst of the gui, and i hate the fact that you have to have a seperate partition for the swapfile alone and that it can't write to ntfs

let's not even get into the lack of gamesupport and full hardware acceleration on the gui.

i have a lot of respect for linux, but it just isn't my game

It can write to NTFS. I loaded Suse on an old box and the NTFS partition on the second drive I had in the system came up as a windows folder in root. Games are about the only thing keeping me from switching. Maybe OSX will go mainstream someday.

linux_0 · Mar 29, 2006

knoppix wouldn't even install on my pretty standard dothan 1.733ghz w. 1gb of ram from september with the recent version of knoppix of that time, so i've had my share of linux for now. though it looks great and work fast, but seem complex with a fully edged commandline interface in midst of the gui, and i hate the fact that you have to have a seperate partition for the swapfile alone and that it can't write to ntfs

let's not even get into the lack of gamesupport and full hardware acceleration on the gui.

i have a lot of respect for linux, but it just isn't my game

Knoppix is a live CD it doesn't need to be installed. You do not have to use a swap partition Linux can swap on the filesystem or you can turn swap off completely if you want.

Knoppix has NTFS read and write support out of the box.

nVidia and ATI both distribute accelerated drivers for Linux.

mpjesse · Mar 30, 2006

I wonder how well that would work decompressing? I think I'll try it when I get home.

It's kind of a waste of time though- it would take longer to tell the frigin OS which instance of WinRAR to use than just decompressing something...

Unless it were really big... like a 4.7GB DVD.

linux_0 · Mar 30, 2006

I wonder how well that would work decompressing? I think I'll try it when I get home.

It's kind of a waste of time though- it would take longer to tell the frigin OS which instance of WinRAR to use than just decompressing something...

Unless it were really big... like a 4.7GB DVD.

On Linux the kernel does that for you automatically.

If you have 2 or more CPUs one compress or decompress will be assigned to CPU0 and the other to CPU1, etc.

If you have 4 CPUs you can run 4 bzip2's or gzip's at the same time.

:-D

mpjesse · Mar 30, 2006

I hope Vista figures that out 'cause you've got to set affinity in WinXP.

Pain in the arse...

linux_0 · Mar 30, 2006

I hope Vista figures that out 'cause you've got to set affinity in WinXP.

Pain in the arse...

Who knows... they are going to re-write 60% of it...

leexgx · Mar 30, 2006

must be an Zip or rar program that supports SMP

the setting affinity thing it make No dif if its set to 1 cpu or 2 cpus as its single treaded so it will only used 50% allways (if 2 cpus) (it use 100% on one or 50% on both)

90gb of files i just copy that to a other disk insted of useing an compresson program what if the zip rar file gets broken

leexgx · Mar 30, 2006

thats the one some one talked about before

i test it on my X2 3800+ duley later on

putzin · Mar 30, 2006

There's more to it than that. Compression (compress or decompress operations) acts on the entire contents of a serial stream (i.e. file) which must be managed by a single thread of execution. The act of generating compression on a file depends on the entire content of that file. Effectively, you can't break a file down into multiple threaded pieces and recombine at the end of the operation. A program that would start multiple threads for multiple file operations could benefit, but you would need more intimate konwledge of the system. The ability to force starting thread 1 on a CPU, and then thread 2 on a different CPU (you can't trust the OS to do this for you) which isn't as easy as it sounds. There are challenges to which the solution would really provide less bang for the buck than optimizing the algorithms would. The 7zip algorithim is similar to above, so the claim might just be marketing gimmickry (they don't _not_ support SMT or SMP), or they are doing something with multiple files. I would bet it's not a 2x (or even 1.5x) improvement in time though.

mopeygoth · Mar 31, 2006

Blah.. i was tired and miswrote. I meant to say that i tried Ubuntu!! not knoppix, don't really see the point of running my os on a 4-7mb/s device

mopeygoth · Mar 31, 2006

7-zip addded that now? Sweet, i'll def. check it out.. but for some reason i think i will be solely disappointed, let's see how it goes

EDIT: okay, i checked it out, they surely did improve the program. The load was roughly 70-90% as well as extremely high dictionary support

seems to be working pretty damn fast, tnx

linux_0 · Mar 31, 2006

Blah.. i was tired and miswrote. I meant to say that i tried Ubuntu!! not knoppix, don't really see the point of running my os on a 4-7mb/s device

Knoppix works amazingly well.

Installable distros like FC5 and ubuntu work even better :-D

mopeygoth · Apr 4, 2006

lol

i did however, like that the made a linux os that would fit a floppy.. and one that fit a 128mb memorystick, that's just showoff

Compression Program with SMP support

Distinguished

Distinguished

Splendid

Distinguished

Splendid

Distinguished

Distinguished

Splendid

Splendid

Splendid

Splendid

Splendid

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Splendid

Distinguished

Share this page