Rapid delete and/or reformat on many partion system

[Guest]

Archived from groups: microsoft.public.windowsxp.perform_maintain (More info?)

Hello,
I'm hoping someone can offer
suggestions for the following situation.

In a real-time system we need to
create a large number of smallish
files - say 20-50k bytes and about 140,000 of
these file per hour. We need to recycle
our storage space on about a weekly
interval.

It appears that the process of deleting
files on XP can be really lengthy and
we add to that inevitable fragmentation
that will occurr as this storage is
re-used again and again.

The solution I have in mind is to partition
our drives into say 125 GB partitions with
perhaps 8-12 such partitions on a server.
The recycling would then be accomplished
by doing a quick reformat on the FS partion
holding the oldest data.

Can anyone comment on potential
pitfalls, etc with this approach?
(I already know about the pitfall of formatting
the wrong drive

Alternatively any "secrets" to making XP delete
files/directories more quickly? (I don't think
recycle bin is a factor for our system)

Cheers,
Roy

 

[galen]

Archived from groups: microsoft.public.windowsxp.perform_maintain (More info?)

In news:wuh2e.187$EE2.89@newsread2.news.pas.earthlink.net,
Roy Terry <royterry@earthlink.net> had this to say:

My reply is at the bottom of your sent message:

> Hello,
> I'm hoping someone can offer
> suggestions for the following situation.
>
> In a real-time system we need to
> create a large number of smallish
> files - say 20-50k bytes and about 140,000 of
> these file per hour. We need to recycle
> our storage space on about a weekly
> interval.
>
> It appears that the process of deleting
> files on XP can be really lengthy and
> we add to that inevitable fragmentation
> that will occurr as this storage is
> re-used again and again.
>
> The solution I have in mind is to partition
> our drives into say 125 GB partitions with
> perhaps 8-12 such partitions on a server.
> The recycling would then be accomplished
> by doing a quick reformat on the FS partion
> holding the oldest data.
>
> Can anyone comment on potential
> pitfalls, etc with this approach?
> (I already know about the pitfall of formatting
> the wrong drive
>
> Alternatively any "secrets" to making XP delete
> files/directories more quickly? (I don't think
> recycle bin is a factor for our system)
>
> Cheers,
> Roy

Two things...

1) Can you partition a drive into more than 4 partitions? I'm not sure but I
thought the limit was 4 so I thought I'd ask you instead. Chances are you
know more than I.

2) This has helped here but this is basically a standalone PC (I have a LAN
but it's my own pathetic LAN and it's actually easier for me to turn around
then it is to use NAT tools or learn more information) but I've completely
disabled said recycle bin. This MIGHT be something you can look into as well
as it seems to make deletion faster here. Here's a link for a standalone PC
but I imagine you could do what you needed with remote registry or the like:

http://www.theeldergeek.com/enable_disable_recycle_bin.htm

These are just some basic ideas that I thought you MIGHT be able to use.
Hope that they do something valuable for you even if it's only to make you
think along new lines and find alternate solutions that are more fitting for
you.

Galen
--
Signature changed for a moment of silence.
Rest well Alex and we'll see you on the other side.

 

[pop]

Archived from groups: microsoft.public.windowsxp.perform_maintain (More info?)

"Roy Terry" <royterry@earthlink.net> wrote in message
news:wuh2e.187$EE2.89@newsread2.news.pas.earthlink.net...
> Hello,
> I'm hoping someone can offer
> suggestions for the following situation.
>
> In a real-time system we need to
> create a large number of smallish
> files - say 20-50k bytes and about 140,000 of
> these file per hour. We need to recycle
> our storage space on about a weekly
> interval.
>
> It appears that the process of deleting
> files on XP can be really lengthy and
> we add to that inevitable fragmentation
> that will occurr as this storage is
> re-used again and again.
>
> The solution I have in mind is to partition
> our drives into say 125 GB partitions with
> perhaps 8-12 such partitions on a server.
> The recycling would then be accomplished
> by doing a quick reformat on the FS partion
> holding the oldest data.
>
> Can anyone comment on potential
> pitfalls, etc with this approach?
> (I already know about the pitfall of formatting
> the wrong drive
>
> Alternatively any "secrets" to making XP delete
> files/directories more quickly? (I don't think
> recycle bin is a factor for our system)
>
> Cheers,
> Roy
>
>

Wow, if I understood your post, that's a lot of disk-work. I can imagine
the amount of time it must take to housekeep that and the deletion times
would defnintely be excessive. It sounds like it's working; which is
surprising in itself at that file creation rate.

Anyway, I suspect you need to get out of the GUI world. Sincei it sounds
like you aren't into scripting and such, I'd probably advise going for batch
files. If you've ever used DOS, you know how fast it is compared to
windows: Taking XP to the equivalent, the Command mode, would give you
pretty much those same benefits. I'd think using batch files, Xcopy or
XXcopy, and judicious partition arrangements would be a decent experiment to
try. Use the windows method for your control data and compare the DOS
(command mode) operations to it to see if it'll be successful. If you have
windows, you have Xcopy; XXcopy however is better, more flexible, and freely
available on the 'net. Search Google for it. It's sort of a "super" Xcopy
with more features and uses.

IMO, partitions aren't really as important as separate physical drives.
With multiple drives and XP's hyperthreading, you could get nearly parallel
disk I/O going on, nearly halving some of the timing delays. e.g. do
deletes on two drives at a time. You can do that with partitions too, but
since there's only one set of heads per drive, you don't get that much
benefit. But with two or more drives, they can almost be operating in
parallel. Not literally, but with two head sets working at the basic same
time on two drives, it doubles the amount of work done.

Without a lot more information about the system and how/where/when data
comes in/out to where and so on, speed, RAM, drive sizes, etc. etc. etc.
it's pretty hard to be precise.

HTH

Pop
--
---
No, I won't get dressed.
I'm retired!

 

[pop]

Archived from groups: microsoft.public.windowsxp.perform_maintain (More info?)

See inline:

"Galen" <galennews@gmail.com> wrote in message
news:e0HufYNNFHA.1096@tk2msftngp13.phx.gbl...
> In news:wuh2e.187$EE2.89@newsread2.news.pas.earthlink.net,
> Roy Terry <royterry@earthlink.net> had this to say:
>
....
>
> 1) Can you partition a drive into more than 4 partitions? I'm not sure but
> I thought the limit was 4 so I thought I'd ask you instead. Chances are
> you know more than I.
Yes, but not with fdisk or anything in XP. You'd need a 3rd party program
like Partition Magic that will allow up to 26 drive letters. Some apps will
allow even more than that. The caveat however is, once you go past 4
partitions, you can no longer use fdisk; in fact, fdisk will crash the
entire drive. And fdisk can no longer "see" the drives correctly even to
show info - you'd need the 3rd party app to replace it.
>
> 2) This has helped here but this is basically a standalone PC (I have a
> LAN but it's my own pathetic LAN and it's actually easier for me to turn
> around then it is to use NAT tools or learn more information) but I've
> completely disabled said recycle bin. This MIGHT be something you can look
> into as well as it seems to make deletion faster here. Here's a link for a
> standalone PC but I imagine you could do what you needed with remote
> registry or the like:
>
> http://www.theeldergeek.com/enable_disable_recycle_bin.htm
>
> These are just some basic ideas that I thought you MIGHT be able to use.
> Hope that they do something valuable for you even if it's only to make you
> think along new lines and find alternate solutions that are more fitting
> for you.

I'm not sure "getting rid of recycle bin" does anything but stop displaying
the system icon for it. If "Recycler" remains on the drives, then it hasn't
helped anything. But, you CAN easily bypass the recycle bin, which will
save some time, from within XP. Files are automatically deleted instead of
going to the recycle bin, just like the Shift-Delete keys. I do know that
this will speed up the deletion times when a lot of files are being deleted.

Pop

 

[galen]

Archived from groups: microsoft.public.windowsxp.perform_maintain (More info?)

In news:esxiH8WOFHA.2704@TK2MSFTNGP15.phx.gbl,
Pop <nobody@devnull.spamcop.net> had this to say:

My reply is at the bottom of your sent message:

> I'm not sure "getting rid of recycle bin" does anything but stop
> displaying the system icon for it. If "Recycler" remains on the
> drives, then it hasn't helped anything. But, you CAN easily bypass
> the recycle bin, which will save some time, from within XP. Files
> are automatically deleted instead of going to the recycle bin, just
> like the Shift-Delete keys. I do know that this will speed up the
> deletion times when a lot of files are being deleted.
> Pop

AFAIK the tweak on the page doesn't just get rid of the recycle bin but
rather makes all deletes like your shift-delete process as there's no bin
left to place it to. The tweak basically amounts to doing the same thing as
opting to not move files to the recycle bin at all within the properties
context menu for the recycle bin icon.

Galen
--
Signature changed for a moment of silence.
Rest well Alex and we'll see you on the other side.