Xcopy script, only copy a file once?

Catsrules · Sep 12, 2011

I need to make a scrip that will only copy a file once. Then if that file is deleted from the destination and the script is run again, it will ignore that file, and move on to the next.

Any Ideas?

mibix19 · Sep 12, 2011

Hi. Have you looked at all the DOS commands The For command may give you what you want . In the Dos prompt type, help for and it will describe what this command does.(leave a space between help & for

Catsrules · Sep 12, 2011

mibix19 :

Thanks for the quick response, I have looked at all of the switches I could use for xcopy and info on what they do found here
http://www.computerhope.com/xcopyhlp.htm
The only one that looks like it would be helpful is the /EXCLUDE. But I would need it to update it self every time it ran. So if I could get a log created with the file names it copied then the next time it runs it looks at the log file and adds then names to the exclude list.
Any ideas how this could be done?

riser · Sep 12, 2011

Can you explain this better? What kind of files are being copied, why they're getting deleted, and why a new file would be copied? Is it going to copy multiple files each time, or a single file each time?

I'm kind of curious on this because there are easy ways to do it but it would depend on what you're really trying to achieve.

MrBig55 · Sep 12, 2011

For XCOPY to exclude files not present in the destination, use the /U switch.

I use ROBOCOPY instead of XCOPY. It works in all Windows from Windows XP to the latest, but some functions are not present when using Windows XP. If you want to use a script for both Vista/7 and XP, then type ROBOCOPY /? (or XCOPY /?) in both XP and Vista/7 and look for the switches that are not there in XP, then only use the switches that both OSes supports. Here is an example of one line of my backup script:

robocopy "C:\ProgramData\FarmFrenzy3" "C:\GreaterBox\Backup2011\ProgramData\FarmFrenzy3" /e /b /copyall /R:3 /x /v /ts /fp /np /log+:"C:\NovaCleaner\Logs\RobocopyLog-%DATE%.log"

It does backup the specified file/folder to the specified location, then creates a very detailed .log file with today's date in the filename. Type ROBOCOPY /? to see every switches that you can use and there uses.

I think that /XX switch (for ROBOCOPY) is the one your looking for if you wanna exclude all files/folders not present in the destination folder.

Catsrules · Sep 12, 2011

riser :

Yeah sure,
My ultimate goal is to convert my media center recording to a mp4 format so I can take them with me, on like a itouch or something. My problem is my HTPC is not very powerful and would take forever to convert a show. So I need to copy them to a more powerful computer. I have a file server that should do the job and being a file server is it on 24/7 so I figure might as well take advantage of that.
I have a program what will convert my show then delete the original file, so I want to make a scrip that every night will copy all of my TV shows to my server then the server will convert them and delete the files that where copied over. That should work just fine the first night. But the next night my scrip will copy that same files again as well as new ones. And they will all converted a second time. With in a few days it will get so backup with just reconversion that is will be useless.

I was thinking maybe just moving the files instead of copying them. But other family members use the HTPC as well and like that watch the recorded shows as well. So I don't want to mess with the normal operations of the HTPC.

Catsrules · Sep 12, 2011

MrBig55 :

Yeah I was look at that switch, but I don't think it will work because there will be new files added every few days. And I think they will get skipped because they are not in the destination.

I will have to give ROBOCOPY a look I haven't heard of it before!

nordlead · Sep 12, 2011

Why not just leave all the content on the file server.

So, move all the new TV shows into an input folder on the server. Once converted they get placed in their final location on the file server. The HTPC points to the file server for its media library.

No fancy copy scheme required and everything is in a central location.

cadder · Sep 12, 2011

Catsrules :

xcopy and robocopy both have switches that will copy files newer than a certain date. I forget which is which but for one of them you set the actual date, for the other one you tell it to copy files newer than x number of days. This switch might let you copy only the files that are 1 day old. Depending on when you run the copy it might get only today's files or it might want to get today's and yesterday's. You would have to experiment with it a bit.

Catsrules · Sep 12, 2011

nordlead :

Sadly media center doesn't supports this

.
I think I could get it to work, but it be like shows older then 1 day would be under the video section of MEC and stuff recorded to day would be under recorded TV section. And this would be to complex for some of my family members. They are doing good it they can figure out the right input on the TV

Catsrules · Sep 12, 2011

cadder :

Hmm that is an idea! Didn't even think about time stamps.
I think it is a /d switch if I remember right.
I will give it a look

bucknutty · Sep 12, 2011

could you xcopy the whole folder? That way if 1 file in the source folder changes the script wont stop?

Also I use MS Sync toy. It is an awsome free tool for keeping different computers, folders the same. You can set it to run at midnight, and it has a cool GUI.

nordlead · Sep 12, 2011

Catsrules :

If you are using Windows 7 Media Center you should be able to add multiple folder locations to the "Recorded TV" library. If you are using XP, I really don't know what is different about that version.

Ok, back to the original request, I do know how to pull this off using rsync and a text file.

Use rsync with the --exclude-from=FILE option. When you copy the file to the server add it to your exclusion file (one line per file). Once excluded the files will never be copied again. You could use the exclude filter to exclude directories if it is structured that way, which would reduce your exclude file size and probably speed up rsync ever so slightly.

You can install rsync on windows with cygwin, and then add cygwin to the path so your bat file can execute rsync. You can google rsync's man page for the filter rules.

MrBig55 · Sep 12, 2011

So you say that once a day:

- you need to copy media files over to your server,
- convert them to another format,
- delete these that were processed,
- then on the next day you just want to ignore the files that were processed already.

I've got an idea: instead of deleting files that were already converted, you can just create a script that will replace processed files with empty files with the same names. Then you just need the proper switch that will ignore all files present in your server, and only copy new files to it. That way you won't need to edit each day your batch file and it will work on it's own every day without any mod.

for XCOPY, use the switch /d (without any date) to ignore files that are already present in the destination and date is newer than source (which is the case if you created empty files after conversion)

for ROBOCOPY to do the same, use /xo

EDIT: oh well I don't know how to do a script that will replace all files within a folder with empty ones... guess it's not much help there... :-(

EDIT2: You can delete the content of a file with the command:

Code:

copy nul "C:\test\test.txt"

but you cannot do a command like:

Code:

copy nul "C:\test\*.*"

You can create empty files using:

Code:

type nul>"C:\test\test.txt"

but you cannot do that:

Code:

type nul>"C:\test\*.*"

You can use "forfiles" to do a command for every files it finds, but the "copy nul" command does not work within it...

Screw command prompt it's no use when you want it to be very specific...

krop · Sep 12, 2011

Catsrules :

forfiles /P c:\forfiles\ /M *.* /D -0 /C "cmd /c echo copying... @file & copy /y @file \\server\share\"

Take this line above and copy it to notepad and save it with a .bat extension. Schedule it once a day and you should be good to go. This will copy the files you specify on a specific directory c:\forfiles /Mask *.mp4 and only for the current day (-0). You would need to run the script with a user account that has network access to the \\server\share.

Good luck and you are welcome ;-)

Catsrules · Sep 13, 2011

@nordlead
Your right sorry about that, I must have been thinking of MEC for Vista. I still would prefer to keep the files local if at all possible. (In case the server goes down or something)
I will have to give rsync a look as well thanks.
@MrBig55
interesting idea. Assuming that the server completed it task in under 24 hours, all I would need to do it make the first part of my copy scrip to replace all of the files on my server with the empty ones. Then start the Xcopy /d. I just need to figure out how to do that.

Edit 1:thanks for the copy nul, at least it is a step in the right direction. I also tried copy nul *.wtv but it didn't work either.

@cadder and krop
Thank for the code krop. BTW I had to change the /D -0 to /d:0 to get the date thing working.
But I found a problem. It is using the modified date and not the created date. And it looks like every time you watch a show it changes the modified date to the time you watch the show. grrrr... Any ideas of how to change this? Or copy files based on an attribute?

Catsrules · Sep 13, 2011

Had another idea. Using the Archive attribute and just do a xcopy /M. It has the same problem as the date. When I play the file with media center it modifies it and turns the Archive attribute back on

Catsrules · Sep 13, 2011

Hey I think I figured it out!!!
If anyone wants to know here is how to do it.

You need to make a txt file to list all of your excluded files. I am using Exclude.txt as my file.

[cpp]net use "\\server\recorded tv"
xcopy "C:\Users\Public\Recorded TV\*.wtv" "\\server\recorded tv\" /d /y /EXCLUDE:Exclude.txt >> "C:\Scripts\Exclude.txt"[/cpp]

The first time it runs the txt is empty so it will copy everything (except folders). when it runs it add all of the files it copied to the exclude.txt and when you run it again it will exclude all of the files in the exclude.txt. So all of the files unless you added a new one.
The only way to get it to copy it again is to delete if from the exclude list or change the file's name.

So far it has just be working in a test environment I still need to set a few things up in the working environment before I can give it a try.

MrBig55 · Sep 13, 2011

This is good to know ^^

Catsrules · Sep 13, 2011

Yeah the only problem I can find with this, is that it adds the file to the txt when it starts copying. So if for some reason the copy fails, it will skip the file on any future copies.

Hmm it might be better to do it this way.
[cpp] net use "\\server\recorded tv"
xcopy "C:\Users\Public\Recorded TV\*.wtv" "\\server\recorded tv\" /y /EXCLUDE:Exclude.txt
xcopy "C:\Users\Public\Recorded TV\*.wtv" "\\server\recorded tv\" /L /y /EXCLUDE:Exclude.txt >> "C:\Scripts\Exclude.txt"[/cpp]

This way the copy will completely finish before anything gets added. So if something goes wrong power outage network loss, or the copy get canceled the next time will do the entire copy again.
It still can easy get messed up if the copy fails but the list works. It will mess up all the files that where being copied files instead of just one

MrBig55 · Sep 13, 2011

I just tried your command but it tells me that "Exclude.txt" couldn't be read. After some debugging I found out this error comes from the /EXCLUDE switch. Even running command prompt as admin doesn't help at all. I even created the file but it still won't read it.

Also you copy the whole files twice with xcopy which is time consuming and suppose that command prompt does crash, you will still be left over with a corrupted file so you did not solve that issue yet.

Another issue: that command will only write "n files copied" into the Exclude.txt file.

Catsrules · Sep 13, 2011

MrBig55 :

The Exclude.txt needs to be in the same location with scrip file or in the location your command prompt is in. You can also do something like /Exclude:C:\Scripts\Exclude.txt You can't use "" they don't work for some reason so you can't have spacing in the path.
I have used this on a 2003 server and Win 7 Home prem both have worked fine.

The first xcopy command just copies the files, the seconds xcopy commands has a /L switch that only displays the files that will be copied and doesn't copy them. But the list goes to the txt file so it adds to the excludes.
I think with just the /y switch it will override the files in the destination, so if the copy fails the next copy will override the corrupted files.

MrBig55 :

What do you mean?

MrBig55 · Sep 13, 2011

Oh didn't know that the /L switch ONLY showed files to be copied over but actually doesn't copy those. When I did my tests, I used these switches: /e /i /q /k /y /j. Well that explains why you wrote the xcopy function twice. ^^

You can also do something like /Exclude:C:\Scripts\Exclude.txt You can't use "" they don't work for some reason so you can't have spacing in the path.

writing /Exclude:C: gives me an error: bad analyse (my system is in french so I guess this is the error in plain english) so I have to put double-quotes (even tho there is no space in the actual path), but it then gives me the error: couldn't read from file.

Also I did this command:

Code:

xcopy "C:\GreaterBox\test" "C:\GreaterBox\test2" /y >> "C:\GreaterBox\testLOG\LOG.txt"

Looking in the LOG.txt file I only see: "3 files copied" but no filenames listed there. Running the script again makes the LOG.txt file to be:

3 files copied
3 files copied

Catsrules · Sep 13, 2011

MrBig55 :

I guess you are just going to have to keep the script file and the txt file in the same location so you can just say /exclude:textfilename.txt

MrBig55 :

Also I did this command:

Code:

xcopy "C:\GreaterBox\test" "C:\GreaterBox\test2" /y >> "C:\GreaterBox\testLOG\LOG.txt"

Looking in the LOG.txt file I only see: "3 files copied" but no filenames listed there. Running the script again makes the LOG.txt file to be:

3 files copied
3 files copied

Hmm weird. if you remove the >> "C:\greaterBox\testLOG\LOG.txt" and then run it does it display the files it copied in the command prompt like this
http://dl.dropbox.com/u/1754769/Internet%20PIx/xcopy1.jpg
make sure you don't use the /q command or elce it will not display the files at all.
http://dl.dropbox.com/u/1754769/Internet%20PIx/xcopy.jpg

You need to have that text displayed for this to work, that is what is inputted into the text file

MrBig55 · Sep 13, 2011

Well you're right, using /q switch does not only hide copied files from command prompt window, but also hides it in the log file as well...

Also I tried using these commands as a script (have been trying as a single command in the command prompt so far) and it worked. The /EXCLUDE switch does not give any error this time around.

All these issues are retarded if you ask me ^^

So your code is perfect but I can still think of a "potential" problem. This needs a bit of explaination:

Say you run your script, it does start to copy files over to your server. Meanwhile, the media center records additionnal videos. The program might miss them, but when it will do a list of files in the "Recorded TV" folder on the media center pc, it WILL list even those newly recorded videos, even tho it did not copy them.

I know that I'm searching for bugs, but it MAY happen if the file copy is not done when media center pc records a new video.

Else your program is strong man ^^ I will use the /EXCLUDE switch in some programs since it's very convenient :-D

Xcopy script, only copy a file once?

Distinguished

Distinguished

Distinguished

Illustrious

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Distinguished

Share this page