Any way to compare files from 2 directories?
Friday, August 4th, 2006 - 10:19am
Somewhere copying about 65GB of mp3's from an external HD to a NAS drive the copying died on some illegal file..
Now this is the second of 2 external drives filled with MP3's is there an easy way to see what files from the external made it to the NAS or do I just start over again?
Lots of duplicate artists folders exist.. just with different albums under them.
Pain in the ass
Aint nothing new about the world order..it's been playing since the day they put George Washington on a quarter
85's face the truth you're too dumb.



Same tree structure? I use SyncBackSE to compare and copy up to terabyte volumes. It's really cheap for home use to: $25 for a 5 computer license. The free demo will do what you need too.
Vice Versa is another one I've successfully used in production but SyncBackSE has more active development and the developers have an active forum.
Live: LiquidmantisGWJ | PSN: LiquidmantisGWJ
Web: Mantis on the Mountain
--
Women can't be in the same room with me without abandoning men forever - rabbit
Get a copy of windiff.exe. Open up a command prompt and navigate to the directory you have your collection in (ie. "My Music").
Type "dir *.* /s > drive1.txt"
Now type "dir X:\*.* /s >drive2.txt" (X being the external drive letter)
Then type "windiff drive1.txt drive2.txt"
That should bring up a directory structure list between both drives and compare them to each other. I'm sure someone else can explain the process better than I can though.
Any diff tool will work (I use the one from http://unxutils.sourceforge.net/ since I'm on a Windows machine).
diff -qr dir1 dir2
should output something like:
Only in dir2: a.txt
Only in dir1: b.txt
Sure these diff tools will help but that's still a lot of hands on interaction and should it fail again then you have to start over. The sync tools I mentioned are all-in-one diff/copy (either using time & date stamp or CRC comparison) utilities with shadow copy support and are absolutely painless to use.
Live: LiquidmantisGWJ | PSN: LiquidmantisGWJ
Web: Mantis on the Mountain
--
Women can't be in the same room with me without abandoning men forever - rabbit
I guess I'm confused, are you trying to consolidate the artist folders while copying? If not and you're just doing a straight up copy then read on.
I run two scripts, one uses Robocopy to do the actual copy. There are switches to copy just changed files after the original copy for use when doing a migration from one server to another and other neat stuff (like retry to copy failed files). I have it dump the results to a log file that looks like this.
I then run a script that uses Diruse.exe (a resource kit tool) that spits out a .csv with just summary info of the source and destination sizes of the directories I back up. Using the logs from the robocopy I can find any files that might be off.
Let me know if you want copies of the scripts.
Do you ever walk alone like a drifter in the dark?
Let me be a bit clearer..
I have 2 external Hard drives filled with MP3 (and FLAC's but thats another story) I'm trying to consolidate the 2 external drives into a single 1TB NAS drive.
I copied the contents of one drives MP3 folder which looks like this /artist name/album name/song title.mp3 into the NAS no problem..
The second external drive during the copy failed somewhere.. in my bleary pre Latte phase I didnt write down where it failed.
since there are many of the same artists on both drives its confusing to exactly figure out where it left off and failed.
Aint nothing new about the world order..it's been playing since the day they put George Washington on a quarter
85's face the truth you're too dumb.
http://www.myspace.com/armyofthepharaohs
In that case I would start a robocopy from the second drive to the nas.
I use these options with the xp/2k3 version (XP010) of robocopy:
/E - copies all sub directories, even if empty.
/NP - No progress indicator.
/FFT - Assumes FAT, helps with some file types on NTFS volumes.
/TEE - Displays to the output window and/or log.
/COPY:DATSOU - For NTFS volumes copies all the attributes, ACLs, auditing, ownership and fun stuff like that.
R:1 - Number of retries on failed copies, the default is 1 MILLION.
W:1 - Wait time in seconds to wait between retries, the default is 30 seconds.
/LOG+:\\server\share\file.log - The log location. I actually use a variable for the log location since it's dynamically named based on date/time.
If you log the copy process when done you can open it in your favorite editor (or notepad) and search for fail and see what files failed. The thing is the copy won't stop on a failure like with the 'normal' copy command so you're not wasting time restarting everytime you find an F'd up file.
Do you ever walk alone like a drifter in the dark?
doh.. I cant get by this error
Invalid Parameter #1 : "G:\The Associated Group MP3\MP3\*.*
Thats my source drive.. copying to a network mapped drive Y:
thats the correct path.. but for some reason Robocopy keeps barfing.
Aint nothing new about the world order..it's been playing since the day they put George Washington on a quarter
85's face the truth you're too dumb.
http://www.myspace.com/armyofthepharaohs
Do you have "'s around the entire path?
"G:\The Associated Group MP3\MP3\*.*"
They're usually required when you have folder names with spaces. Or you could just rename the folder without the spaces for the copy, then rename it back when it's done.
Mr T broke the speed of light in the A-Team van because he wanted to prove that quantum physics was a bunch of Jibba Jabba.
welp.. I tried g:\mp3\mp3\*.*
no " this time.. and I still get
Invalid Parameter #1 : "G:\MP3\MP3\*.*"
Aint nothing new about the world order..it's been playing since the day they put George Washington on a quarter
85's face the truth you're too dumb.
http://www.myspace.com/armyofthepharaohs
What's the full command line you're using?
Mr T broke the speed of light in the A-Team van because he wanted to prove that quantum physics was a bunch of Jibba Jabba.
Leave off the *.*
Edit1 - Hang on, error reading my script give a few minutes to review...
Edit2 - Nope, I'm not insane. Leave the \*.* off your source and destination directories, by default it does *.*.
Do you ever walk alone like a drifter in the dark?
Here is my script:
My Shares.txt file just has 1 line per top level folder on the source drive that I want to back up, in my case:
Users
Shares
You can see what the command passed to the CMD Line is by changing the Robocopy line to this:
And that will spit out something like this:
Keep in mind these are single lines but have been wrapped by the page.
Do you ever walk alone like a drifter in the dark?
yeah it was the *.*
ok its moving since last night
Aint nothing new about the world order..it's been playing since the day they put George Washington on a quarter
85's face the truth you're too dumb.
http://www.myspace.com/armyofthepharaohs