Copy and rename files recursively
 

I want to copy Windows files from a DVD, preserving directory structure, renaming them by first truncating filenames greater than 50 characters in length, and then adding a sequential number at the end of the file name (but before the extension).

I've searched the internet and found some hints, but nothing that works. I'm relatively new to Terminal, but not overly so. From the little I've read, the "Find" command seems to be the way to start. I'm just unsure where to head from there.

Thought I'd clarify. I want to copy Windows files from a DVD onto a Windows computer. I'm using my Mac to do this, as my Mac reads DVDs and the Windows computer doesn't. I want to maintain the directory structure. Some of the file names are too long for the Windows computer to accept/read, and I want to truncate those file names when copying. I thought adding a number to the end of these file names would ensure that no files are overwritten during the process, but I'm open to suggestions as to the best way to handle this.

Any suggestions?

Windows shouldn't have a problem with long filenames. The Joliet filesystem used by Windows for CDs is a lot more restrictive then Windows itself is.

And yet, it does. I don't know why.

Please excuse me for the delay in posting this. I went to search for problem filenames. Some examples of file names now:
I know that truncating the first three files will either produce an error or overwriting, and I'm thinking that appending a number to the end of files will solve that problem. I'm unconcerned about how the numbers are added to the filename. And if there's another way that works without appending numbers, I'm open to that too.

Is it just one huge directory of filenames, or is there some directory structure involved, too?

Brett

Oh boy is it nested. Hundreds of folders, easy, and of varying depths.

And it's important to duplicate the folder structure.

That is a very important part of the equation.

Have you tried the tar command?

To me that would be easiest, tar it up, and transfer the tarball to the windows machine and then untar it.

Brett

I don't use windows too much. I know that winzip can handle tar files, does the machine have winzip?

How are you transfering files from the mac to the windows machine?

Brett

I used tar, following instructions on copying files recursively and received various error messages. I've played around the commands and parameters, but haven't had any success.

I'm copying from the Mac via smb.

I was reading about rsync and cp -R, and both of those seem likely candidates. Ditto as well. Actually there seems to be a great deal of stuff for copying the files, it's the renaming part that seems to be complicating things.

Well that is just terrible advice, talk about obscure.

Do you have a file that contains all of the filename, including the path to the files?

What errors did you get using tar?

Did you try a subset of directories?

Did you try rsync?

How fast is your connection between computers?

I'm not sure that you can do this with vanilla OS X, but maybe try to mount the dvd so that it is reshareable and then share it out as a samba share? I able to put a CD in my Mac, and set up the CD as a samba share and open the share on my windows box and copy the contents over. Of course I have the Server version of OS X but you should be able to do that with the client version, too.

It would be really better if you showed us what you have tried to do and why it failed, rather than just saying it didn't work.

Again, this shouldn't be too difficult, I mean a DVD only holds a couple of Gb.
I think that in this case, you can not provide too much info.
Maybe some insight into the directory structure.

I've got to go home now, but will check the thread this evening.
Brett

Take it in steps, but pick what you want to do first.

Do you want to rename first or copy over first. I did make an assumption that you were trying to rename because you thought that was why the copy operation was failing. If you could magically copy it over without the errors, would you still want to rename?

Gotta Go.
Brett

ps - I don't think that cp -R is the way to go.

No, I don't have a file that contains all of the filenames. I have thought about using that option (redirecting find command output to a text file, reading from that file using AppleScript, renaming and copying the files to the network). I haven't tried that yet, thinking that there might be an easier way.

If I remember correctly, when I followed the above-referenced instructions which used tar as part of the command, the results of the 'find' command didn't translate into a filename tar could use. I remember using "*.*" with and without quotes, as well as just using a directory/folder for find without using filename wildcards and the "-name" option. I don't remember all the specifics of the errors I received.

I have not tried rsync yet. Nor have I tried ditto. I have played with cp -R, but from what I'm reading on the man page for cp -R, when a directory/folder is passed to it, it goes into the directory and processes the files there. If I'm using find in conjunction with cp, I think that might muck up things a bit. Yet, if I'm reading the man page correctly, using cp in conjunction with find will throw errors when find results in a directory/folder.

The connection is fairly fast.

As for your suggestion of copying the contents over a samba share (Excuse me, I know a few terms, but my geek level is 101; I assume by samba you mean smb?), I did that the first time and that resulted in problems with long filenames.

The directory structure is abominable. Let's say, easily, hundreds of folders of varying depths up to 10 levels. Maybe more in some cases.

Yes, I am thinking that renaming means that the copy operation does not fail. If I could magically copy over without errors, I would not need to rename.

"Take it in steps ..." I think that means that my dreams of a one-line Terminal command are ... dreams.

So. I think the following might have to do:

> Redirect the results of a find command listing the entire contents of the DVD to a text file.
> Create an AppleScript to
* repeat for each paragraph of the text file
+ set a variable to the name of the path
+ set a variable to the name of the file
+ create a directory structure at the destination matching the path variable, making directories as needed
+ set a variable to the first 42 characters of the file name
+ set a variable to 0 (number for appending to the file name if needed)
- repeat until the new truncated file name does not exist at the destination path
. add 1 to variable holding number
. append the variable number to the filename
- copy the file to the new location, renaming in the process

I think that you are making this too complicated.

Start out more basic.
Do you have to change the filename?
Don't try to figure out which tools to use yet. Figure it out at a more basic level first.

Brett

Based on what I've said, is there some reason I don't have to change the filename?

Well, communications over the forums are sometimes difficult.

As far as I can tell, you have given no real reason as to why you have to change filenames.

Brett

How big is the DVD?

Brett

Okay. It may be true that I don't have to rename the files. I do need to do something so that when I copy the files, I don't get an error. I believe I've mentioned that several times and I feel confused as to what makes you keep bringing that up.

The DVD is 3GB. I'm leaving work soon, so I may not reply quickly to any new posts.

In Terminal, cd to the DVD.

What happens when you type?

You should see a lot of filenames scroll by. Those are not errors. If there is an error, post it.

This should but a tarball in you home directory. It will be less than 3 Gb.

This may take a bit of time, so hopefully you will get this before you leave and can set it up to run overnight/weekend.

Brett

Just curious - will this DVD - and all its complex file structure - copy more easily from a PC using Windows? Or will Windows also have problems getting a complete copy in the way that you need it...

I'm also curious - what sort of PC do you have, in this modern era, without the capability to read a DVD data disk.

Actually, at one point, we did put the DVD on another Windows machine and tried copying files that way. The errors were legion. I got about 10 errors, each stopping the process, before I gave up and looked for another way to get this done.

Brett, I've already left work, so I'll have to give that tar command a shot later.

If I'm going to copy this tarball over to Windows, I need to rename the files before putting them in the tar. The long file names do not work in Windows.

I apologize, I'm reading what I said earlier, and I have not been clear. It's been a long day.

File names past a certain length do not work in Windows.

How was the DVD created? I made the assumption that it had been created on a windows machine.

Are you trying to put it on old hardware/software like onto a window95 box?

Does it puke at the filename, or the combination of the path+filename? The examples that you gave are not that long. I don't know about the 'special' characters though. On one of the files that it gives you an error, how many characters are in the full path + filename, including spaces?

Another possibility is that the DVD may be bad.

It worries me that a good windows box cannot handle the DVD.

Can you be more specific with what the errors were, if there were only 10 of them, and which machine generated them.

Also, which machine is the samba server and which one is the client?

Brett

I used find last night to create a text file containing a list of all the files and subfolders. I also tinkered with an Automator workflow to do the renaming and copying. The test version went well. I've got a little more tinkering to do, but it looks it will work.

Brett, don't know how the DVD was created. Using Windows XP. Windows doesn't seem to like exceptionally long filenames. I don't remember the magic number, but 50 was well below the number. One of the error-producing filenames is 89 characters in length.

I don't know if the DVD was bad. Is there such a thing as a good Windows box?

For the error-producing files, I get an error message which says something like 'unable to open file'.

Unsure that I know what you mean by 'which machine is the samba server'. I don't have access to the server; that's IT's department.

Well, as long as you have something that you will think work.

What you say, doesn't really make sense, though. If a windows box isn't happy with a 50 character filename, how did it create files with 50+ characters?

The client is the box requesting service, the server is the box giving, well, service. If you IT department is providing a Network Drive, and your local desktop connects to it, you desktop is the client.

Let us know how it goes.
Brett

Yeah, that part boggles my mind, really. I don't understand how that's possible.

I'm working on the client machine, then.

sojourner - Is this DVD a commercial disk - which may have that file corruption built-in as a form of copy protection?
What exactly is the DVD? What company produced it?
Maybe the files are supposed to live on the DVD, similar to some games, and the DVD must be in the drive for those files to be accessible.

It looks like a generic DVD. I don't see a name on it or anything.

The disk didn't magically appear - or did it? :D
The disk came from somewhere - what is is the source of the disk?
Is it a disk that you created, or is it a disk that you bought somewhere? - Or an image that you downloaded? Or is it a disk with a customer's files?
If none of those has any answer - can you tell us what those files are used for? Graphics, or a database of some kind? Is there an app on the disk that allows you to use the files?

Reason that I ask - the source and purpose of the files on that disk may help determine why it's so challenging to copy.

OR - maybe the disk is just damaged somehow.

Tell me more.

"Tell you more" - what?
The disk might have copy protection in some form, or will only be accessible from the DVD.

Did you create the file structure on that disk?
Is there something unusual about the disk file setup, which demands use from that disk only?
Is there an on-disk app that allows use of the files?
what is the ultimate use of the DVD (stock photos, clip art, mailing list, newspaper articles, etc, etc)
You still haven't answered - what is the source of that disk?

It's a generic disk with files on it that someone gave me.

And since several people (not just you) are very curious as to why Windows refuses to copy, I'm curious as to why people are curious. The geeks in IT were of the opinion that long filenames are something that creates problems in Windows. What experiences makes your opinion differ from IT?

Has this thread actually started yet?

My gut says no.

Thanks. That was greek to me, but I'll forward it to IT. Maybe they can make heads or tails of it... when they have time.

According to the internet, Windows has a problem with file names that are too long.

Is this a hazing ritual?

Everything on the internet is right, right :)

Even this post.

All kidding aside, the links that I saw are referring to names with sharing. I can totally see some weird limiting going on with sharing.

Unfortunately, it still isn't really clear what you are doing. Some of what you have described is clear, and some isn't.

It seems that you work at some company that has an IT department. You don't need to name names. Do you have access (perhaps through the IT dept) to windows boxes with XP, Vista, what ever the shortcut for windows7 is?

Do these machines have local harddrives and can you write to them?

I would look at the copy issue first. Can I create a root level directory (say c:\myData) on the local hard drive? Can I write the data there? If not, can I capture some the errors? If there are multiple errors, are they the same. If I do the same thing on a machine with a different OS, say Vista, do I get the same errors?

Now if I try and do the same thing on a network share, do I get the same issues? What about a location with spaces in the path (like c:\Documents and Settings\username)?

If I didn't have access to all of those different machines, or the answers are always yes, the error message is identical, I would maybe try the same thing on a linux or mac box, again going from DVD to a local drive.

If that works, I would then make a little test directory that contains some files that you know will work on a windows box and one file that you know doesn't work.

I would then play around with tar, rsync, ditto, etc to see what works and what doesn't. If nothing works, I would then start evaluating if more time, read money, is really necessary.

Now, perhaps, it is time to think about ways that might fix the copy issue. Renaming the files is not a bad idea, but what if in directory1 there is:

foo.txt
fooLonger.txt

but in directory2 there are the different files with the same names.

Would you go directory1

foo_1.txt
foo_2.txt

directory2

foo_3.txt
foo_4.txt

and so on. Or would you restart with foo_1.txt in directory2. Are any of the files duplicates?

Can you even access the contents of the files? I would try both before and after the name changes. Perhaps the files are not even readable?

I would start making plans, starting with the very basic and/or simple first, lay out the plan in English (or whatever you native tongue is, just don't use computer jargon), and then figure out how to implement it.

This is where I would start.

Brett

If you have more than 100 duplicates, the script will go into an infinite loop. That is, it'll try:

prefix.msg
prefix 01.msg
prefix 02.msg
...
prefix 98.msg
prefix 99.msg
prefix 00.msg
prefix 01.msg <== Oops! Already tried this one
prefix 02.msg <== Ditto.
...


You ignore Murphy at your peril.

Thanks for pointing that out. From what I can tell, in any given folder, I don't have more than 99 dups. But I'll make a note of that in case I have to re-use this script for such a case.