Tar and AFP copy a folder of random name?
 

I'm trying to write script to do the following (and will make an AppleScript droplet or app if that ends up being easier, I'm pretty well versed with applescript).

Take a typed in name as an argument, tar the directories in the specified folder into their own .tar file named based on the folder name and the argument, then upload them to an AFP share.

Example:
Folder containing:
Backup 1
User 8
Backup 4
Archive 7

Run the script, it asks for a name, you give it "jd".
Results in "Backup 1 - jd.tar", "User 8 - jd.tar", "Backup 4 - jd.tar", "Archive 7 - jd.tar".

Then uploads the .tar files to afp://macpro.local./archive/

I could probably do this with a lot of work in AppleScript, but I would think a shell script would be cleaner.

Something like the following should work: (completely untested)

Hayne: You wrote $filename once where you meant $foldername, and you left out a whole bunch of quotes.
(still completely untested)
(still doesn't upload the tar files as requested)

in applescript this would be something like the following, I think:
two points of confusion:

• I don't have an afp share to test on, so I'm basing the afp stuff on this older thread. looks right, but looks can be deceiving. If the open location bit gives you trouble, you might try it with mount volume instead
• I'm not sure about your afp path. As I understand it, normally an afp path (with no user name or password) would look like "afp://server.address/mountname" where mountname shows up as a disk in the Finder. but I can't tell whether the period or the slash after 'local' is a typo - is 'archive' part of the server path, or the volume name? again, my understanding of afp is mostly theoretical.

If you look at the curl command it will allow you to copy files from remote hosts or to remote hosts of HTTP/S and S/FTP. I don't have any way of testing it short of setting up a FTP server and build a test environment from scratch, but that is where I would start.

What do people have against proper quoting?

You know newName needs to be quoted; you just constructed with spaces in it. You know the OP is not shy about putting spaces in file/folder names, elevating the need for quoting anything with baseFolder in it above mere just-in-case bullet-proofing.

You're also mixing Posix paths with HFS paths. How about (still untested)?

Life happens. Besides, if I did everything perfectly, what would you have to complain about? You'd be utterly miserable… :p

That's not really a problem - POSIX is understood most everywhere in applescript except Finder tells, so it's just dealer's choice whether to use posix and convert to aliases for the Finder, or use Finder-compliant references and convert to POSIX for the shell. Your script looks good though. :)

I'd probably complain about that. :rolleyes:

But yeah, I've been at this long enough to know how elusive perfection can be. I was just taken aback to see quoting omissions in the post immediately following my post lamenting quoting omissions. :o

Really? I can't get AppleScript to coerce one into the other, even when it's obvious (to me) which style a given string was supposed to be in. I was going to try sneaking up on ambiguous cases to see when it broke, but I can't even get nearly unambiguous cases to coerce correctly.

For an example of a difficult case, suppose one of the folders we're trying to tar has the name "Version 2010/12/5". Those slashes are perfectly legal in an HFS filename. In a POSIX path, though, the slash character is reserved for the path separator, so:

set baseFolder to "/path/to/basefolder/"
set theRootName to "bv" -- for example
...
set thisFolderName to "Version 2010/12/5" -- implicitly by repeat with ... in subFolders
set newName to thisFolderName & " - " & theRootName & ".tar"
do shell script ("tar -cvf " & newName & " " & baseFolder & thisFolderName)

What you pass to do shell script is:

tar -cvf Version 2010/12/5 - bv.tar /path/to/basefolder/Version 2010/12/5

and no amount of quoting can get around the fact that some of those slashes are unix path separators, and some are from the folder's HFS name.

One of the things that "POSIX path of ..." does is to swap "/" (the POSIX path separator) with ":" (the HFS path separator). The shell script should have read:

tar -cvf 'Version 2010:12:5 - bv.tar' '/path/to/basefolder/Version 2010:12:5'

but there's no way to get there without being scrupulously careful about which strings are POSIX paths and which are HFS paths. Concatenating them willy-nilly has bad results.

yeah, it was stupid of me.

Ah, well, part of the problem here is that I would (personally) never name a Finder file with slashes, so that problem never occurred to me. That's really an issue specific to the list folder command (which returns only the file name) - if we got full file paths from System Events or the Finder we could convert them back and forth with no issue. I probably shouldn't have written it that way, either; my scripting fu was obviously in the toilet when I did that. :o

ok, in that case, we'd probably want to rewrite the center part of the script like so. I think this takes care of the problematic cases:
How does that look to you?

To be honest, it looks like you're trying so hard to make it easy that you're actually making it hard. The only logic flaw is a missing "/" (but at least it's not a quoting omission :rolleyes:):

set newPath to rootPath & "/" & fileName & " - " & theRootName & ".tar"


My version gets it all right, by staying with the HFS paths that AppleScript, Finder, System Events, and pretty much everybody except the shell is happy with, right up until I need to pass paths to the shell. It's only in do shell script that I need to even mention POSIX (or quoted form of ..., for that matter).


Although...
Working with POSIX paths throughout isn't so bad, once you realize there's no need to parse them (for this problem):

Hey, you're the one who came up with this improbable constriction - don't blame me for trying to cover the bases. :) The rest is true, though - the text item processing is unnecessary (though your revision below might mess up minorly if there's a burn folder, smart folder, or package folder in the mix… :p).

Hahaha, I had forgotten how funny some of your guys' threads could get (I haven't posted regularly here for years).

For some reason the forum didn't email me on replies, so I haven't looked at the thread until just now.
edit: I had forgotten my email address was an old non-existant one... whoops, fixed.

You can switch between posix paths and finder/applescript aliases, but the code I had to do that is at home, and I'm at work.

Here's the take on it I threw together in applescript last weekend. It handles spaces in the source. For the record, there won't ever be a /, :, or \ in the original folder names. I know its ugly, and I ran into some weird overflow, I really need to find my old code snippits collection...

This one also doesn't upload, but this is all local network, so I'm pretty sure I can do "tell application Finder to copy "blah blah -jd.tar.gz" to afp://server/archive/", but I haven't tried that part yet. Getting archiving down first.

Cleaner Code
 

Here's my cleaner attempt at it. It mostly works, except I'm running into an issue (which I remember from years ago but am forgetting how to deal with it), where when doing the finder copy at the end, the loop tries to continue.

Anyone know how to stall the end by checking for file busy or done?

I'm not sure what you mean by "the loop tries to continue", but you don't actually need that last repeat loop anyway - the Finder will copy lists just fine. here's a tweaked version of what you gave (with a couple of other stylistic changes -combining the first two finder tells, removing the variable 'currentDone', etc - just because):

Well that makes life easier, thank you. I've never tried to copy a list before, always just individual files.
The excess variables and finder statements are mainly due to the way I build and test pieces of AppleScript at a time. Once it all works I'll usually do a full cleanup and consolidation. ;)

To clarify my earlier statement, what I mean is this...

The repeat loop grabs a file reference, tells the finder to copy it, and then while the finder is copying it, goes to the next item in the list without waiting for the Finder to finish what it was doing. In my specific testing, it was trying to use the same file twice and would say "file already exists", even though the compressedFiles list only had 1 entry. It didn't make any sense, though I remember running into a similar issue with a print script years ago.

you're right, that doesn't make any sense. Just out of curiosity, do you get the same effect if you use:
I'm thinking the problem might be due to fact that eachFile is a reference rather than a value.

Ironically doesn't work. Returns Can't set <<class cdis>> "Archive" of application "Finder" to {file blah:blah:path:to:tar.gz}.

Back at it, but you've given me more ideas at least.

Nope, same <<class cdis>> error as trying to copy the entire list at once.

sorry, I wasn't paying attention - Copy means something other than you think it does to the Finder, and doesn't work anyway. Use duplicate instead:

*facepalm* Of course. Not the first time that has bitten me either.

lol - nor me. Aint appleScript fun? :rolleyes:

More fun than trying to duplicate this functionality on the windows machines at my shop.

Well it works, usually. I'm still trying to find what is breaking it. Sometimes (seemingly right after the shell scripts), I get an apple event error and it all stops. I believe its actually timing out after a large (>50GB) directory to tar, but I'm not sure since it doesn't say time out.

I'm about ready to re-write this in x-code so I can setup breakpoints, as it is I have put alerts at various points to check flow, but it isn't ideal.

Just try it with a with timeout block, see if that fixes it:

I remember that now. Though none of the old code I brought with me today has it. Some of these things could run for hours (multiples of >200GB folders).

I'll give it a shot for troubleshooting.

well, if they could really run for hours (yikes) you might want to rewrite this as a stay-open application: basically fire off the tar command as a subprocess then use the idle loop to periodically check to see if the subprocess is complete. that's a more convoluted script (you'll have to recover and store the process id of the tar process and check for it in the idle loop, and if it's more efficient you might launch multiple subprocess and wait for them all to be done), but at least you'll avoid having that frozen script thing you get when you run long tasks as simple scripts.

Yay rewrites! They literally can run for hours.

Just a high level overview, this project is for a tech shop I do work for that does quite a bit of data recovery. The goal is that once a recovery is done (typical is 80GB, but we're seeing more an more multiple hundred GB ones), to tar up the recovered data, put it on our server for safekeeping until pickup, and clear the space off the tech's workstation's drives so they can keep working.

This is something we'd run overnight, and I wouldn't be surprised to see it take hours.

Thanks for everyone's help and advice so far, by the way. I really appreciate it.

And, just for fun, I tried the timeout thing. Same error (and now I've also seen it happen within 5 minutes, so probably not a timing thing).

well, then what I'd do is throw in a bunch of log or display alert statements and see what the last command executed before the error is. that should point to the issue. which apple event error are you getting?

Yes, I know how to do that too.

The point I was trying to make was that, although you can keep switching back and forth between POSIX paths and HFS paths, the constant switching makes your code both harder to read and more error-prone. It's too easy to get the path separators mixed up and/or duplicated and/or omitted, too easy to forget which variables currently hold POSIX paths, which hold HFS paths, and which hold actual files/folders/aliases/URLs. The constant repetition of the "POSIX path of" mantra clutters up the code, making it less readable.

As foreign as HFS paths may seem to people more comfortable at the command line, your AppleScript code is cleaner if you use them.

When in Rome, speak Latin. When in AppleScript, speak AppleScriptese. Translate to POSIX when you need to go to POSIX-land, but translating prematurely just makes things more complicated.

And, how do you say Piña Colada in Spanish?

La bebida de Gringos.

By way of example, consider:

Create a test file with, for example:
date > ~/Desktop/Summary\ for\ 12:25:2010.txt
and then run:
Only path1 is correct. path2 gets the path separators mixed up. path3 has duplicate path separators.

And if you even dream of playing games with text item delimiters and manipulating path separators explicitly, you're missing the point.

I haven't been able to test again until this weekend. It is throwing "AppleEvent handler failed."

It appears to between the end of the file getting tarred, and the beginning of the finder copy. Usually the error comes up when the .tar.gz file is its final size, but the finder copy never starts.

well, best guess then is that it has something to do with the mounted share. Either the share is not mounting fast enough and the Finder is trying to write to a non-existant disk or there's something wrong with the open location command. try these diagnostics:

1. Make sure the disk is mounted, comment out the if not (exists disk "Archive")... block, and see if the script works if it doesn't have to deal with mounting the share.
2. Copy the if not (exists disk "Archive")... finder tell block to a different script and see it will by itself (a) mount the disk correctly, or (b) throw errors if the disk is already mounted
3. if (1) and (2) check out try moving the if not (exists disk "Archive")... tell block to the front of the script, right after the on open line. that will give the system more time to mount the disk while it's processing the tar files. or you could write in a delay-check loop, but likely mounting the disk earlier will be sufficient.

Logic eludes me when I'm banging my head on a brick wall sometimes...

Archive is a local volume on the machine I'm testing on, it should always be true.

I'll loop the volume check in another script and see what happens.

I'll also put it out there that the same error occurs on 3 machines. 1 running 10.6.8 and the rest running 10.6.7. Archive is a local disk on mine, the other two have to mount it over the network.

The "AppleEvent handler failed" thing is fairly specific - it points to a command in an application dictionary or osax that is not implemented (or not implemented correctly, anyway). literally what it's telling you is that the target app received an apple event from applescript but can't find an internal handler that's supposed to be there. In this particular case (starting from the do shell script and moving down):

• copy in line 20 is part of applescript core - no problems there.
• duplicate in line 28 is a known-functional command in the Finder, no problem there.
• that leaves something to do with the open location command

now it might be a problem with calling open location (part of the Standard Additions osax) from inside a Finder tell block. you could try this and see if it works:

That is something I never knew, and might help with troubleshooting.

Also, I just realized I never posted the current code.

That tell block is now:
Open location didn't work properly. Mount volume (and open location) are both part of the StandardAdditions, which I thought could be used anywhere?

Apple tightened things a bit in 10.6, so that osax couldn't be used quite as freely as they were before. the problem is name conflicts: for instance, if you have a command in an app called "cease fire" and a command in an osax called "cease" there is some ambiguity about which command was intended. So now what applescript sometimes does is seek out a command in the application dictionary but not propagate the search to osaxen (hence you get a 'handler not found' error rather than the script trying to run an incorrect command).

It's more complex than that, obviously - you often can use SA commands inside app tell blocks - but I don't know where or why it works when it does, so I take the conservative route and try not to use osax commands in tell blocks.

That's a lot of good info. I haven't really used AppleScript since 10.5. I made a few adjustments to the script and am testing it again (unfortunately my test file is 300GB, so it takes a bit to fail).

And it works, finally.

Ended up doing the trick for that AppleEvent error. I also added timeouts to the finder duplicate, and the shell script (which probably doesn't do anything, but it isn't hurting). Tested with a 200GB and a 350GB folder, one with a local copy and one with a remote copy.

Thanks everyone.

Now I get to learn how to do essentially the same thing on a Windows machine, as well as find a good way to make .tgz files on windows.

Ack! Just did a 1MB test file for a "how to" video for my co-workers, and it threw an error again. Back to the drawing board.