The macosxhints Forums

The macosxhints Forums (http://hintsforums.macworld.com/index.php)
-   UNIX - Newcomers (http://hintsforums.macworld.com/forumdisplay.php?f=15)
-   -   What's Wrong With This Shell Script? (http://hintsforums.macworld.com/showthread.php?t=113131)

hayne 07-29-2010 09:24 AM

I understand that you've been running this script from the command-line as 'root'.
Are you sure that the CCC environment runs the script as 'root' ? If not, permission problems are an obvious thing to look at.

mnewman 07-29-2010 06:14 PM

I've tried running it both as "me" and as root. Should I try running it as some other user?

Kyd 07-30-2010 07:56 AM

I don't really think it'll help, but I would try the code without the trailing & as that seems to be the only difference between the older working and the current non-working code.

The problem isn't in mail directly as you've demonstrated merely tailing the contents into a file doesn't work.

The other difference is when you run the code manually, there is a controlling terminal. When run via Postflight, there isn't. Maybe a nohup is needed.

mnewman 07-30-2010 07:19 PM

This has to be run with the trailing & because it has to run asynchronously. Otherwise the disk gets unmounted before CCC is finished causing CCC to exit with an error.

Where would the 'nohup' go?

mnewman 07-30-2010 08:03 PM

It Finally Works
 
Here's the version that works:

Code:

(sleep 60; diskutil unmount 02352EBA-9C0A-3992-AA28-BBCBC7A29367 >> /library/logs/CCC.log 2>&1; sleep 10; tail -n 25  /library/logs/CCC.log | mail -s "CCC" root) &
The only thing I can see that is different from the versions that didn't work is the length of the second 'sleep' command; ten seconds instead of 60 or 120.

Oddly, although this script works, it fails to unmount the disk:

Code:

07/31 06:56:54        Running postflight script: /usr/share/Clone.sh
07/31 06:56:54        Updating the dynamic linker shared cache
07/31 06:56:54        Summary statistics:
        Data copied: 6.91 GB [Please see http://www.bombich.com/software/docs/CCC/en.lproj/troubleshooting/log.html for a comment about this figure]
        Total data in file set: 192.39 GB
        Regular files copied: 10504
        Directories: 218863
        Regular files: 853622
        Symlinks: 20033
        Devices: 0
        Special files: 44
        Hard links: 34458
        Extended attributes: 736088 (546.59 MB)

07/31 06:56:55        The backup operation has completed. Elapsed time: 00:25:51
================================================================================


Volume Clorox on disk1s2 failed to unmount


Kyd 07-31-2010 08:38 AM

Quote:

Originally Posted by mnewman (Post 591452)
This has to be run with the trailing & because it has to run asynchronously. Otherwise the disk gets unmounted before CCC is finished causing CCC to exit with an error.

Where would the 'nohup' go?

The only thing the & is doing in this case is making the shell script return immediately while everything else happens in the background synchronously. I've broken the code down to be more readable:

(
sleep 60;
diskutil unmount 02352EBA-9C0A-3992-AA28-BBCBC7A29367 >> /library/logs/CCC.log 2>&1;
sleep 10;
tail -n 25 /library/logs/CCC.log | mail -s "CCC" root
) &

The & is clearly outside the () and is just putting everything inside the () in the background. The first thing happening is the program sleeps for 60 seconds for no apparent reason. AFTER sleeping, it unmounts the drive putting both stdout and stderr into your log file. I would really expect diskutil to NOT return until it has finished, but a short sleep can't hurt.

You said this worked, but failed to unmount the drive. Did you have anything else open on the drive when you ran the script?.

When you were testing it before, were you waiting 3 minutes for all the sleeps to expire before deciding it hadn't worked?

The nohup would go at the top of script right below the opening line, but I don't think now that is your problem.

mnewman 07-31-2010 07:40 PM

Quote:

Originally Posted by Kyd (Post 591500)
The first thing happening is the program sleeps for 60 seconds for no apparent reason.

Have a look here for the reason: Mounting a volume before a scheduled task and unmounting a volume after the scheduled task

Quote:

Originally Posted by Kyd (Post 591500)
You said this worked, but failed to unmount the drive. Did you have anything else open on the drive when you ran the script?.

Nothing.

Quote:

Originally Posted by Kyd (Post 591500)
When you were testing it before, were you waiting 3 minutes for all the sleeps to expire before deciding it hadn't worked?

Yes.

CCC ran as a scheduled task last night. From the log file you can see that the postflight script ran. But, it neither unmounted the drive nor sent e-mail. This is driving me nuts:

Code:

08/01 04:05:43        Running postflight script: /usr/share/Clone.sh
08/01 04:05:44        Updating the dynamic linker shared cache
08/01 04:05:44        Summary statistics:
        Data copied: 18.10 GB [Please see http://www.bombich.com/software/docs/CCC/en.lproj/troubleshooting/log.html for a comment about this figure]
        Total data in file set: 192.23 GB
        Regular files copied: 794
        Directories: 218894
        Regular files: 853200
        Symlinks: 20033
        Devices: 0
        Special files: 43
        Hard links: 34458
        Extended attributes: 736046 (4.67 KB)

08/01 04:05:44        The backup operation has completed. Elapsed time: 00:35:34
================================================================================


acme.mail.order 07-31-2010 07:54 PM

Why not simply eliminate the problem entirely? Skip the CCC postflight step and do the entire job as a regular shell script. CCC is little more than a wrapper around a couple of commands like hdiutil and rsync - duplicating it's functionality in a shell script is very easy.

mnewman 07-31-2010 08:00 PM

Quote:

Originally Posted by acme.mail.order (Post 591535)
Why not simply eliminate the problem entirely? Skip the CCC postflight step and do the entire job as a regular shell script. CCC is little more than a wrapper around a couple of commands like hdiutil and rsync - duplicating it's functionality in a shell script is very easy.

I wouldn't have the slightest idea where to begin.

If I don't even have the brains to get a one line shell script to run I can't imagine doing something as complex as cloning a drive. Remember, this is the "Unix Newcomers" subforum…..

acme.mail.order 07-31-2010 08:09 PM

Cloning a drive is easy! And you've already worked out some essential concepts.

But lets back up a bit - what is your primary goal? A bootable backup or a restorable image?

Is this on a personal machine or on a server?

mnewman 07-31-2010 08:38 PM

This is on our main personal machine. My goal is a bootable backup. I run CCC automatically once a week. I want to be notified via e-mail that CCC ran. I want to have the backup drive mount only on demand and unmount when CCC is done.

I want the backup drives (CCC and Time Machine) to be unmounted when not in use because it makes a significant improvement in the speed of loading file dialog boxes.

acme.mail.order 07-31-2010 09:06 PM

The bootable backup portion of CCC will do something like this:
(note carefully the spacing around the single / )

rzync -aqAXx --del --log-file=/var/log/myrsync.log / /Volumes/backupVolume

As for the mount/unmount part, Bombich does some fancy things to hide the process from the Finder - no need to duplicate.

To mount an unmounted volume we need to find it's device name, diskutil plus some string massaging:

Code:

mountThis=`diskutil list | grep iWantThisVolume | sed -e 's/.*disk\([0-9].*\)/disk\1/'`
diskutil mount $mountThis

Everything else can be run syncrounously entirely in the background. Get Lingon to create an auto-launch background job.

mnewman 07-31-2010 09:16 PM

AMO - Thanks for this. I will have a look and see what I can do. Right now the monitor for that machine (a MacMini) is in for repairs and I'm doing everything via VNC from an ancient iMac. When I get the monitor back I'll give it a shot.

I'm still curious as to why the script in the OP doesn't work as expected.

acme.mail.order 07-31-2010 09:18 PM

Why don't you enable remote login on the Mini and use Terminal and ssh? Much faster than VNC to the other desktop then using Terminal there.

mnewman 07-31-2010 09:23 PM

Good idea. I've got SSH enabled with public key authentication.

Kyd 08-02-2010 07:00 AM

The long standing way to make sure disk io was completed in the past was to issue "sync; sync; sync", not sleeping and hoping it magically got done. I'd be hard pressed to believe that no longer works on a system that is "certified unix".

mnewman 08-02-2010 08:51 PM

Quote:

Originally Posted by Kyd (Post 591662)
The long standing way to make sure disk io was completed in the past was to issue "sync; sync; sync", not sleeping and hoping it magically got done. I'd be hard pressed to believe that no longer works on a system that is "certified unix".

I tried this:

Code:

mdutil -i off /Volumes/Clorox/ | mail -s "MDS Off" root
sync; sync; sync
(sleep 60; diskutil unmount 02352EBA-9C0A-3992-AA28-BBCBC7A29367 >> /library/lo$

Both e-mail messages got sent, but the drive still failed to unmount:

Code:

Volume Clorox on disk1s2 failed to unmount
I thought the unmount failure might be because mds was trying to index the clone, so I turned indexing off. Obviously, that didn't work. Neither did the sync; sync; sync…

There can be nothing using this volume as it is not mounted until just before CCC runs. No other process uses the volume.

I remain baffled.

hayne 08-03-2010 01:24 AM

You might try using 'lsof' to check for open files on that volume.

mnewman 08-03-2010 01:42 AM

Quote:

Originally Posted by hayne (Post 591725)
You might try using 'lsof' to check for open files on that volume.

Here it is:

Code:

notifyd      26          root  15r      DIR      14,5      7412  607655 /Volumes/Clorox/System/Library/Extensions
notifyd      26          root  17r      REG      14,5  18661932  1104465 /Volumes/Clorox/mach_kernel
notifyd      26          root  18r      REG      14,5      290  509287 /Volumes/Clorox/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
notifyd      26          root  19r      REG      14,5  9111563  1222921 /Volumes/Clorox/System/Library/Caches/com.apple.kext.caches/Startup/Extensions.mkext
notifyd      26          root  20r      REG      14,5    319152  579219 /Volumes/Clorox/System/Library/CoreServices/boot.efi
notifyd      26          root  21r      DIR      14,5      1258        2 /Volumes/Clorox
notifyd      26          root  22r      REG      14,5      479  1121616 /Volumes/Clorox/System/Library/CoreServices/SystemVersion.plist
notifyd      26          root  24r      REG      14,5      389  1232522 /Volumes/Clorox/System/Library/CoreServices/.disk_label
mds          62          root    7r      DIR      14,5      1258        2 /Volumes/Clorox
mds          62          root  40r      DIR      14,5      1258        2 /Volumes/Clorox
Path\x20F  307        mnewman  642r      DIR      14,5        68  1232530 /Volumes/Clorox/.Trashes/501
kextd    14931          root    5r      REG      14,5      1575  1104403 /Volumes/Clorox/usr/standalone/bootcaches.plist
grep      97391        mnewman    1w      REG      14,2        0 52540728 /Users/mnewman/Clorox.txt


mnewman 08-03-2010 09:34 PM

This is too weird. If I put 'sleep 60' as follows:

Code:

(sleep 60; diskutil unmount "$2" >> /Library/Logs/CCC.log 2>&1; sleep 1; tail -n 25  /Library/Logs/CCC.log | mail -s "CCC" root) &
unmount FAILS
mail SUCCEEDS

If I put 'sleep 120' as follows:

Code:

(sleep 120; diskutil unmount "$2" >> /Library/Logs/CCC.log 2>&1; sleep 1; tail -n 25  /Library/Logs/CCC.log | mail -s "CCC" root) &
unmount SUCCEEDS
mail FAILS

Here's what was happening with the volume (lsof) just before the 'sleep' command:

Code:

notifyd      26          root  15r      DIR      14,5      7412  607655 /Volumes/Clorox/System/Library/Extensions
notifyd      26          root  17r      REG      14,5  18661932  1104465 /Volumes/Clorox/mach_kernel
notifyd      26          root  18r      REG      14,5      290  509287 /Volumes/Clorox/Library/Preferences/SystemConfiguration/com.apple.Boot.plist
notifyd      26          root  19r      REG      14,5  9111563  1222921 /Volumes/Clorox/System/Library/Caches/com.apple.kext.caches/Startup/Extensions.mkext
notifyd      26          root  20r      REG      14,5    319152  579219 /Volumes/Clorox/System/Library/CoreServices/boot.efi
notifyd      26          root  21r      DIR      14,5      1258        2 /Volumes/Clorox
notifyd      26          root  22r      REG      14,5      479  1121616 /Volumes/Clorox/System/Library/CoreServices/SystemVersion.plist
notifyd      26          root  24r      REG      14,5      389  1234254 /Volumes/Clorox/System/Library/CoreServices/.disk_label
mds          62          root    7r      DIR      14,5      1258        2 /Volumes/Clorox
mds          62          root  49r      DIR      14,5      1258        2 /Volumes/Clorox
Path\x20F  307        mnewman  643r      REG      14,2      1544 52540728 /Users/mnewman/Clorox.txt
Path\x20F  307        mnewman  664r      DIR      14,5        68  1233691 /Volumes/Clorox/.Trashes/501
kextd    14931          root    5r      REG      14,5      1575  1104403 /Volumes/Clorox/usr/standalone/bootcaches.plist
update_dy 66559          root  txt      REG      14,5  8405696  1121813 /Volumes/Clorox/System/Library/CoreServices/Finder.app/Contents/MacOS/Finder

Anything there that would keep the volume from unmounting?


All times are GMT -5. The time now is 10:39 PM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
Site design © IDG Consumer & SMB; individuals retain copyright of their postings
but consent to the possible use of their material in other areas of IDG Consumer & SMB.