The macosxhints Forums - launchd WatchPaths not triggered by CGI file change

The macosxhints Forums (http://hintsforums.macworld.com/index.php)

- UNIX - General (http://hintsforums.macworld.com/forumdisplay.php?f=16)

- - launchd WatchPaths not triggered by CGI file change (http://hintsforums.macworld.com/showthread.php?t=87273)

launchd WatchPaths not triggered by CGI file change

I have a Launch Agent watching a directory (WatchPaths). When a file is added, removed, or altered by user interaction, the Agent is triggered. However, I need it to be triggered by updates being made by a CGI script -- which it is not currently doing.

What version of Mac OS X? Tiger or Leopard? Server or Client?

Hi Mikey-San. It is:
Intel Mac Mini 2.0
Mac OS X 10.4.10 Client

It might be due to it being a LaunchAgent as opposed to a LaunchDaemon.
As I recall, there were various problems with the former in Tiger. I don't think they ever really worked quite right from what I recall reading.
So try making it a LaunchDaemon instead.

Can I make it a LaunchDaemon just by moving it to /Library/LaunchDaemons, or is there more to it?

http://developer.apple.com/macosx/launchd.html

http://developer.apple.com/documenta...01791-BBCBHBFB

Quote:

Originally Posted by ADC

f you need to provide user-specific services, you should create an agent instead of a daemon. An agent is essentially the same thing as a daemon, except that it runs in the context of a user session. Agents can communicate with other processes in the same user session and with system-wide daemons in the root context. Because they have access to the window server, agents can also post a user interface, although they should do so sparingly, if at all. Like daemons, agents should be launched automatically.

The only difference between a daemon and an agent is location: daemons are installed in /Library/LaunchDaemons, while agents are installed in /Library/LaunchAgents or in the LaunchAgents subdirectory of an individual user’s Library directory.

Agents run for specific users, or when users log in, so you can toss it in /Library/LaunchAgents or ~/Library/LaunchAgents

If you want it to run at start up regardless put it in /Library/LaunchDaemons

To make sure your launchd item is running you can invoke this command

sudo launchctl list

if your plist item is listed then its running

OK, I've tried it in:
/~Library/LaunchAgents
This worked most consistently, but didn't acknowledge changes made by CGI.

/Library/LaunchAgents
Also didn't acknowledge CGI.

/Library/LaunchDaemons
This only worked when I trashed a file.

/System/Library/LaunchAgents
Also didn't acknowledge CGI.

/System/Library/LaunchDaemons
Also didn't acknowledge CGI.

Here's the basic XML I'm working with:

Code:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

<dict>

        <key>Label</key>

        <string>com.mdc.rsync1c2</string>

        <key>LowPriorityIO</key>

        <true/>

        <key>OnDemand</key>

        <true/>

        <key>ProgramArguments</key>

        <array>

                <string>/Users/myuseraccount/myworkingshellscript.sh</string>

        </array>

        <key>WatchPaths</key>

        <array>

                <string>/Users/myuseraccount/Sites/sitedirectoryiwanttowatch</string>

        </array>

</dict>

</plist>

When the CGI modifies the file, the changes are reflected in the text, and the modification date changes. So why isn't the system acknowledging the event? Does it have anything to do with the socket settings? (Sorry if that's a dumb question.)

is your launchd item even running? Have you verified that?

Quote:

Originally Posted by tlarkin (Post 458081)

is your launchd item even running? Have you verified that?

Yes, through launchctl and the Lingon interface both. The script will be triggered by some events, just not the event I want (the CGI modifying the file).

Quote:

Originally Posted by hsthompson (Post 457921)

What happens if you run the CGI script manually (from Terminal)?
Does it trigger the Agent?

Are you sure that watching a folder is supposed to trigger upon changes to the content of files in that folder? (I haven't looked up the documentation.)
Maybe when you alter the file via "user interaction" you are actually deleting and recreating the file - that is the way that a lot of text editors work for example - but that your CGI script keeps the same file but alters it.

Consider running 'sudo fs_usage | grep name_of_your_file'
and looking at what operations are being done in the two cases.

if his paths are right Hayne, the ondemand switch in the plist should keep the launchd item always monitoring that path.

Just curious what is the ownership of your launchd plist file? Is it owned by root:wheel or by localadmin:admin?

I have had issues in the past with launchd items and 10.4 if it wasn't owned by root. Here go some official guide lines from Apple on the subject

Quote:

To support launchd(8), it is recommended that you obey the following guidelines when writing your daemon code:

*

Wait until your daemon is fully initialized before attempting to process requests. Your daemon should always provide a reasonable response (as opposed to an error) when processing requests.
*

Register the sockets and file descriptors used by your daemon in your launchd configuration property list file.
*

Check in with launchd as part of your daemon initialization using the routines in launch.h.
*

During checkin, get the launch dictionary from launchd, extract its contents, store those contents locally, and get rid of the dictionary. Caching the dictionary locally and accessing it frequently could hurt performance.
*

Provide a handler to catch the SIGTERM signal.

In addition to the preceding list, the following is a list of things it is recommended you do not do in your code:

*

Do not set the user or group ID for your daemon. Include the UserName, UID, GroupName, or GID keys in your daemon’s configuration property list instead.
*

Do not set the working directory. Include the WorkingDirectory key in your daemon’s configuration property list instead.
*

Do not call chroot to change the root directory. Include the RootDirectory key in your daemon’s configuration property list instead.
*

Do not call setsid to create a new session.
*

Do not close any stray file descriptors.
*

Do not change stdio to point to /dev/null. Include the StandardOutPath or StandardErrorPath keys in your daemon’s configuration property list file instead.
*

Do not set up resource limits with setrusage.
*

Do not set the daemon priority with setpriority

Although many of the preceding behaviors may be standard tasks for daemons to perform, they are not recommended when running under launchd. The reason is that launchd configures the operating environment for the daemons that it manages. Changing this environment could interfere with the normal operation of your daemon.

OK, I ran the command you suggested, and accessed the file from the Finder and edited it in TextEdit (you can see rsync come up in the log). Then I accessed it from the CGI.

Code:

//Accessing in Finder

Intel-Mini-2-0:~ mycomputer$ sudo fs_usage | grep myfile.txt

15:06:38  getattrlist     /.vol/234881026/557421/myfile.txt                                             0.000016   Finder      

15:06:38  open            /.vol/234881026/557421/myfile.txt/..namedfork/rsrc                            0.000053   Finder      

15:06:38  stat            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000009   Finder      

15:06:38  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000007   Finder      

15:06:40  open            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000034   mds         

15:06:40  stat            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000028   TextEdit    

15:06:40  listxattr       sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000009   TextEdit    

15:06:40  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000009   TextEdit    

15:06:40  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000005   TextEdit    

15:06:40  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000005   TextEdit    

15:06:40  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000012   TextEdit    

15:06:40  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000010   TextEdit    

15:06:40  stat            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000007   TextEdit    

15:06:40  listxattr       sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000007   TextEdit    

15:06:40  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000010   TextEdit    

15:06:40  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000007   TextEdit    

15:06:40  open            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000012   TextEdit    

15:06:40  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000009   TextEdit    

15:06:40  open            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000009   TextEdit    

15:06:40  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000011   TextEdit    

15:06:40  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000025   TextEdit    

15:06:40  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000010   TextEdit    

15:06:40  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000011   TextEdit    

15:06:40  open            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000035   mds         

15:06:49  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000008   TextEdit    

15:06:49  stat            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000005   TextEdit    

15:06:49  listxattr       sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000006   TextEdit    

15:06:49  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000009   TextEdit    

15:06:49  access          sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000006   TextEdit    

15:06:49  chmod           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000034   TextEdit    

15:06:49  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000010   TextEdit    

15:06:49  stat            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000007   TextEdit    

15:06:49  listxattr       sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000006   TextEdit    

15:06:49  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000008   TextEdit    

15:06:49  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000005   TextEdit    

15:06:49  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000006   TextEdit    

15:06:49  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000030   TextEdit    

15:06:49  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000009   TextEdit    

15:06:49  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000011   TextEdit    

15:06:49  open            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000028   mds         

15:06:49  stat            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000019   Finder      

15:06:49  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000006   Finder      

15:06:49  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000029   mdimport    

15:06:49  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000010   mdimport    

15:06:51  lstat           myfile.txt                                                                    0.000004   rsync       

15:06:51  open            sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000031   rsync       

//Launching CGI

15:07:41  stat            myfile.txt                                                                    0.000005   mc          

15:07:41  stat            myfile.txt                                                                    0.000004   mc          

15:07:48  stat            myfile.txt                                                                    0.000005   mc          

15:07:48  stat            myfile.txt                                                                    0.000003   mc          

fs_usage: buffer overrun, events generated too quickly

15:07:54  lstat           sers/mycomputer/sites/mylocalsite/myfile.txt   0.000020   mc          

15:07:54  open            sers/mycomputer/sites/mylocalsite/myfile.txt   0.000017   mc          

15:07:55  stat            myfile.txt                                                                    0.000006   mc          

15:07:55  stat            myfile.txt                                                                    0.000003   mc          

15:08:03  lstat           sers/mycomputer/sites/mylocalsite/myfile.txt   0.000033   mc          

15:08:03  stat            sers/mycomputer/sites/mylocalsite/myfile.txt   0.000015   mc          

15:08:03  lstat           sers/mycomputer/sites/mylocalsite/myfile.txt   0.000016   mc          

15:08:03  open            sers/mycomputer/sites/mylocalsite/myfile.txt   0.000078   mc          

15:08:03  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000028   mdimport    

15:08:03  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000014   mdimport    

15:08:03  lstat           sers/mycomputer/sites/mylocalsite/myfile.txt   0.000022   mc          

15:08:03  open            sers/mycomputer/sites/mylocalsite/myfile.txt   0.007622 W mc          

15:08:04  lstat           sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000015   mdimport    

15:08:04  getattrlist     sers/mycomputer/Sites/mylocalsite/myfile.txt   0.000010   mdimport    

^C

Intel-Mini-2-0:~ mycomputer$

Quote:

Originally Posted by tlarkin (Post 458152)

Which folder would I be keeping it in to test this?

Quote:

Originally Posted by hsthompson (Post 458154)

Which folder would I be keeping it in to test this?

If you want it always running keep it in /Library/LaunchDaemons, Apple was pretty specific in their launchd white page to never toss anything in /System leave that for apple because a system update could hose your daemons.

Quote:

Originally Posted by tlarkin (Post 458155)

I think the one I logged was set up in /Library/LaunchDaemons.

I just tried editing that file again through the CGI. It worked once, triggering the rsync script; then it failed to trigger it subsequent times. What does it mean?!

I don't see 'root' in the list of ownership options. Am I missing something?

If you tossed the plist in /Library/LaunchAgents this simple command will tell owner:group

Code:

ls -al /Library/LaunchDaemons | grep com.mydaemon.plist

Change the grep part to the name of your file

Here's what it said:

-rw-r--r-- 1 root wheel 587 Mar 14 15:24 com.mdc.mylaunchdtest.plist

Ok, its owned by root:wheel so that looks good, that was just a guess and an issue I had ran into in the past....

Should it be executable or anything?

Quote:

Originally Posted by hsthompson (Post 458170)

Should it be executable or anything?

no, it doesn't need to be executable as far as I can tell. The documentation I have read never indicated it must be executable, but it must be in the standard plist format of

domain.mycompany.plist

for example

Quote:

Originally Posted by tlarkin (Post 458152)

if his paths are right Hayne, the ondemand switch in the plist should keep the launchd item always monitoring that path.

I notice for 10.5 that the onDemand has been deprecated

Quote:

OnDemand <boolean>
This key was used in Mac OS X 10.4 to control whether a job was kept
alive or not. The default was true. This key has been deprecated and
replaced in Mac OS X 10.5 with the more powerful KeepAlive option.

KeepAlive

Quote:

KeepAlive <boolean or dictionary of stuff>
This optional key is used to control whether your job is to be kept con-tinuously continuously
tinuously running or to let demand and conditions control the invocation.
The default is false and therefore only demand will start the job. The
value may be set to true to unconditionally keep the job alive. Alterna-tively, Alternatively,
tively, a dictionary of conditions may be specified to selectively con-trol control
trol whether launchd keeps a job alive or not. If multiple keys are pro-vided, provided,
vided, launchd ORs them, thus providing maximum flexibility to the job to
refine the logic and stall if necessary. If launchd finds no reason to
restart the job, it falls back on demand based invocation. Jobs that
exit quickly and frequently when configured to be kept alive will be
throttled to converve system resources

I am not familiar with CGI scripts, but I get the impression that the script is modifying a file rather than moving files, Is this correct?

When I test a folder that is being watched, it will trigger if a file is moved To or From its path, but will not trigger if a file has been updated/modified

This appears to be the intended behaviour for watchPaths.

*edit*
Maybe adding something like this to the end of the script..

Code:

mv /Users/username/Desktop/foo_folder/foo.txt /Users/username/Desktop/foo_folder/mvFolder/foo.txt;mv /Users/username/Desktop/foo_folder/mvFolder/foo.txt /Users/username/Desktop/foo_folder/foo.txt

Quote:

Originally Posted by hsthompson (Post 458159)

launchd will automatically terminate a task if it runs into errors (certain kinds of errors, anyway). since it worked once, this makes me think there's something buggy in your shellscript. is it short enough to post here?

Quote:

Originally Posted by mark hunte (Post 458246)

When I test a folder that is being watched, it will trigger if a file is moved To or From its path, but will not trigger if a file has been updated/modified...

launchd is supposed to pick up on any change to a file path, including changes to files (in fact, I've used that feature before...). now it's possible that watchpath that's set to watch a folder might not pick up changes to a file within that folder - I've never tried that before. is that what you were doing?

There's also a binary at /bin/wait4path which should provide the same (or similar) action. [see man wait4path]

From a very limited test last night, mine seems to stay asleep forever (well, longer than I felt like waiting).

-HI-

Quote:

Originally Posted by tw (Post 458476)

OK.

Code:

#! /bin/sh

rsync -avz --delete-after ~/Desktop/testfolder myserverdomain@ssh.myserverdomain.com:./public_html/

Quote:

Originally Posted by mark hunte (Post 458246)

Code:

mv /Users/username/Desktop/foo_folder/foo.txt /Users/username/Desktop/foo_folder/mvFolder/foo.txt;mv /Users/username/Desktop/foo_folder/mvFolder/foo.txt /Users/username/Desktop/foo_folder/foo.txt

The CGI script is written in Runtime Revolution, which I'm not very familiar with. The author of the CGI script told me running shell commands from within that environment was not possible. I don't know enough about it to tell.

I was thinking, maybe an approach could be to to have one watchpath to look at the folder. when a new file is added it writes the path of the new file to a second watchpath which watches the files.
The second one should then trigger on changes the the file.

I have tested parts of this idea and it seems it would work.
The write part is the easy bit,

Code:

defaults  write ~/Library/LaunchAgents/watchpath  "WatchPaths" -array-add '<string>/Users/username/Desktop/foo_folder/foo.file</string>'

But still working on getting the list of items in the folder and then comparing to what is already in the second watchpath plist

mark -- that's awesome. I tested watching the individual file, rather than the folder as I had been, and it does trigger launchd when the CGI edits the file. Keep me posted, and I'll research a solution, too.

This is the code I have been working on, to add into the (folder) watchpath plist.

Code:

#!/bin/bash

##Bash's Internal File Separator (IFS). Normally the separator is any of space, tab, or newline. 'Find' (and 'ls' ) produce new-line separated lists,

#so changing the IFS to just new-line does the trick to cope with the spaces in file names

IFS="

"

#search for files in the given folder, do not use recursive  beyond this level

thenames=`find /Users/username/Desktop/foo_folder  -maxdepth 1 -type f ! -name '.*'`



#for each file found, use grep to see if a matching path is found in the (folder) watchpath plist

#return 0 for no match, 1 for a match

for file in $thenames; do

 thegrep=`/usr/bin/grep -ic "$file" ~/Library/LaunchAgents/watchpath.plist`

#####for testing in terminal, you can add echo $thegrep



#check to see if match was made, if yes, do nothing. If No, '0' , then write the path to the (file) watchpath plist

if [ "$thegrep" = "0" ]; then 

 /usr/bin/defaults  write ~/Library/LaunchAgents/watchpath "WatchPaths" -array-add \<string\>"$file"\<\/string\>

######for testing in terminal, you can add echo /usr/bin/defaults  write ~/Library/LaunchAgents/watchpath "WatchPaths" -array-add \<string\>"$file"\<\/string\



fi

done

Need to stop it adding hidden files like the .DS_Store, which should not be that hard.

*Note, I only have a small amount of knowledge of shell.

**edit 1**
Changed the find part to ignore hidden files

**edit 2**
added to the script, to cope with spaces in filenames.

Also I have now tested this and it works.

The folder watching watchpath plist calls this shell script when a new file is added.
The shell in turn checks against the paths in the file watching watchpath plist and adds any that are not already in it.

Looks good. I'll set set up a sandbox account and try it out later.

Thanks for your help, everybody!