Terminal Shell Rsync Backup Question...
 

Hello everyone,

I have a question. I am fairly new to Macs, and I recently got a new job as IT support at a company which has a Mac server for their out of state office. Problem is the guy who worked here before me setup a terminal shell file that users could click on at the end of the day to backup their files to their individual "home" folder on the server. Problem is for some reason the shell is not doing what it is supposed to. Here is the error that one person is getting:

Last login: Wed Apr 16 08:30:37 on console
Welcome to Darwin!
/Volumes/lizw/My\ Backups/Daily.command; exit
Lizs-Computer:~ lizw$ /Volumes/lizw/My\ Backups/Daily.command; exit
/Volumes/lizw/My Backups/Daily.command: line 2: /usr/local/bin/rsync: No such file or directory

real 0m0.007s
user 0m0.001s
sys 0m0.003s
logout
[Process completed]
Basically the backup is to include the Documents, Desktop, Public, and shared folders, while excluding the Music, Library, Movies, and sites folders.

I am kind of at a loss here, any help would be greatly appreciated.

Here is the shell contents:

#rotating backup script - v1.0
time /usr/local/bin/rsync --rsync-path=/usr/local/bin/rsync -az --eahfs --showtogo --exclude=/Users/$USER/Library --exclude=/Users/$USER/Pictures --exclude=/Users/$USER/Music --exclude=/Users/$USER/Movies --exclude=/Users/$USER/Sites --exclude=/Users/$USER/.Trash --exclude=/Users/.DS_Store --link-dest="/Volumes/$USER/My Backups/Archive/" "/Users" "/Volumes/$USER/My Backups/Archive/"

Basically the backup needs to include the Documents, Desktop, Public, and shared folders, while excluding the Music, Library, Movies, and sites folders.

I am kind of at a loss here, any help would be greatly appreciated.

And line 2 reads:

Is rsync in that location? Run:

How are these macs managed, through an open directory master? If so, you can turn on home folder synchronizing and modify it from a GUI and not even have to deal with the scripting.

Wow! You guys work fast around here! I am pretty much a newbie to Macs, so I apologize in advance for not quite knowing what the responses meant. I will try to answer questions to the best of my ability.

I am not for sure if the rsync is located in that directory, as I just started on Monday. I am pretty proficient with PCs but still green with Mac servers. Upon running "which rsync" in Terminal I get "usr/bin/rsync"

As far as the second question goes, I am not sure what an open directory master is. Again I apologize for not being much help, but I inherited another person's work and am still in the feeling out process.

You answered the first question ... rsync is not where the script thinks it should be. The script calls rsync with /usr/local/bin/rsync but it's actually in /usr/bin/rsync

If you change that portion of the script, I believe it should run (at the very least). Additionally, you could probably just change that portion of the script to just "rsync", as the path should work out.

btw, this has nothing to do with the Mac -- this is a problem/issue you could encounter in running any script on any OS

I haven't checked this for Leopard, but in Tiger the version of rsync that Apple shipped with OS X was a branch from the main rsync trunk. It is very likely that your predecessor installed the most current build of rsync on each of the machines that he was supporting. That might explain why the script looks in /usr/local/bin/rsync instead of /usr/bin/rsync. From what I have read, the newer builds of rsync on the main trunk have better support for ACLs and extended attributes than the version that Apple shipped with Tiger. Something you may want to look into for yourself. As I mentioned above, though, I have no idea if this is still the case with Leopard.

unless rsync was installed in /usr/local ???:confused:

a simple command will tell you its path.

whereis rysnc

Already determined above....

I removed "/local" from the path to rsync, and tried to create and run a new script. When I did, I got the following error:

rsync: --eahfs: unknown option
rsync error: syntax or usage error (code 1) at /SourceCache/rsync/rsync-30/rsync/main.c(1099)

Here is the update script I used:

#rotating backup script - v1.0
time /usr/bin/rsync --rsync-path=/usr/bin/rsync -az --eahfs --showtogo --exclude=/Users/$USER/Library --exclude=/Users/$USER/Pictures --exclude=/Users/$USER/Music --exclude=/Users/$USER/Movies --exclude=/Users/$USER/Sites --exclude=/Users/$USER/.Trash --exclude=/Users/.DS_Store --link-dest="/Volumes/$USER/My Backups/Archive/" "/Users" "/Volumes/$USER/My Backups/Archive/"

Again I cannot stress how much I appreciate the help her. I would be at a total loss without everyone here. Thanks again.

In the stock rsync, the -h option shows the help, so there's a pretty good chance that, coupled with the alternate location of rsync originally specified, that your predecessor compiled a different version of rsync on the client machines.

Thanks again for the quick reply Mikey. I guess my next question would be, what would be my next course of action? Thanks,

-Dom

Did your predecessor leave any documentation? A contact number/e-mail address? Anyone in the office know him on AIM/MSN/Yahoo/whatever?

It sounds dumb, yeah, but if you can ask him what he did and why, you can write it down yourself and know exactly what's going on in your environment.

No he did not document anything and that has been my greatest challenge this week, trying to figure out what the heck he had going on. Contacting him is probably not a good idea, as he really didn't know he was being replaced.

Yeh... the "eahfs" [extended attributes hierarchal file system] option is now simply -E (or --extended-attributes)
[maybe this script used some "special build" of rsync]

Also, that "--showtogo" option doesn't appear in the current rsync man page.

The rest looks like it might fly.

[odd... the script is excluding only one .DS_Store file. :)
changing that to --exclude=.DS_Store will skip them all.]

Just be aware that: all stuff in Music, Movies, Pictures, Library, Sites, etc., is not backed up there!

-HI-

Thanks for the feedback Hal! So I need to change "eahfs" to just "-E" and get rid of "showtogo?" And changing the exclusion to "--exclude=.DS_Store" should get me on the right track? I am getting more confused about this by the minute. I definitely appreciate all the help I can get!

The best course of action here really depends on your environment. If the macs in your work environment all run the same version of the OS (Tiger for example), then you can probably adapt your predecessor's script to run with that version of rysnc as Hal suggests above. However, if the macs you will be supporting run different versions of the OS (some on Panther, some on Tiger, some on Leopard), then I would suggest downloading and installing rsync from the main development site (as it would seam that your predecessor did), since different versions of rsync (with different flags and support options) shipped with each version of OS X.

That's about the size of it (AFAIK).

There is an option there: --link-dest="/Volumes/$USER/My Backups/Archive/"
which creates hard links for unchanged files. I can't say whether or not this
script -- in totality -- is fully configured to do rotating backups (or whatever).

On the surface, it doesn't appear so. [is there more to this script?]

Anyway, it appears he's trying to do something like this...

Time Machine For Tiger
Time Machine for every Unix out there

...and I simply haven't tested rsync to that extent, to comment constructively.

--

Do we know yet which version you have?

Thanks again for the help gang. My rsync is version 2.6.3. protocol version 28. That being said I am getting really frustrated with this. If I continue to have problems , I might have to look for another solution and just start fresh. That being said, all of the machines that would be using this solution are running Tiger 10.4.11.

If I can't get this working does anyone have a good backup suggestion? We have a 500gb server here and each person has their own folders, it would just be a matter of getting their files to backup to their folders. Again I appreciate everyones help, I have been feverishly trying to learn everything I can about Mac OS X Server and fast!

If scripting rsync is too tedious, maybe the company might shell out some bucks for a
commercial product... such as TOLIS Group's Backup & Restore Utility (BRU) for example.

rysnc is excellent but not without its bugs....

Carbon copy cloner beta 3.1.1 has rysnc 3.02 within the package and i would advise using 3.02 and install into /usr/local/bin. Make sure you chown it to root:wheel and check it is 755. The built in rsync 2.6.6 (/usr/bin) has plenty of issues especially if you ACLs and finder Meta data.

I also think BRU is a very good solution. We phased out Retrospect a while ago and it has performed very well.

However once you get your head round rsync, launchd and terminal stuff it is brilliant. Here is a good article that got me on the right track.

good luck

I had a look at Retrospect and it might be a viable option. That being said I was directed to RsyncX by another person. I downloaded it and installed and went through the wizard to set up a revolving backup. However I can only backup one thing at a time with it.

yes rsyncx has not worked for me either. The version is fairly depreciated and uses an old version of rsync. As i said rsync 3.02 has resolved many issues i was having with finder meta data and ACL's.

but it does require some serious reading up !

My company does not mind spending some money on retail software if it means that we will have a solution. That being said I think Retrospect seems to make sense as long as I am able to setup a weekly backup that will run automatically and backup documents, desktop, public, and shared folders on each machine to our server. If it can do that then I can pitch it and get working on it! That being said I am always open to learn more especially about macs, but this is an issue that needs a quick resolution. Then later on I can learn to work with rsync, terminal, etc...

I don't think you ever answered one of my original questions...

How are these Macs managed? Do they all authenticate against an ODM, and are their home directories local or network folders?

Sorry tlarkin, I am not sure what you mean by an ODM. Their home directories are networked folders if I understand. Each person has to "connect to server" to reach their folders. Again I apologize if I am being unclear, I am totally new to Mac servers. I am kinda flying by the seat of my pants right now. :(

ODM = Open Directory Master

Do the homes live on the Mac server?

If so you can manage them to background sync or run scripts from the server side GUI. Or you could make a launchd item run a rsync script every day, or once a week or whatever.

I have a 2 TB FW drive sitting on a few servers that gets a back up tossed onto it. All done via cron, launchd, and simple scripts.

Leopard server (which I am still learning) has more robust GUI options with in server admin and WGM.