Go Back   The macosxhints Forums > OS X Help Requests > UNIX - General



Reply
 
Thread Tools Rate Thread Display Modes
Old 04-17-2008, 01:49 PM   #1
hunzinker
Triple-A Player
 
Join Date: Apr 2006
Posts: 59
Rsync Error Code 23 Kills Script

Hi,

I am using an Automator workflow running a rsync shell script to backup multiple computers to a NAS server on the LAN. During this process rsync displays the error code "rsync error: some files could not be transferred (code 23)" and it kills my workflow. Does anybody have any advice on how would I go about modifying my shell script to tell rsync to ignore or log such errors in a text file?

Current script:
rsync -aE --delete ~/(home directory) /Volumes/(NAS Server Partition)/

I am using the bash shell in Mac OS 10.5.2. Any help would be much appreciated.

Thanks
hunzinker is offline   Reply With Quote
Old 04-17-2008, 02:18 PM   #2
tlarkin
League Commissioner
 
Join Date: Mar 2003
Location: Bay Area, CA
Posts: 11,351
is it over SMB or AFP to the NAS?

Both require authentication you may need to toss in a username and password into your script.
tlarkin is offline   Reply With Quote
Old 04-17-2008, 02:24 PM   #3
hunzinker
Triple-A Player
 
Join Date: Apr 2006
Posts: 59
It is over AFP.

Yes I have the username and password in the workflow. The problem is not the connection, it's just the rsync error code 23 kills my workflow when backing up select computers.

Some computers don't throw the error, so the workflow executes perfectly.

Thanks
hunzinker is offline   Reply With Quote
Old 04-17-2008, 02:38 PM   #4
tlarkin
League Commissioner
 
Join Date: Mar 2003
Location: Bay Area, CA
Posts: 11,351
Try putting afp://ip.of.nas/shared/folder

See if that helps
tlarkin is offline   Reply With Quote
Old 04-17-2008, 07:21 PM   #5
hunzinker
Triple-A Player
 
Join Date: Apr 2006
Posts: 59
Thanks tlarkin, but the problem doesn't lie with the afp connection. The connections are working properly and connecting to server doesn't cause the rsync error. The connection takes place before the shell script is executed.

Rsync displays the error mostly after or during a backup.

Thanks
hunzinker is offline   Reply With Quote
Old 04-17-2008, 08:23 PM   #6
cwtnospam
League Commissioner
 
Join Date: Jan 2005
Posts: 8,475
From the man pages for rsync, under Exit Values:
Code:
       23     Partial transfer due to error
http://fvue.nl/wiki/Bash:_Error_handling

This one seems to explain traps better:
http://www.davidpashley.com/articles...l-scripts.html

Last edited by cwtnospam; 04-17-2008 at 08:33 PM.
cwtnospam is offline   Reply With Quote
Old 04-20-2008, 07:34 PM   #7
hunzinker
Triple-A Player
 
Join Date: Apr 2006
Posts: 59
No worries - figured it out! After the rsync script include this bit of code

> (file_name) 2>&1

Where file_name is the name of your file 2>&1 directs all output to the file.
hunzinker is offline   Reply With Quote
Old 09-01-2008, 09:14 PM   #8
Nuke
Triple-A Player
 
Join Date: Jan 2005
Posts: 72
Hello hunzinker

Did you ever figure out why you were getting this error? I am having the same problem rsync'ing to a firewire drive. I did the send info to log file but I can't find any error except at the end where it states

Quote:
sent 104420322 bytes received 281320 bytes 2463568.05 bytes/sec
total size is 40879470266 speedup is 390.44
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-35.2/rsync/main.c(992) [sender=2.6.9]

The transfer was so fast it couldn't have done all the transfer....

I have run rsync multiple times and it dies in exactly the same spot.

Any suggestions would be appreciated.

Thanks in advance.
Nuke is offline   Reply With Quote
Old 09-02-2008, 10:10 PM   #9
hunzinker
Triple-A Player
 
Join Date: Apr 2006
Posts: 59
Hi Nuke,

To answer your question, No I haven't narrowed the rsync error down to anything specific, but I have found a few clues.

Here are few possible explanations that may cause rsync to throw error code 23.
- Permission problems
- Using *NIX unfriendly characters in directory / filenames

I have tired using --delete-after and --ignore-errors to combat these issues. Sometimes they work and sometimes they don't. If you come across anything I haven't tried here, or you have any suggestions, please let me know.

Again, it's very strange to me because I successfully backup my machine daily via Firewire, while the scripts I use over AFP work somedays and fail others. It would be nice to review the TCP dump on both nodes during the script, but I haven't figured out how to do that yet.

If anything else comes up, I'll be sure to post it.

Thanks
hunzinker is offline   Reply With Quote
Old 09-04-2008, 01:13 PM   #10
Nuke
Triple-A Player
 
Join Date: Jan 2005
Posts: 72
Hi Hunzinker

Thank you for your reply.

If I am running sudo rsync would that not alleviate the "permissions problem"? Or are there other permissions that I might not know about/understand?

I'm having the problem doing the daily backup to a firewire drive. Since it works OK for you, I'll have to do some more testing.

Is there another way to get a list of file errors other than the > (file) > 2&1? I'm not getting a full list as error 23 happens too quickly but at the same point. Would the last file in the list be the problem?

Thanks again and regards.
Nuke is offline   Reply With Quote
Old 09-04-2008, 01:37 PM   #11
hunzinker
Triple-A Player
 
Join Date: Apr 2006
Posts: 59
Hi Nuke,

Quote:
If I am running sudo rsync would that not alleviate the "permissions problem"?

I am not sure about this, but my thoughts are no.

Quote:
Is there another way to get a list of file errors other than the > (file) > 2&1? I'm not getting a full list as error 23 happens too quickly but at the same point.

Try running rsync with the --progress flag or -vv (Use one for v flag for a summary and two for what files are being and why.) in the command line for more information during transfer.

Quote:
Would the last file in the list be the problem?

Just make sure you don't have any single quotes, double quotes, back slashes etc. in your directory or file names. I have had rsync die once it encounters a "bad" character.

What does your rsync command look like? Are your files backing up? Rsync will display error code 23 even if the rest of your files are being transfered. Give it a run using -vv and see what happens.
hunzinker is offline   Reply With Quote
Old 09-04-2008, 01:54 PM   #12
Nuke
Triple-A Player
 
Join Date: Jan 2005
Posts: 72
Thanks, I'll try and run these with the additional options tonight when I get home.

The command I am running is <sudo rsync -avE /Users /Volume/BKUP/>

/Volume/BKUP/Users folder is to be sync'd with the /Users folder.

Thanks again.
Nuke is offline   Reply With Quote
Old 09-07-2008, 12:58 PM   #13
Nuke
Triple-A Player
 
Join Date: Jan 2005
Posts: 72
Hi!

I ran the command:
Code:
sudo rsync -avvE /Users /Volumes/Files/ > logsync.txt 2>&1
The log says:
Quote:
total: matches=0 hash_hits=0 false_alarms=0 data=1969871441

sent 1972631965 bytes received 538276 bytes 10197262.23 bytes/sec
total size is 40912319484 speedup is 20.73
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-35.2/rsync/main.c(992) [sender=2.6.9]

Every file that I can see (46265 lines) says "is uptodate". But I have been changing files for a week and nothing appears to be copying by rsync??? The process takes a minute or two to run.

This doesn't make sense. I feel like bashing my head against the wall.
Nuke is offline   Reply With Quote
Old 09-09-2008, 04:34 PM   #14
hunzinker
Triple-A Player
 
Join Date: Apr 2006
Posts: 59
Hi Nuke,

Quote:
Every file that I can see (46265 lines) says "is uptodate". But I have been changing files for a week and nothing appears to be copying by rsync???

I have experienced this issue in the past, but only when running rsync over afp. Try the following:

Change this line of code

Quote:
sudo rsync -avvE /Users /Volumes/Files/ > logsync.txt 2>&1

to

rsync -avvE /Users /Volumes/Files/<Some New Directory>

just to see if rsync will copy the files. If it works, great, if not, hmmmm.
hunzinker is offline   Reply With Quote
Old 09-10-2008, 08:03 AM   #15
Nuke
Triple-A Player
 
Join Date: Jan 2005
Posts: 72
Thanks for your suggestions. I'll try it tonight when I get home.

I have compiled rsync 3.0.4 and installed in /usr/local/bin yesterday to see if the build in rsync 2.6.9 is part of the problem.

I'll provide an update based on these tests.

Regards
Nuke is offline   Reply With Quote
Old 09-17-2008, 11:29 AM   #16
Nuke
Triple-A Player
 
Join Date: Jan 2005
Posts: 72
Update to my last comment re use of rsync 3.0.4.

I tried to rsync the folders again but used rsync 3.0.4 and still end up with error code 23.

Hmmmm. I have to ponder what to try next.
Nuke is offline   Reply With Quote
Old 06-15-2009, 11:34 AM   #17
Pandrake
Prospect
 
Join Date: Jun 2009
Posts: 2
rsync error

I just now happened upon this thread, and have been having the same problems.

I found the solution was to use a directory other than the NAS for temporary files, which rsync apparently creates in the destination directory by default. I'm guessing that the NAS drive can't handle long file names or special characters that are used for the temporary files.

rsync -aE --delete --temp-dir="~/(home directory)/Desktop/" ~/(home directory) /Volumes/(NAS Server Partition)/
Pandrake is offline   Reply With Quote
Old 06-16-2009, 08:43 AM   #18
Nuke
Triple-A Player
 
Join Date: Jan 2005
Posts: 72
Thank you for the suggestion. I'll give it a try.

One question though.

If you are using ~/Desktop as the temp directory, what keeps it from trying to copy this temp directory over to the NAS?

In the past, I use --exclude to keep ensure that I don't get Caches, /Volumes/ etc. from rsync'ing over to the NAS. My initial thought is that using ~/Desktop as a temp directory would be a problem??

Should this command also include --exclude=~/Desktop/(tempdir)????

Thank you.
Nuke is offline   Reply With Quote
Old 06-16-2009, 09:14 AM   #19
hunzinker
Triple-A Player
 
Join Date: Apr 2006
Posts: 59
Hi,

I stopped using AppleScript to execute rsync for my weekly backups. Instead, I wrote a bash script and now everything works perfectly. Error code 23 is still present, but the backup executes 100%.

I placed my .sh file in a directory that is not included in the backup path since I only care about the home directory ~/. Instead I place my backup script in /backup and log my errors in /backup/errors.

Maybe that will help.
hunzinker is offline   Reply With Quote
Old 06-19-2009, 06:36 PM   #20
Pandrake
Prospect
 
Join Date: Jun 2009
Posts: 2
Quote:
Originally Posted by hunzinker
Hi,

I stopped using AppleScript to execute rsync for my weekly backups. Instead, I wrote a bash script and now everything works perfectly. Error code 23 is still present, but the backup executes 100%.

Quote:
Originally Posted by Nuke

If you are using ~/Desktop as the temp directory, what keeps it from trying to copy this temp directory over to the NAS?

In the past, I use --exclude to keep ensure that I don't get Caches, /Volumes/ etc. from rsync'ing over to the NAS. My initial thought is that using ~/Desktop as a temp directory would be a problem??

Should this command also include --exclude=~/Desktop/(tempdir)????

AppleScript can also execute the rsync and tell it to ignore errors, but that defeats the purpose of any error letting you know that something has gone wrong even if that "wrongness" doesn't matter. I had thought of that approach to the problem but decided to look into why there was an error that doesn't effect the backup rather then ignore all errors.

I don't believe these "temp" files are files that are being backed up and so whether you use the desktop as a temp directory or the default of the target directory for rsync's temp files won't matter. I'm not really certain that's what they are, but can see no other reason why I get errors on files that don't exist in either the source or the target directories when the target directory is a NAS with a 32 character filename limit but when I tell rsync to use a directory that allows 32 character filenames for it's own temp directory there are no errors.
Pandrake is offline   Reply With Quote
Reply

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



All times are GMT -5. The time now is 09:11 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
Site design © IDG Consumer & SMB; individuals retain copyright of their postings
but consent to the possible use of their material in other areas of IDG Consumer & SMB.