back up/ file compression - Page 3

bluehz · 06-01-2002, 04:28 PM

Per hints here and elsewhere I finally decided to d/l hsfpax and try it. PIcked up the latest version - 1fc4. No compile necessary - just install in /usr/bin. The drop scripts work ok - but in the terminal I am getting no response from the binary. I have tried:

hfspax --help
hfspax usage
hfspax
et.al.......

all of those lead to basically nothing in the terminal that needs to be ctrl+c to get out of it. I am not sure that hfspax is properly working. Seems like at the bare minimum I should be able to get basic usage info on it. Any one have any tips? FYI - I did check the permissions on it and set them appropriately.

mervTormel · 06-01-2002, 04:44 PM

did you install the man pages?

$ man -w hfspax
/usr/local/share/man/man1/hfspax.1

if that's not there, you'll want it there so you can

$ whatis hfspax
hfspax(1) - read, write file archives, copy directory hierarchies, HFS+ support

$ man hfspax
...

hfspax doesn't have usage spew or --help, it'll just try to swallow whatever stdin is.

btw, those things you d/l and install should get into /usr/local/ because
/usr/local/ is guaranteed not to be stepped on by a vendor (apple).

bluehz · 06-02-2002, 01:07 AM

Thx - thats what I needed to know. I just found it odd that it output nothing when you simply enter "hfspax". Most binaries do.

mervTormel · 06-02-2002, 01:17 AM

yeah, but most binaries weren't written in a haze of bong smoke. well, ya know, it's getting hard to tell. at fist glance and the mention of Berkeley, many think unix is the result of coding under the influence of some bodacious peyote.

the 'ideas' behind unix tools are usually governed by a 'convention' but there are no hard rules. just look at tar. there must have been good reason for this odd 'convention' but i can't imagine what it is.

bluehz · 06-02-2002, 01:23 AM

I have been playing around with psync, hfspax, and rsync - my personal fav. I think I still prefer rsync as it allows some filtering to allow a subset selection within a set (e.g. filters).

I must admit though that psync is VERY fast if you just want to make a straight duplicate.

Only beef I have with rsync is it does not copy simple Mac type alias well - it creates a 0k file with the same name in the destination directory. I have tried many an incantation with the various options and never have managed to get it to work - if anyone knows a method - sure would like to hear.

ashevin · 06-02-2002, 09:39 AM

merv: Both BSD and LSD come from Berkeley... Not everyone believes this is a coincidence

bluehz: Rsync doesn't understand Mac resources, of which Aliases are one. You won't be able to get it to handle them correctly without changing the source.

[mildly offtopic]

Anyone know where I can find documentation on Mac OS (HFS in particular) filesystem coding? Or should I ask in the developer forum?

- Avi

bluehz · 06-02-2002, 09:44 AM

I have no problems at all copying rsrc with rsync....I have rscyned between a G4 and an iBook for a while now, apps, system, everything.

ashevin · 06-02-2002, 09:49 AM

I didn't know that rsync had a -rsrc flag. Does it come with OS X? (I suppose I should check for myself, but I'm lazy, so nyeh!

)

Perhaps aliases don't work because they don't appear to have any contents to the unix subsystem, as opposed to resource forks, which appear as regular directories and files.

- Avi

bluehz · 06-02-2002, 10:31 AM

It does not have a -rsrc flag, but by default it appears to do just fine with resources. I think your deduction is correct about the alias have 0 length data.

ashevin · 06-02-2002, 02:15 PM

Doesn't have a -rsrc flag? Are you sure it's backing things up correctly then? rsync an app bundle and see if the resulting directory is still treated as a bundle.

- Avi

sao · 06-02-2002, 02:33 PM

ashevin,

You can find some FileSystem documentation here:

http://developer.apple.com/techpubs/...and_Fonts.html

http://developer.apple.com/techpubs/...Organized.html

http://developer.apple.com/techpubs/...dationTOC.html

Cheers...

ashevin · 06-02-2002, 03:01 PM

Not quite what I was looking for, sao, but it pointed me in the right direction.

Thanks,

- Avi

mervTormel · 06-02-2002, 03:36 PM

Quote:


	Originally posted by bluehz ...Only beef I have with rsync is it does not copy simple Mac type alias well - it creates a 0k file with the same name in the destination directory. I have tried many an incantation with the various options and never have managed to get it to work - if anyone knows a method - sure would like to hear.

this is troubling news as there are other zero byte files that carry their brains in the rez fork...

Code:

$ ls -lG ~/Library/Preferences/DragThing\ Preferences{/rsrc,}
-rwxr--r-- 1 501        0 Jun  1 01:19 DragThing Preferences
-rwxr--r-- 1 501  4827627 Jun  1 01:19 DragThing Preferences/rsrc

bluehz, ya might wanna script something that looks for this occurrence in your source and discover the integrity of hits in your backup, besides aliases.

i suppose it would be a useful scipt for all to validate their backups. perhaps we can get one started...

pseudo-code:

find files in source with resource forks
restore those files from the backup to some tmp space
cmp the source and tmp files

i think that would be a pretty thorough test. if that cmp makes a peep, backup no good.

mervTormel · 06-03-2002, 05:24 AM

here's a crude, rude, and snarky little script to find files with rez forks. it needs improvement and it'll make your rig sweat.

Code:

$ cat grez

#!/bin/sh

set +o noclobber

tmpfs=/tmp/$UID/gotRez.$$

echo '
#!/bin/sh \
local fs="$@" ; \
[ -s "${fs}/rsrc" ] && /bin/ls -l "${fs}/rsrc" ; \       
' > $tmpfs

/bin/chmod +x $tmpfs

/usr/bin/find "$@" -type f -print0 | /usr/bin/xargs -0 -n1 $tmpfs

rm $tmpfs

i wish i could make xargs swallow just some scriptage out there, but it woodn't, ergo, the tmp file wonkery.

feed it a directory ref (perhaps just ~/Library to start) and stand back.

Code:

$ ./grez .
-rw-r--r--  1 merv  test  2347 May 31 23:11 ./foo.txt/rsrc
-rw-r--r--  1 merv  test  492 May 31 23:13 ./foo2/rsrc
-rwxr--r--  1 merv  staff  4827627 Jun  2 21:55 ./rezzy doo/rsrc

the intentions here are to assist in checking the validity of a backup with your chosen method. so, fill a dir with files known to have rez forks, backup them, then restore them somewhere innocuous and check them against the originals. roger that?

--
Ferro: **** "Switch to DCS ranging."
Spunkmeyer: "Two-Four-Oh. Nominal to profile."
Ferro: *****"We're in the pipe, five by five."

Well that's great, that's just ****in' great man, now what the **** are we supposed to do? We're in some real pretty **** now man... That's it man, game over man, game over, man! Game over! What the **** are we gonna do now? What are we gonna do? -- Private W. Hudson

mervTormel · 06-04-2002, 04:03 AM

something happened today that causes me great pause in our current backup techniques.

<aside>
i was testing some shell commands piped to xargs and i would get "segment violation" messages in the shell! and xargs crash logs in ~/library/logs/ . so, out of "bad" habit, i eventually rebooted. this provided no relief with my shell command, same error (it's a bug in xargs and my args to it).
<aside over>

but my dragthing prefs were returned to stock (my fault, mv dt prefs to ~/work). nothing of the <time> of hard work in creating my dragthing palettes was around, except in my backup. but, the app continued to work with my personal prefs in memory.

if i hadn't rebooted and discovered this debacle, my ditto backup would have gladly written my new stock dt prefs over my personal dt prefs in my backup target. i would have lost my "version"

this lack of differential incremental backup is a pox upon us. be very wary about the integrity of your backups.

JayBee · 06-04-2002, 06:28 AM

Bah - incremental backups are the devil's backups

Fresh dailies/weeklies may take a lot of space, but dang they're worth it!

stetner · 06-05-2002, 08:33 AM

Quote:


	Originally posted by sao Also, I you would like to use hfspax, I found 'genecutl' wrote a nice perl script for making incremental MacOSX Backups with hfspax. Here is his web page, so you can check it up : http://smalltime.com/gene/bax.html

I have been using hfspax for a while and it is pretty good. Only thing that it has problems with is that folder (and only folder) icons do not 're-attach' to folders on the restore. The icons are there, same size, same permissions both in UNIX and with GetFileInfo, but the folder looks like a plain folder in the finder.

Being a long time UNIX hacker I liked the dump/restore programs and liked the way bax mimics it. I did find a bug in the perl script:

Code:

if ($prev_level <= $BACKUP_LEVEL) {

in the get_last_backup_date function should read:

Code:

if ($prev_level le $BACKUP_LEVEL) {

and in fact I prefer it to be:

Code:

if ($prev_level lt $BACKUP_LEVEL) {

which will make it more like dump/restore. Anyway, I told Gene about the bug and he has acknowledged it. (I checked t see if he had updated the code on the web site and it won't let me down load it now, so he probably has fixed it but forgot to set permissions right on the file, I will email him after this 8-).

Anyway, I am going to use it until I get a working dump/restore (I will just hack up the code in my spare time ha ha!).

sao · 06-05-2002, 08:59 AM

Stetner,

Thanks a lot for discovering the bug.

Cheers...

PS:-- (Could be good if you could post your hack...)

stetner · 06-05-2002, 09:12 AM

Quote:


	Originally posted by sao PS:-- (Could be good if you could post your hack)

Quote:


	(I will just hack up the code in my spare time ha ha!)

Thats sarcasm Sao! Hell will probably freeze over before I get time to look at the dump/restore code 8-)

sao · 06-05-2002, 09:34 AM

stetner,

Yes, I realized that, after I posted...

Cheers...

06-02-2002, 01:23 AM	#45
bluehz MVP Join Date: Jan 2002 Posts: 1,562	I have been playing around with psync, hfspax, and rsync - my personal fav. I think I still prefer rsync as it allows some filtering to allow a subset selection within a set (e.g. filters). I must admit though that psync is VERY fast if you just want to make a straight duplicate. Only beef I have with rsync is it does not copy simple Mac type alias well - it creates a 0k file with the same name in the destination directory. I have tried many an incantation with the various options and never have managed to get it to work - if anyone knows a method - sure would like to hear. Last edited by bluehz; 06-02-2002 at 07:00 AM.

06-02-2002, 02:33 PM	#51
sao Moderator Join Date: Jan 2002 Location: Singapore Posts: 4,237	ashevin, You can find some FileSystem documentation here: http://developer.apple.com/techpubs/...and_Fonts.html http://developer.apple.com/techpubs/...Organized.html http://developer.apple.com/techpubs/...dationTOC.html Cheers... Last edited by sao; 06-02-2002 at 02:52 PM.

06-03-2002, 05:24 AM	#54
mervTormel League Commissioner Join Date: Jan 2002 Posts: 5,536	here's a crude, rude, and snarky little script to find files with rez forks. it needs improvement and it'll make your rig sweat. Code: $ cat grez #!/bin/sh set +o noclobber tmpfs=/tmp/$UID/gotRez.$$ echo ' #!/bin/sh \ local fs="$@" ; \ [ -s "${fs}/rsrc" ] && /bin/ls -l "${fs}/rsrc" ; \ ' > $tmpfs /bin/chmod +x $tmpfs /usr/bin/find "$@" -type f -print0 \| /usr/bin/xargs -0 -n1 $tmpfs rm $tmpfs i wish i could make xargs swallow just some scriptage out there, but it woodn't, ergo, the tmp file wonkery. feed it a directory ref (perhaps just ~/Library to start) and stand back. Code: $ ./grez . -rw-r--r-- 1 merv test 2347 May 31 23:11 ./foo.txt/rsrc -rw-r--r-- 1 merv test 492 May 31 23:13 ./foo2/rsrc -rwxr--r-- 1 merv staff 4827627 Jun 2 21:55 ./rezzy doo/rsrc the intentions here are to assist in checking the validity of a backup with your chosen method. so, fill a dir with files known to have rez forks, backup them, then restore them somewhere innocuous and check them against the originals. roger that? -- Ferro: ** "Switch to DCS ranging." Spunkmeyer: "Two-Four-Oh. Nominal to profile." Ferro: *"We're in the pipe, five by five." Well that's great, that's just in' great man, now what the are we supposed to do? We're in some real pretty now man... That's it man, game over man, game over, man! Game over! What the ** are we gonna do now? What are we gonna do? -- Private W. Hudson Last edited by mervTormel; 06-03-2002 at 06:08 AM.

06-04-2002, 04:03 AM	#55
mervTormel League Commissioner Join Date: Jan 2002 Posts: 5,536	backup ate my scooby snacks something happened today that causes me great pause in our current backup techniques. <aside> i was testing some shell commands piped to xargs and i would get "segment violation" messages in the shell! and xargs crash logs in ~/library/logs/ . so, out of "bad" habit, i eventually rebooted. this provided no relief with my shell command, same error (it's a bug in xargs and my args to it). <aside over> but my dragthing prefs were returned to stock (my fault, mv dt prefs to ~/work). nothing of the <time> of hard work in creating my dragthing palettes was around, except in my backup. but, the app continued to work with my personal prefs in memory. if i hadn't rebooted and discovered this debacle, my ditto backup would have gladly written my new stock dt prefs over my personal dt prefs in my backup target. i would have lost my "version" this lack of differential incremental backup is a pox upon us. be very wary about the integrity of your backups.

06-04-2002, 06:28 AM	#56
JayBee Major Leaguer Join Date: Jan 2002 Location: Edinburgh, Scotland Posts: 437	Bah - incremental backups are the devil's backups Fresh dailies/weeklies may take a lot of space, but dang they're worth it! __________________ JayBee -- It's all relative, you know

06-01-2002, 04:28 PM	#41
bluehz MVP Join Date: Jan 2002 Posts: 1,562	Per hints here and elsewhere I finally decided to d/l hsfpax and try it. PIcked up the latest version - 1fc4. No compile necessary - just install in /usr/bin. The drop scripts work ok - but in the terminal I am getting no response from the binary. I have tried: hfspax --help hfspax usage hfspax et.al....... all of those lead to basically nothing in the terminal that needs to be ctrl+c to get out of it. I am not sure that hfspax is properly working. Seems like at the bare minimum I should be able to get basic usage info on it. Any one have any tips? FYI - I did check the permissions on it and set them appropriately.

06-01-2002, 04:44 PM	#42
mervTormel League Commissioner Join Date: Jan 2002 Posts: 5,536	did you install the man pages? $ man -w hfspax /usr/local/share/man/man1/hfspax.1 if that's not there, you'll want it there so you can $ whatis hfspax hfspax(1) - read, write file archives, copy directory hierarchies, HFS+ support $ man hfspax ... hfspax doesn't have usage spew or --help, it'll just try to swallow whatever stdin is. btw, those things you d/l and install should get into /usr/local/ because /usr/local/ is guaranteed not to be stepped on by a vendor (apple).

06-02-2002, 01:07 AM	#43
bluehz MVP Join Date: Jan 2002 Posts: 1,562	Thx - thats what I needed to know. I just found it odd that it output nothing when you simply enter "hfspax". Most binaries do.

06-02-2002, 01:17 AM	#44
mervTormel League Commissioner Join Date: Jan 2002 Posts: 5,536	yeah, but most binaries weren't written in a haze of bong smoke. well, ya know, it's getting hard to tell. at fist glance and the mention of Berkeley, many think unix is the result of coding under the influence of some bodacious peyote. the 'ideas' behind unix tools are usually governed by a 'convention' but there are no hard rules. just look at tar. there must have been good reason for this odd 'convention' but i can't imagine what it is.

06-02-2002, 09:39 AM	#46
ashevin Triple-A Player Join Date: Jan 2002 Posts: 158	merv: Both BSD and LSD come from Berkeley... Not everyone believes this is a coincidence bluehz: Rsync doesn't understand Mac resources, of which Aliases are one. You won't be able to get it to handle them correctly without changing the source. [mildly offtopic] Anyone know where I can find documentation on Mac OS (HFS in particular) filesystem coding? Or should I ask in the developer forum? - Avi

06-02-2002, 09:44 AM	#47
bluehz MVP Join Date: Jan 2002 Posts: 1,562	I have no problems at all copying rsrc with rsync....I have rscyned between a G4 and an iBook for a while now, apps, system, everything.

06-02-2002, 09:49 AM	#48
ashevin Triple-A Player Join Date: Jan 2002 Posts: 158	I didn't know that rsync had a -rsrc flag. Does it come with OS X? (I suppose I should check for myself, but I'm lazy, so nyeh! ) Perhaps aliases don't work because they don't appear to have any contents to the unix subsystem, as opposed to resource forks, which appear as regular directories and files. - Avi

06-02-2002, 10:31 AM	#49
bluehz MVP Join Date: Jan 2002 Posts: 1,562	It does not have a -rsrc flag, but by default it appears to do just fine with resources. I think your deduction is correct about the alias have 0 length data.

06-02-2002, 02:15 PM	#50
ashevin Triple-A Player Join Date: Jan 2002 Posts: 158	Doesn't have a -rsrc flag? Are you sure it's backing things up correctly then? rsync an app bundle and see if the resulting directory is still treated as a bundle. - Avi

06-02-2002, 03:01 PM	#52
ashevin Triple-A Player Join Date: Jan 2002 Posts: 158	Not quite what I was looking for, sao, but it pointed me in the right direction. Thanks, - Avi

06-05-2002, 08:59 AM	#58
sao Moderator Join Date: Jan 2002 Location: Singapore Posts: 4,237	Stetner, Thanks a lot for discovering the bug. Cheers... PS:-- (Could be good if you could post your hack...) Last edited by sao; 06-05-2002 at 09:25 AM.

06-05-2002, 09:34 AM	#60
sao Moderator Join Date: Jan 2002 Location: Singapore Posts: 4,237	stetner, Yes, I realized that, after I posted... Cheers...