scripting 'touch' for mass file changes

[Spiff]

I know this is possible.

I just don't know how to do it.

any help would be appreciated.


Situation: I have 12,000 files uniquely named in the following format:

<YY.MM.DD.RR_##>

YY=2 number year
MM=2 number month
DD=2 number day

RR_## are irrelevant, but included for completeness.

What I want to do is to set the creation, access, and modification date of each file to the same as its name.

i.e.

a file with the name '99.04.01.01 02'

would have the modification/creation/access date set to 99/04/01

these files are all embedded in nested directories, up to 4 levels deep.

so how do I do this?

thx for any help

[honestpuck]

This would be best done in Perl.

While it is feasible to do it in a shell script it wouldn't be easy. If you do want to do it in the shell here are a couple of hints :-

If you echo the filename to awk you can get the right time to feed to touch, i.e.

echo [filename] | awk -F \. '{print 19$1$2$3 1101}')`

so

touch -t `(echo [filename] | awk -F \. '{print 19$1$2$3 1101}')` [filename]

(replacing '[filename]' with the name of the file)

will do it once you find a way of getting the filename into that command.

Tony Williams

[mervTormel]

yeah, this is prolly pretty slick with perl, and i have no idea how to start, but i would like to.

i'll see if i can't get our perl guru to chime in here, so hold on...

[jmeador]

Here's my shot at it.


Note1 : You have to change $dir to the path for each nested folder. So you'll run the script 4 times.

Note 2 : the only error checking I'm doing verifying that
the file splits into 4 pieces based on periods and that
the first 3 elements are 00 - 99, 01 - 12 and 01 - 31.

Note 3 : The hour and minute get set to 00.

Note 4 : The correct perl command to set access and mod
date is utime. I don't have a good handle on the syntax so
I used a system call to touch instead.

Good Luck and give me a shout if you have problems or ???s

jas


Merv-

Have your perl guru look it over and give me crits!

Thanks-

jas.

#### auto_touch.pl
#### usage perl auto_touch.pl
####
#### perl script to set creation, access and modification date based on file name.

#### v .2 - 04-09-02 jhm
#### rev added \" around path in touch statement to handle names w/ metachars
#### improved date check code
#### known issues : could still have months w/ too may days -- feb 30, april 31, june 31 etc.

#### v .1 - 04-08-02 jhm
#### known issues : date checking code doesn't work


#### This script is distributed in the hope that it will be useful,
#### but WITHOUT ANY WARRANTY; without even the implied warranty of
#### MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$dir= "/volumes/storage/test/";
opendir DIR, $dir or die "Can't open directory $dir: $!\n";
@files=grep !/^\./, readdir(DIR);
foreach $file_name(@files) {
if (-f $dir . $file_name){
@time = split(/\./, $file_name);
if (4 == scalar @time){
if (($time[0] =~ /[0-9][0-9]/) &&
($time[1] =~ /[0][1-9]|[1][0-2]/) &&
($time[2] =~ /[0-2][1-9]|[3][0-1]/)) {
system("touch -acm -t $time[0]$time[1]$time[2]0000 \"$dir$file_name\"");
}
}
}
}

[mervTormel]

Quote: if (($time[0] =~ /[0-9][0-9]/) && ($time[1] =~ /[0-1][0-2]/) && ($time[2] =~ /[0-3][0-9]/))

jmeador, thanx for the magic...

does that month and day check filter properly?

month = 00, 01, 02, 10, 11, 12 ?
day = 00, 32, 33-39 ?

and what century is being used for year? need to supply CC to touch for
1999 < year < 2000

spiff, it needs thorough testing before unleashing, but, since it doesn't change the file name, you could rerun without damage ?

[jmeador]

yep, there's a bug in the month and day check code for sure...
I'll keep tinkering with it. (or maybe we'll just leave that
as an exercise for the reader )

If you KNOW your file names are good, you could just delete that
"if" and the first "}". To avoid skipping files, it needs to be opened up to atleast :

if (($time[0] =~ /[0-9][0-9]/) &&
($time[1] =~ /[0-1][0-9]/) &&
($time[2] =~ /[0-3][0-9]/))

CC should be okay though, here's a quote from the man pages
on how it decides what to use :

YY The second two digits of the year. If ``YY'' is
specified, but ``CC'' is not, a value for ``YY''
between 69 and 99 results in a ``CC'' value of 19.
Otherwise, a ``CC'' value of 20 is used.

jas.

[mervTormel]

thanx, jas. an exercise for the reader is a good thing...

vanilla OSX's touch man pages don't have that CC blurb, but the finkified touch info pages do. thanks for pointing that out.

Spiff, if you use the OSX vanilla /usr/bin/touch, you might want to test that century presumption.

[honestpuck]

You should probably keep the file check in regardless.

If you want to up this to a recursive descent solution then the easiest way is to wrap the entire script so far in a function and add to the file check 'if' the following as an elseif

} elseif ((-d $dir . $file_name) && !($file_name[0] == '.')) {
auto_touch($dir . $file_name . "/");
}

and then change the $dir = at the top to

$dir = shift;

Oh, and you'll need a function call at the bottom to start the whole thing off

auto_touch("/path/to/start/)";

Tony

[jmeador]

Okay, this date code is a little better :

(($time[0] =~ /[0-9][0-9]/) &&
($time[1] =~ /[0][1-9]|[1][0-2]/) &&
($time[2] =~ /[0-2][1-9]|[3][0-1]/))

but you could still end up w/ feb 30, april 31, etc.

The $dir$filename vars in touch should also be
escaped w/ ", just in case there are any metachars
hiding in there

\"$dir$file_name\"

[Spiff]

this is all fantastic info, thanks a bunch!

in answer to the question 'what is the century range', yes, this will cover the range from 1986 - present, so I guess that has to be included in the script.

So since I am a unix newbie (merely an advanced Mac user/trouble-maker ;-) ), time to dig out Ray's Unleashed book and see if I can follow how to do this with perl, based upon the suggestions so far! =)

[pmccann]

Fun thread, and definitely an interesting problem: here's my attempt at a reasonably safe and complete script to do things. Of course there are millions of other approaches that might work as well or better!

I *hope* that the original poster just needs to change the $dir variable at the top of the script and "press go"; see below for what "press go" means, and reply here if anything kicks up a fuss and I'll try and work out what's going on. In the best perl tradition it hands off some of the work to helper modules, in this case Time::Local to convert date strings to number of seconds since the unix epoch, and File::Find to do the traversal of a directory tree. Note that both of these are already installed as part of the base perl system. Take out the comments and the script becomes pretty compact!

Note that $File::Find::name contains the full path to the current element of the directory walk, and $_ contains just the basename. So the regular expression that sets $year,$month and $day is checking against the basename of the file to see that it starts with two digits, then a period, then two digits and so forth. If the thing matches the first parenthetical chunk --that is, the first two digits-- is thrown into $year, the second into $month and so on and on.

Date checking is (as per the comment) given to Time::Local. It should have no trouble gulping the usual yy.mm.dd format in just the way that you'd like it to, so that 89.03.21 is interpreted as occurring in 1989 and 02.04.10 is today. The actual time changes to the file info are done using "utime", which takes three or more arguments; the first two are the new atime and mtime for the file, and the rest are files to change. I've played it simple here and just done them one by one as they're encountered: it might be better to push found files onto a list and change the times all at once (if you have a whole heap of them with the same date part), but it's not a big deal, and would add a bit of complication. (You'll need to use the *full* path to the files if you want to do this; see $File::Find::name as above.)

Hope this is not too far from what's required, or at least gives someone some nutrition for their noggin,
Paul

To use it, simply cut and paste the script into a file: the line beginning #! **must** be the first line of the file, and make sure that you're using "unix" line endings!

Change the $dir variable in the script to the directory that you wish to start from. Now open a terminal window and move to the directory in which you've saved the script, make the file executable (chmod u+x filename), and finally execute the thing using:

./filename

Please please please point it at a "scratch" directory first: I don't think there's any possibility of real damage, but it's always prudent to let the new kids play in a fenced off pen by themselves for a while (well, at least until they demonstrate that they can play *nicely*).

#!/usr/bin/perl -w
use strict;
use File::Find;   # does the directory traversals
use Time::Local;  # converts to seconds since epoch via "timelocal" function
my $dir="/Users/pmccann/Desktop/thisdir/";  # the directory to start from

############################################################################
# Converts the atime and mtime of any file of the form 99.01.23***** (where
# ***** is anything) to the number of seconds since the unix epoch. Here the 
# filename is supposed to represent the 23rd of January, 1999. See the 
# documentation for Time::Local to see how dates are interpreted: it pretty 
# much does what you'd like it to do, but because it croaks when fed a bad
# date we wrap its use in an eval statement (which returns undef if there's
# a problem, and the number of seconds corresponding to the date otherwise).
# Files whose times are not changed are printed to stdout (under the 
# directory heading where they occur). The change to the atime/mtime dates
# are visible in the terminal via eg "ls -l", and using "Get info" in the
# GUI. Use at your own (relatively minor!) peril
#############################################################################

sub files{
if (-d) 
{ print "\nExamining directory: $File::Find::name\n","-"x72,"\n"; return}; 
if ( my ($year,$month,$day)=($_=~/^(\d\d)\.(\d\d)\.(\d\d)/)){ # eg 01.04.23****
my $seconds=eval "timelocal(1,1,1,$day,$month-1,$year)";
# That's 1.01:01 am on the day in question. $month is converted to the
# range 0 up to 11 as that's what timelocal wants to see.
if(!$seconds) {print "Problem found with date on file: $_\n"; return }
utime($seconds,$seconds,$_) or die "no luck with $_ : $!";
}else{
print "Filename doesn't have required format: $_ \n";
}
}

find(\&files,$dir);  # Go and do it!

[honestpuck]

Yeah, that'll do it.

I thought using File::Find was cheating tho' Why load a module just to do a tree traverse, everyone should write a couple before they take the easy way.

Cool solution. 10 brownie points and applause from the gallery.

Tony

[sao]

A novice asked the Master:

"Here is a programmer that never designs, documents or tests his programs.
Yet all who know him consider him one of the best programmers in the world.
Why is this?"

The Master replied:

"That programmer has mastered the Tao. He has gone beyond the need for design; he does not become angry when the system crashes, but accepts the universe without concern. He has gone beyond the need for documentation; he no longer cares if anyone else sees his code. He has gone beyond the need for testing; each of his programs are perfect within themselves, serene and elegant, their purpose self-evident. Truly, he has entered the mystery of Tao."


pmccann,


'Smooth' as Santana,

great script!


Cheers...

[Spiff]

Hot diggity!!

yep, it worked just fine...

[it=pmccann's script]

mostly

most file dates were changed en masse. =)

some files weren't. Even if I specified the problem directories on their own, it wouldn't change the dates. There doesn't seem to be anything unique about these particular files and their filenames.

the script gives out this error message:

"Problem found with date on file:"

so far as I could tell, the filename itself was fine and in the YY.MM.DD format.

manually doing a 'touch -acm -t' to the files that gave the script a problem resulted in the date being changed appropriately.

odd.

the other addendum to this is, how to get folders with the same name format to be 'touched' too?

thanks so much to everybody for all your help, i really appreciate it!

[pmccann]

Hi again,

not sure what to make of the files that weren't changed: can you post a few of the filenames in question? (Just out of interest; I'm not sure *how* that could happen, and would like to track down the cause!).

Regarding the directory issue: it's just really a matter of removing the "return" from the line after the "-d" (so that finding a directory doesn't *only* lead to a printing out of its name). That is, change that line to read:

{ print "\nExamining directory: $File::Find::name\n","-"x72,"\n"}

(I've removed a couple of optional semi-colons as well.) Note that you will, however, get a "problem with date" printout for every directory *not* of the required form. Maybe this isn't a problem? You could customise that message so that it's more helpful, but it should be pretty clear what's happening. Here's a quick change:

print -d $_?"Directory name":"Filename"," doesn't have required format: $_ \n";

One small thing to note is that the directory named in $dir will be fed to the routine as ".", so even if it has the required form it won't be changed. Again, that shouldn't be a big deal!!

Cheers,
Paul

ps thanks Sao and honestpuck for the kind words (and remember that laziness is the first great virtue of a programmer! As Prog Perl 3rd edn puts it: "Laziness is the quality that makes you go to great effort to reduce overall energy expenditure. It leads you to write labor-saving programs that other people will find useful, and document what you wrote so that you don't have to answer so many questions about it. [...] See also *impatience* and *hubris*")

[Spiff]

ok, here's an example of a file that wasn't changed

Problem found with date on file: 99.09.28.03 06
Filename doesn't have required format: 99.09.28.03 06

...

Problem found with date on file: 99.09.28.02 07
Filename doesn't have required format: 99.09.28.02 07

...

Problem found with date on file: 99.09.28.01 36
Filename doesn't have required format: 99.09.28.01 36

...

etc.

which is still valid as per the original filename format description of

Quote: <YY.MM.DD.RR_##>

example:

the hierarchy of the folders is like this:

00.01.12
---00.01.12.01
-------00.01.12.01 01
-------00.01.12.01 02
-------00.01.12.01 03
-------...
-------00.01.12.01 36
---00.01.12.02
-------00.01.12.02 01
-------00.01.12.01 02
---00.01.12.01 03

in this case, the script correctly set the dates.

if a file has a name:

99.09.28.01 36

the hierarchy is guaranteed to be

99.09.28
---99.09.28.01
-------99.09.28.01 36

does this help?

[pmccann]

Yes, yes, yes!! That now makes perfect sense. My quotient of laziness clearly went through the roof when testing that bit of the script.

What's happening is that when the month is present as 01 up to 07 everything is hunky dory, because then $month-1 makes sense. Unfortunately all the months that I tested were in that range. So what goes haywire when there's 08 or 09 in there? Well, the only reason 01-07 work is that there's a "behind the scenes" octal subtraction happening. (07 is interpreted as octal 7, and you can take one from that to give 6, and localtime is perfectly happy with this). The problem comes with 08 because it's *not* a valid octal number specification. So the line with eval in it returns zero and the error re the format being incorrect results. Harumph!

This doesn't cause difficulties for $year or $day as we're not using either variable "as a number" anywhere; locatime is fed these as strings.

Here's a quick and dirty line that should drive a knife through the heart of the problem: add it in as a new line, directly after the line that begins "if ( my ($year,$month,$day)".

$month=~s/^0//;

Gack! UBE! It just strips a leading zero on $month, if such a thing exists. Sorry about the slip-up, and thanks a lot for the data needed to correct it.

Cheers,
Paul (suitably chastisted)

[Spiff]

'put a knife through the heart of the problem' eh? how gruesome... but effective!

octal conversions? egads...and here I thought octals were only things to be considered with chmod and was otherwise a bizarre 'feature' on scientific calculators.

well we're getting closer and closer to the solution by the looks of things.

I added in the line of code you suggested and a few more thousand files were 'dated' appropriately. So far it looks like 12,000 out of the 14,000 files have been re-dated as I had hoped.

looking good. =)

[you know there's a 'but' coming along right? ]

Quote: Problem found with date on file: 02.03.09 Directory name doesn't have required format: 02.03.09 ... Problem found with date on file: 02.03.09.04 09 Filename doesn't have required format: 02.03.09.04 09 Problem found with date on file: 02.03.09.04 10 Filename doesn't have required format: 02.03.09.04 10 ... Problem found with date on file: 01.07.08.02 08 Filename doesn't have required format: 01.07.08.02 08 Problem found with date on file: 01.07.08.02 09 Filename doesn't have required format: 01.07.08.02 09 ... Problem found with date on file: 00.02.09 Directory name doesn't have required format: 00.02.09 Problem found with date on file: 00.02.09 01 Filename doesn't have required format: 00.02.09 01 ....

I can't spot the common point of failure here.

Thanks again, this is really quite a fun problem and I'm learning tonnes.

[pmccann]

Hi again,

I'm just about running on empty here, and can't really explain why the following works; let me know if we've finally chased away the gremlins. It's a very, very funky little bug creeping in (and I hope to find a decent explanation), but for now try this:

instead of the line $month=~s/^0//; that I asked you to substitute above, try the following:

s/^0// foreach ($year,$month,$day);

(This just strips any leading zeroes off the two digit numbers $year, $month and $day)

Gods be willing this might shut up a few thousand more of those little blighters! Good lesson here about making sure you're not relying on hidden conversions, but instead filter the data through correctly.

Cheers,
Paul ("fresh" from a 7 hour straight installation nightmare! I *think* we got there in the end, but it was a *long* way beyond the two hours that were allocated. Yeah, we knew it would probably take a little longer. Shock, horror, and all that. Sleep beckons...)

[Spiff]

that did it!

a brief perusal of the file listings shows that everything that should've changed has changed. No unusual errors were encountered.



I, um, have one last thing to ask and then I think this script can be called 'done' and put to bed...

what could we add to the script to get the time set of each file to 12:00? (Reason being that all HFS+ time stamps are in GMT. If GMT is 12:00, then no matter the time zone, the date displayed in the finder will be the same (albeit a different time). Here, depending on the Time Zone I set, the day can be one day 'off' (since I'm in Japan that's +9 GMT, whereas N.America can be anywhere from -6 to -9 GMT.)

thanks a bajillion....

yea, I hear you about installs and how they go over time allotted. Been there, done that a few too many times. Now I choose the wildest estimate possible and give it to the client after inflating that by another 50%. If I finish ahead of time, they're happy.

thanks so much Paul, your help is most appreciated!