Spotlight, mdfind, date formats
 

I want to locate files using mdfind based on the meta-data tag kMDItemLastUsedDate (or any meta-data using a date for that matter). However, Apple's prescribed method does not work for me in Terminal:

Whereas, of course, something like this works fine:

What's my trouble with the date format? Eventually, I want to use this query in an AppleScript do shell script routine.

TIA!

Hi Doug,

this works in terminal and as a do shell script,

I have been playing with these also too use them in AS, they are very handy.

Also here is a script to quickly check metadata, I have it as a hot key using spark.
** Edit**

Mark,

Hey, I owe you a six pack! It never occurred to me to re-arrange the parameters.

Have you seen this article I wrote on basic Spotlight scripting for iTunes?

Thanks again. This opens up WORLDS for me :)

That a good read..

Thanks

Drat....

It works with ~/ but any other directory returns up "". So, OK, what is wrong with this:

I'm looking for the files that iTunes played today in my "Music" directory.

I will have a look at that..

in the mean time I had a play and came up with this.
another way of dealing with the " ' " single quotes.
Do not know if I really need to put the try bits in but what the heck.
Also put in a -onlyin ~/Desktop/Music/ which works

It looks like the meta data does not get updated on an mp3 after it has been played. (I just finished listening to Jesus Is Just Alright)

$ mdls /Volumes/G5Repos/iTunes\ Music/The\ Doobie\ Brothers/Best\ Of\ The\ Doobies/07\ Jesus\ Is\ Just\ Alright.mp3
/Volumes/G5Repos/iTunes Music/The Doobie Brothers/Best Of The Doobies/07 Jesus Is Just Alright.mp3 -------------
kMDItemAlbum = "Best Of The Doobies"
kMDItemAttributeChangeDate = 2005-04-29 19:03:28 +1000
kMDItemAudioBitRate = 128
kMDItemAudioChannelCount = 2
kMDItemAudioEncodingApplication = "iTunes v1.0"
kMDItemAudioSampleRate = 44100
kMDItemAuthors = ("The Doobie Brothers")
kMDItemContentCreationDate = 2001-01-14 12:27:47 +1000
kMDItemContentModificationDate = 2004-09-11 12:28:07 +1000
kMDItemContentType = "public.mp3"
kMDItemContentTypeTree = (
"public.mp3",
"public.audio",
"public.audiovisual-content",
"public.data",
"public.item",
"public.content"
)
kMDItemDisplayName = "07 Jesus Is Just Alright.mp3"
kMDItemDurationSeconds = 827
kMDItemFSContentChangeDate = 2004-09-11 12:28:07 +1000
kMDItemFSCreationDate = 2001-01-14 12:27:47 +1000
kMDItemFSCreatorCode = 1752133483
kMDItemFSFinderFlags = 264
kMDItemFSInvisible = 0
kMDItemFSLabel = 4
kMDItemFSName = "07 Jesus Is Just Alright.mp3"
kMDItemFSNodeCount = 0
kMDItemFSOwnerGroupID = 80
kMDItemFSOwnerUserID = 503
kMDItemFSSize = 4414027
kMDItemFSTypeCode = 1297106739
kMDItemID = 837
kMDItemKind = "MP3 Audio File"
kMDItemLastUsedDate = 2004-09-11 12:28:07 +1000
kMDItemMediaTypes = (Sound)
kMDItemMusicalGenre = "Rock"
kMDItemTitle = "Jesus Is Just Alright"
kMDItemTotalBitRate = 128
kMDItemUsedDates = (2004-09-11 12:28:07 +1000)

If a track plays all the way through then kMDItemLastUsedDate will be updated, thus:

kMDItemLastUsedDate = 2005-06-10 06:45:37 -0400

Except it does not seem to be updating here.....

Maybe DougAdams has the same problem as me....

This is an old thread, but I'm finding the same problem now in OS 10.4.5.

I cannot get any of the $time variables to work with mdfind. Does anyone know if this is a known bug? Or a better place to ask?

Here's a simple example. Create a New Event in iCal. Wait a min ute or two until a new item turns up in ~/Library/Caches/Metadata/iCal/ : sort that folder in List view in the Finder by Date Modified. When you see a subfolder with the modification date/time of the time you made the folder jump to the top, go inside it and, also sorted by List and Date Modified, the .icalevent at the top with the same Modified and Created date/time is the event. (Double-click it to be sure - it will open iCal to the correct event.)

In Terminal, type

mdfind -onlyin

and a pace, then drag the ~/Library/Caches/Metadata/iCal/ folder there. In my own case that results in:

250-166:~ berkowit$ mdfind -onlyin /Users/berkowit/Library/Caches/Metadata/iCal/

Now if you look for events with the name of the new one (e.g. "New Event") this way:

250-166:~ berkowit$ mdfind -onlyin /Users/berkowit/Library/Caches/Metadata/iCal/ "kMDItemTitle == '*New Event*'"

and press Return, that metadata file does show up in the list of results. (It will have the long GUUID name you see in the Finder.)

But if you use any of the $time methods, looking for items created or modified today, or since yesterday, or in the last hour (now(-3600)), you get no results at all:

mdfind -onlyin '/Users/berkowit/Library/Caches/Metadata/iCal/' "kMDItemContentModificationDate == $time.today"

mdfind -onlyin '/Users/berkowit/Library/Caches/Metadata/iCal/' "kMDItemContentModificationDate > $time.now(-3600)"

mdfind -onlyin '/Users/berkowit/Library/Caches/Metadata/iCal/' "kMDItemContentModificationDate > $time.yesterday"

mdfind -onlyin '/Users/berkowit/Library/Caches/Metadata/iCal/' "kMDItemContentCreationDate > $time.yesterday"

all produce zero results.

Am I doing something wrong, or is this a bug with mdfind?

Someone I know who does Carbon programming did it with MDQuery and says it works fine that way. (You have to create an application first...)

The following command works fine for me (on 10.4.5):

mdfind 'kMDItemFSCreationDate >= $time.yesterday'

Use 'mdls' to check one of the files you are expecting to find to see if the metadata is there.

I get no results for

mdfind 'kMDItemFSCreationDate >= $time.yesterday'

either, even omitting the -onlyin option. I.e., supposedly I have no files anywhere at all created on my computer since yesterday.

When I do an mdls on the file in question, all these date attributes show up there:

kMDItemContentCreationDate = 2006-02-17 10:32:06 -0800
kMDItemContentModificationDate = 2006-02-17 10:32:06 -0800
kMDItemFSContentChangeDate = 2006-02-17 10:32:06 -0800
kMDItemFSCreationDate = 2006-02-17 10:32:06 -0800

So why isn't mdfind finding it? I also tried:

mdfind "kMDItemFSCreationDate >= $time.iso(2006-02-17 10:32:06 -0800)"

and got no results.

Oops:

mdfind "kMDItemFSCreationDate >= $time.iso('2006-02-17 10:32:06 -0800')"

got no results either.

man mdfind has no mention of the $time variable anywhere - I just got that out of the Spotlight Carbon docs. Is there maybe something you need to enable somewhere to get it working?

Not as far as I know. I certainly didn't do anything special.
But do you have the developer tools installed? That might be one difference.

Yes, I have Xcode Tools 2.1 installed (about to install 2.2.1). But that shouldn't make any difference. This should work on any OS 10.4 computer: in fact it's essential of shell scripts I write are going to work on other users' computers.

It must be something else. I'll try a reboot soon.

Here's a Bash script that tests 'mdfind':
Be careful that you haven't excluded the folders you are testing from Spotlight. When I first tried the above script it didn't seem to work since I was running it in a sub-folder of my ~/Dev folder which I had excluded from Spotlight.

I note that 'mdls' seems to list the metadata even if the files are not being indexed by Spotlight. E.g. the files under my ~/Dev folder show results from 'mdls' even though they are not subject to Spotlight.

Well, it works. Thank you, hayne. I was afraid it was going to keep going forever - when run from the Terminal it did not stop with the original 'now' because you keep resetting it, so thanks for that i < 100.

if ((i % 10 == 0)) may not quite work quite as you intended, but close enough. It made the folder, made one file, found it, made 10 more (up to 11) found them, etc. up to 91, made 9 more (up to 100) but never got to find them. It also never printed "Current date is: ..."

But it certainly found 91 files using
"mdfind -onlyin /Users/berkowit/1140216408 'kMDItemFSCreationDate >= $time.now(-3600)'"

(Each time it printed all of thm to date, so there were 91 at the end.)

Therefore $time.now(-3600) works in that directory. I can't understand why it doesn't work in the ~/Library/Metadata/iCal/ directory when it has not been excluded and mdfind works perfectly well on text attributes there.

Does this make any sense?

No - it works as intended.
Initially the index 'i' is zero, so the test 'if ((i % 10 == 0))' succeeds. The next time it succeeds is when i is 10 (the 11th iteration).
The "Current date is: ..." is being printed into the files that are created. Look at one of the files with 'cat' or your favourite text editor.

Try running the script from that directory.
I.e. do:
cd ~/Library/Metadata/iCal
and then run the script (using the full path to the script if necessary)

Well, I would say it is a bit flakey (foo is an empty file) :
So it did not see it untile I moved it to a '.txt' file but now it still sees it if I move it back to a file with no extension.

It is well known that Spotlight only indexes those files that it recognizes as being textual in nature. For other types of files, it relies on applications providing an "importer".
If a file doesn't have an extension or a creator code or type code, then Spotlight will generally ignore it. It doesn't look inside the file to see if it can guess what type it is from the contents (like the 'file' command does).

I think that once it is indexed, however, the index follows it around by using its internal file-id instead of its name.

PS and my mp3s are still not getting updated. Listened to this in its entirety a few minutes ago with iTunes 6.03 on 10.4.5, not even a 2006 date!:

I ran into this problem with Spotlight in 10.4.4 from time to time.

The only way I seemed to ensure that it always worked was to calculate the number of seconds since midnight 01/01/2001 to the point in time I was interested in separately, and feed that number of seconds into the search.

I wrote an Applescript in this thread using this idea.

OK, I can buy that, Actually I recall now that I have an apple bug report in on the fact that it does not index everything.

Still no joy on my mp3s and iTunes!

I'm not sure if Spotlight indexes external disks by default. I think you need to do something to tell it to do that. There was at least one macosxhints article about this.

Well, it was an odd intenrion to make files 91-100 but not try to find them then. ;-) it looked to me rather as if you hadn't noticed that you had not yet incremented i yet. Never mind, that's not important.

I have to run now, but I'm going to check later. I'm wondering if maybe using $time somehow constrains mdfind not to look in subdirectories - although it does so for text attributes. That seems to peculiar that it's hard to believe, but it's the only thikng that looks even halfway plausible at the moment.

No - that isn't it either. Try this slightly modified script:

It looks like I was right. The scrip abobe creates the files, but every time it tries to use mdfind to find them, I get this:

Files found by "mdfind -onlyin /Users/berkowit/1140233658 'kMDItemFSCreationDate >= $time.now(-3600)'":
-bash: Data/eval: No such file or directory

So there we are: OMM, mdfind -onlyin /Users/berkowit/1140233658 can't find any of the 100 files in its subfolder Data when using $time.now().

But on yours it can?

No - you seem to be getting an error from the Bash script when you run it.
So the script is failing for some reason. I don't think you have really tested mdfind at all here.
I've been trying to figure out what might make the script give that error message but I haven't got any ideas yet. It is very strange.
Why would it say "Data/eval: No such file or directory" ?
I really can't figure out what circumstances would lead to that error message.

Could you please try a few things?
First off make sure that your script has the correct (Unix style) line endings. There is a section in this Unix FAQ about the line endings issue. That section has a one-line Perl script that will fix the line endings if needed. I mention that since incorrect line endings sometimes result in strange error messages.

Secondly, try changing the name of the subdir - e.g. make it be:
subdir=Paul
instead of
subdir=Data
This is just to verify that the name in the error message is indeed the name of the subdir.

No, I thought of that one and I do have indexing on that volume :) :

Paul Berkowitz:
Any more word on that script - did you figure out what was going wrong with it? (see my suggestions above)

stetner:
Is the problem perhaps confined to iTunes? What if you manually modify one of the MP3 files on that volume - does the change show up in the results from 'mdls' and/or 'mdfind' ?

If I play the tune with Quicktime it still does not update. If I 'touch' the file it updates

For the songs that I have tried (that are on my internal disk), it seems that playing them in iTunes (by selecting them in the list and pressing the Play button) does not update the LastUsed entry.
But if I use the command-line 'open' to play the file then it does update "Last Used". (And 'open' does play the song in iTunes since that is the app registered for MP3's)

I selected "Free Man in Paris" from the list in iTunes and started it playing. I waited until the song ended.
Then I went into the Terminal and ran the following commands:

Now we are onto something! 'open seems to do a whole lot....Look at all the new stuff that open has put in.....

Interesting!

Hmmm, the composer field isn't part of a song's metadata!
Comments are included, but not the composer.
That's kinda lame.