use du to find files

thatch · 04-25-2002, 03:59 PM

A recent tip from the Unix Guru Universe, #1936, April 20, 02, says:

USE DU TO FIND FILES

Normally:

find . -name *.txt -print

is the command to find a file.
But you can locate find more
efficiently using du and grep

Use:

du -a |grep *.txt

This will locate all the files
with the extension .txt in the
current directory.

... but I am not getting it to work. I use the latest fink installed versions of find, du and grep. Maybe that's got something to do with it not working for me or perhaps I have stepped on something causing breakage somewhere, I don't know.

Anyway, I just get the command prompt without any message when I try it with the pattern for .plist files. I threw in the find command for the same to show what should be returned...

Code:

[localhost:~] thatch% cd /Library/preferences
[localhost:/Library/preferences] thatch% du -a | grep *.plist
[localhost:/Library/preferences] thatch% find *.plist
com.apple.ColorSyncDeviceList.plist
com.apple.HIToolbox.plist
com.apple.PowerManagement.plist
com.apple.dockfixup.plist
com.apple.loginwindow.plist
com.apple.windowserver.plist
loginwindow.plist
[localhost:/Library/preferences] thatch%

... if fact, if I use the normal example, that doesn't work either...

Code:

[localhost:/Library/Preferences] thatch% find . -name *.plist -print
find: paths must precede expression
Usage: find [path...] [expression]
[localhost:/Library/Preferences] thatch%

... but at least there I get some message which I interpret as /sw/bin/find doesn't consider relative paths like the . character as valid for CWD. Somebody please correct me if I'm wrong about that.

Side Note: The alias 'ff' doesn't work here either because it is very similar to the normal example and the alias 'files' does work just the same as the plain 'find' does.

Am I missing something really simple here or does this work for anyone else out there?

xchanyazy · 04-25-2002, 08:35 PM

I get the same results... (du v4.1)

If I write

Code:

[20:12][brandg /Users/brandg] %cd /Library/Preferences/
/Library/Preferences
DirectoryService                     com.apple.dockfixup.plist
com.apple.AEServer.plist             com.apple.loginwindow.plist
com.apple.ColorSyncDeviceList.plist  com.apple.windowserver.plist
com.apple.HIToolbox.plist            loginwindow.plist
com.apple.PowerManagement.plist
[20:30][brandg /Library/Preferences] %du -a | grep *.plist
[20:30][brandg /Library/Preferences] %find *.plist
com.apple.AEServer.plist
com.apple.ColorSyncDeviceList.plist
com.apple.HIToolbox.plist
com.apple.PowerManagement.plist
com.apple.dockfixup.plist
com.apple.loginwindow.plist
com.apple.windowserver.plist
loginwindow.plist

I get nothing from du -a. However, if I do

Code:

[20:30][brandg /Library/Preferences] %du -a
4       ./.Eng_Store
4       ./.GlobalPreferences.plist
4       ./com.apple.AEServer.plist
4       ./com.apple.ColorSyncDeviceList.plist
4       ./com.apple.dockfixup.plist
4       ./com.apple.HIToolbox.plist
4       ./com.apple.loginwindow.plist
4       ./com.apple.PowerManagement.plist
8       ./com.apple.windowserver.plist
0       ./DirectoryService/.DSRunningSP1
4       ./DirectoryService/ContactsNodeConfig.plist
8       ./DirectoryService/DSLDAPPlugInConfig.clpi
4       ./DirectoryService/SearchNodeConfig.plist
16      ./DirectoryService
4       ./loginwindow.plist
60      .
[20:30][brandg /Library/Preferences] %du -a > out.txt
[20:31][brandg /Library/Preferences] %grep .plist out.txt 
4       ./.GlobalPreferences.plist
4       ./com.apple.AEServer.plist
4       ./com.apple.ColorSyncDeviceList.plist
4       ./com.apple.dockfixup.plist
4       ./com.apple.HIToolbox.plist
4       ./com.apple.loginwindow.plist
4       ./com.apple.PowerManagement.plist
8       ./com.apple.windowserver.plist
4       ./DirectoryService/ContactsNodeConfig.plist
4       ./DirectoryService/SearchNodeConfig.plist
4       ./loginwindow.plist

grep can get the text through the file just fine.

While looking at the man pages, a solution came to me. Use du -ah (the h option makes it human readable output) | grep *.plist

This works as expected.

thatch · 04-25-2002, 08:53 PM

Well then, I'm glad someone was willing to try this. Thanks xchanyazy.

So, now I know that I haven't necessarily broken anything. And I see that putting the process into two parts by redirecting the output into a text file and then grep from there worked. But adding the -h option to the original command didn't change anything for me:

Code:

[localhost:~] thatch% cd /Library/Preferences/
[localhost:/Library/Preferences] thatch% du -ah | grep *.plist
[localhost:/Library/Preferences] thatch%

xchanyazy, are you certain that this worked for you? Could you please double check for me? TIA

xchanyazy · 04-25-2002, 08:57 PM

At first it didn't work, and then I realized why.

It's the grep *.plist.

You need to use grep .plist (no *).

When I try with *.plist, I get nothing. However, when I try with .plist, I get the expected results. This was not the case with the output redirected to the text file.

Hope this helps.

thatch · 04-25-2002, 09:12 PM

You're right. No need for that wild card asterisk. I'll drop a note to the author about it.

Thank you xchanyazy

thatch · 04-25-2002, 10:31 PM

I just found out by playing around a bit that the 'normal' command line of:

find . -name *.plist -print

...doesn't work because it needs an escape character before the wildcard, (asterisk):

find . -name \*.plist -print

Again, I will let the author of the unix guru universe tip know about that as well.

blb · 04-25-2002, 11:37 PM

Note that grep uses regular expressions, so if you literally want to match the . (dot), use

Code:

du -a |grep "\.plist"

otherwise, a dot in regex is any character (which would cause a match with strings like aplist, 0plist, etc).
If you want to get really fancy, and make sure the match ends with .plist, use

Code:

du -a |grep "\.plist$"

which tells grep to match the string .plist followed by the end of line.
See manpages for grep and re_format for gory detail.

thatch · 04-26-2002, 12:16 AM

Thanks blb. Your first example works and makes perfect sense. And the second example makes sense too, but it gives me an 'Illegal variable name'.

Code:

[localhost:/Library/preferences] thatch% du -ah | grep "\.plist$"
Illegal variable name.

...and terminal.app seems to want to put an escape character after the end of line character but still returns the illegal variable name business.

Code:

[localhost:/Library/preferences] thatch% du -ah | grep "\.plist$\"
Illegal variable name.

...so, I'm stumped about that one. I'll bet there's a simple answer though.

Anyone? Please feel free to enlighten me, us or whoever.

Regards

mervTormel · 04-26-2002, 12:45 AM

works fine here... [edit - i lied, the following is not entirely accurate, see below]

$ du -ah | grep "\.plist$"
4.0k ./.MacOSX/environment.plist
4.0k ./Default Folder X.prefPane/Contents/Info.plist
...

% which du grep
/sw/bin/du
/sw/bin/grep

perhaps you have your grep or du aliased to something?

what does

% /usr/bin/du -ah | /usr/bin/grep "\.plist$"

get you?

i think there's a way to get the canonical command if you have it masked by an alias, but i can't remember what it is. but, it's best not to mask existing commands with aliases.

mervTormel · 04-26-2002, 12:50 AM

whoops, i had a du in my ~/bin which worked, but the /sw/bin/du produces your 'illegal variable name' error

i'm flummoxed, but please disregard my previous post as being anything approaching accurate.

mervTormel · 04-26-2002, 01:17 AM

it appears to be a tcsh anomaly ?...

bash$ tcsh

% mydu -ah | grep "\.foo$"
Illegal variable name.

% exit

bash$ mydu -ah | grep "\.foo$"
4.0k ./x.foo

thatch · 04-26-2002, 01:27 AM

It is a bit confusing for a relative newby like myself to have to remember to check which du or whatever constantly because of subtle and sometimes not so subtle differences in them, meaning the stock version or the fink installed version. It's sort of assumed that the fink one will have improvements over the older stock version.

Ah, and now I see you've posted again about the tcsh anomaly. I'm not surprised as I continually read things about tcsh here and its various oddities.

Thanks for spending some time on this with me. All in good fun!

ashevin · 04-26-2002, 09:15 AM

Hi,

The reason for this is that tcsh will try variable substitution in double-quoted strings. Since "$" is the characted used to indicate a variable, you get the error because you aren't specifying one. To work around this, using single-quotes in all the examples where double-quotes are used.

I suspect bash works because it is intelligient enough to ignore the "$" at the end of the string.

- Avi

P.S. As one poster indicated, the various troubles are caused by the shell trying to do filename expansion. Unless you know what you are doing, the parameter to "-name" in a find command should always be single-quoted. The same is true for the pattern passed to grep. What happens is that the shell replaces the "*.plist" with a blank string because no files match. This causes syntax errors in the programs that require that parameter.

mervTormel · 04-26-2002, 05:17 PM

avi, excellent. thanks for the news. it's time to get that drill down pat. sometimes it just takes a choker like this to reinforce that notion.

double quoted strings are candidates for expansion rules, single quoted strings are not.

so, if you want your strings to be considered strings (qua strings), single quote them.

thatch · 04-26-2002, 06:20 PM

Great lesson! Well explained and now well understood after some tinkering with find and grep keeping the filename expansion tip and single quotes in mind.

edited 'file expansion' to 'filename expansion'

04-25-2002, 08:53 PM	#3
thatch All Star Join Date: Jan 2002 Posts: 534	Hmm.. Well then, I'm glad someone was willing to try this. Thanks xchanyazy. So, now I know that I haven't necessarily broken anything. And I see that putting the process into two parts by redirecting the output into a text file and then grep from there worked. But adding the -h option to the original command didn't change anything for me: Code: [localhost:~] thatch% cd /Library/Preferences/ [localhost:/Library/Preferences] thatch% du -ah \| grep *.plist [localhost:/Library/Preferences] thatch% xchanyazy, are you certain that this worked for you? Could you please double check for me? TIA

04-25-2002, 08:57 PM	#4
xchanyazy All Star Join Date: Jan 2002 Location: Dexter, MI, USA Posts: 704	At first it didn't work, and then I realized why. It's the grep .plist. You need to use grep .plist (no ). When I try with *.plist, I get nothing. However, when I try with .plist, I get the expected results. This was not the case with the output redirected to the text file. Hope this helps. __________________ - Greg Happy user of OS X since the Public Beta. Help Team Mac OS X cure cancer, Alzheimer's, ALS, Parkinson's, and more!

04-25-2002, 09:12 PM	#5
thatch All Star Join Date: Jan 2002 Posts: 534	Yup! You're right. No need for that wild card asterisk. I'll drop a note to the author about it. Thank you xchanyazy

04-25-2002, 10:31 PM	#6
thatch All Star Join Date: Jan 2002 Posts: 534	also... just to note I just found out by playing around a bit that the 'normal' command line of: find . -name .plist -print ...doesn't work because it needs an escape character before the wildcard, (asterisk): find . -name \.plist -print Again, I will let the author of the unix guru universe tip know about that as well.

04-25-2002, 11:37 PM	#7
blb All Star Join Date: Jan 2002 Location: CO, USA Posts: 908	Note that grep uses regular expressions, so if you literally want to match the . (dot), use Code: du -a \|grep "\.plist" otherwise, a dot in regex is any character (which would cause a match with strings like aplist, 0plist, etc). If you want to get really fancy, and make sure the match ends with .plist, use Code: du -a \|grep "\.plist$" which tells grep to match the string .plist followed by the end of line. See manpages for grep and re_format for gory detail.

04-25-2002, 03:59 PM	#1
thatch All Star Join Date: Jan 2002 Posts: 534	use du to find files A recent tip from the Unix Guru Universe, #1936, April 20, 02, says: USE DU TO FIND FILES Normally: find . -name .txt -print is the command to find a file. But you can locate find more efficiently using du and grep Use: du -a \|grep .txt This will locate all the files with the extension .txt in the current directory. ... but I am not getting it to work. I use the latest fink installed versions of find, du and grep. Maybe that's got something to do with it not working for me or perhaps I have stepped on something causing breakage somewhere, I don't know. Anyway, I just get the command prompt without any message when I try it with the pattern for .plist files. I threw in the find command for the same to show what should be returned... Code: [localhost:~] thatch% cd /Library/preferences [localhost:/Library/preferences] thatch% du -a \| grep .plist [localhost:/Library/preferences] thatch% find .plist com.apple.ColorSyncDeviceList.plist com.apple.HIToolbox.plist com.apple.PowerManagement.plist com.apple.dockfixup.plist com.apple.loginwindow.plist com.apple.windowserver.plist loginwindow.plist [localhost:/Library/preferences] thatch% ... if fact, if I use the normal example, that doesn't work either... Code: [localhost:/Library/Preferences] thatch% find . -name *.plist -print find: paths must precede expression Usage: find [path...] [expression] [localhost:/Library/Preferences] thatch% ... but at least there I get some message which I interpret as /sw/bin/find doesn't consider relative paths like the . character as valid for CWD. Somebody please correct me if I'm wrong about that. Side Note: The alias 'ff' doesn't work here either because it is very similar to the normal example and the alias 'files' does work just the same as the plain 'find' does. Am I missing something really simple here or does this work for anyone else out there?

04-26-2002, 12:16 AM	#8
thatch All Star Join Date: Jan 2002 Posts: 534	Yes and No... Thanks blb. Your first example works and makes perfect sense. And the second example makes sense too, but it gives me an 'Illegal variable name'. Code: [localhost:/Library/preferences] thatch% du -ah \| grep "\.plist$" Illegal variable name. ...and terminal.app seems to want to put an escape character after the end of line character but still returns the illegal variable name business. Code: [localhost:/Library/preferences] thatch% du -ah \| grep "\.plist$\" Illegal variable name. ...so, I'm stumped about that one. I'll bet there's a simple answer though. Anyone? Please feel free to enlighten me, us or whoever. Regards

04-26-2002, 12:45 AM	#9
mervTormel League Commissioner Join Date: Jan 2002 Posts: 5,536	works fine here... [edit - i lied, the following is not entirely accurate, see below] $ du -ah \| grep "\.plist$" 4.0k ./.MacOSX/environment.plist 4.0k ./Default Folder X.prefPane/Contents/Info.plist ... % which du grep /sw/bin/du /sw/bin/grep perhaps you have your grep or du aliased to something? what does % /usr/bin/du -ah \| /usr/bin/grep "\.plist$" get you? i think there's a way to get the canonical command if you have it masked by an alias, but i can't remember what it is. but, it's best not to mask existing commands with aliases. Last edited by mervTormel; 04-26-2002 at 12:51 AM.

04-26-2002, 12:50 AM	#10
mervTormel League Commissioner Join Date: Jan 2002 Posts: 5,536	whoops, i had a du in my ~/bin which worked, but the /sw/bin/du produces your 'illegal variable name' error i'm flummoxed, but please disregard my previous post as being anything approaching accurate.

04-26-2002, 01:17 AM	#11
mervTormel League Commissioner Join Date: Jan 2002 Posts: 5,536	it appears to be a tcsh anomaly ?... bash$ tcsh % mydu -ah \| grep "\.foo$" Illegal variable name. % exit bash$ mydu -ah \| grep "\.foo$" 4.0k ./x.foo

04-26-2002, 01:27 AM	#12
thatch All Star Join Date: Jan 2002 Posts: 534	Sheesh! It is a bit confusing for a relative newby like myself to have to remember to check which du or whatever constantly because of subtle and sometimes not so subtle differences in them, meaning the stock version or the fink installed version. It's sort of assumed that the fink one will have improvements over the older stock version. Ah, and now I see you've posted again about the tcsh anomaly. I'm not surprised as I continually read things about tcsh here and its various oddities. Thanks for spending some time on this with me. All in good fun!

04-26-2002, 09:15 AM	#13
ashevin Triple-A Player Join Date: Jan 2002 Posts: 158	Re: tcsh anomaly Hi, The reason for this is that tcsh will try variable substitution in double-quoted strings. Since "$" is the characted used to indicate a variable, you get the error because you aren't specifying one. To work around this, using single-quotes in all the examples where double-quotes are used. I suspect bash works because it is intelligient enough to ignore the "$" at the end of the string. - Avi P.S. As one poster indicated, the various troubles are caused by the shell trying to do filename expansion. Unless you know what you are doing, the parameter to "-name" in a find command should always be single-quoted. The same is true for the pattern passed to grep. What happens is that the shell replaces the "*.plist" with a blank string because no files match. This causes syntax errors in the programs that require that parameter.

04-26-2002, 05:17 PM	#14
mervTormel League Commissioner Join Date: Jan 2002 Posts: 5,536	avi, excellent. thanks for the news. it's time to get that drill down pat. sometimes it just takes a choker like this to reinforce that notion. double quoted strings are candidates for expansion rules, single quoted strings are not. so, if you want your strings to be considered strings (qua strings), single quote them.

04-26-2002, 06:20 PM	#15
thatch All Star Join Date: Jan 2002 Posts: 534	Thanks Avi Great lesson! Well explained and now well understood after some tinkering with find and grep keeping the filename expansion tip and single quotes in mind. edited 'file expansion' to 'filename expansion' Last edited by thatch; 04-26-2002 at 06:26 PM.