File searching with grep

[Ty]

I have a complex directory structure full of data-containing folders, and in each folder there is a single acqu.par file that gives parameters relevant to the data in its folder. I want to make a list of all the data directories that match all of 5 different parameters. In other words, I want a list of all the data folders that I have the have 5 specific conditions, all listed in the acqu,par file.

I've tried to work with grep to do this, including recursive searching and searching only *.par files (otherwise it takes forever since the directories are large). I've gotten the following to work:

grep -HRn --include=*.par "condition1" .
(I want the filelocations and line numbers, hence the -H and -n)

As far as I can tell, the way to get grep to implement an "AND" operator (list only files that match 1 AND 2 AND 3 AND...) is to pipe it along:

grep -HRn --include=*.par "condition1" . | grep -HRn --include=*.par "condition2" .

However, this seems so give an OR, rather than AND, interpretation, listing all files that match either cond1 or cond2. I've tried removing the options and location parameters as well, giving something like:

grep -HRn --include=*.par "condition1" . | grep "condition2" | ...

But this doesn't list any files while I am certain that files exist that match all the conditions. (I've checked and checked.)

Any suggestions on how to do a recursive search with grep using an effective AND operator and limiting the search by file type?

Cheers.

[fracai]

Pipes don't work that way. When you pipe something, you're taking the output of the first command and piping it through the second. If all your conditions are on the same line you'd be fine. If they're on separate lines, you'll need to either have grep print additional context (see -A, -B, or -C) or inspect each file and store whether it satisfied each individual condition. Then AND those yourself.

[hayne]

You'll want to write a shell script (instead of trying to do it all on one command line).
In your shell script, you can run a grep command and record the result of that command in a variable. And then test (and combine) your variables later in the script.

[dmacks]

Something like this might work:

find . -name acqu.par | xargs grep -l "thing to match" | xargs grep -l "thing to match" | ... | xargs basename

[Ty]

After taking the advice to put this into a shell script, I've come up with something that works for me. Admittedly, it's not elegant, but it works. Thanks for the help, all.


#############################
#!/bin/bash

dataFolder='/topLevelDataFolder/'
fileDir='/folderForOutputList/'
fileName1='list1.out'
fileName2='list2.out'
...

cond1='aaa = 100'
cond2='bbb = 2--'
...

grep -HRn --include=*.par "$cond1" "$dataFolder" | cut -f 1 -d : > "$fileDir$fileName1"
#the cut command is to remove the part of the grep output that contains the matched text.

cat "$fileDir$fileName1" | while read LINE
do
grep -Hn "$cond2" "$LINE" | cut -f 1 -d : >> "$fileDir$fileName2"
done
#############################

(The cat/line reading can be done as many times as necessary to catch all the strings for matching.)