PDA

View Full Version : unix multitasking; what are some strategies


Quanto
08-11-2004, 02:32 PM
Hello Hinters!

It's been nearly a year since our last post here. Your previous help has been invaluable to us.

Have a problem as a unix ~beginner.

What we need to do:

o run a unix script that has 200-300 commands (same command over and over) each filtering a different text file

o have each command run to completion prior to next command since they use same resources

Each line in our script looks something like this:

mac2win1zhtml filexyz

(mac2win1zhtml is an alias containing 10-20 unix commands; it has worked well for months when we run it at Terminal's command line)

We execute this script like this:

prompt% tcsh -s <command_script_file

When we do this all those commands appear to try to run simultaneously, and bad things happen. :)

We'll stop there to keep it relatively simple.

Thanks,

Quanto

breen
08-11-2004, 04:26 PM
o run a unix script that has 200-300 commands (same command over and over) each filtering a different text file


In which case you want to learn about flow control in your shell of choice.

I use bash and run as fast as I can if I see any csh variant. This is how it would work in bash:

#! /bin/bash

for i in *;do mac2win1zhtml $i;done


There should be nothing in the current directory except the files you want processed. This will execute your alias sequentially over each file that matches '*' -- that is, all files. Somebody will add the correct c-shell construct, I'm sure.

It will probably be helpful if you post what's in your alias definition -- I suspect something may be off there too.

Breen

hayne
08-11-2004, 05:53 PM
If you have the commands 'foo' and 'bar' in a shell script (of any type, Bourne shell or C Shell, or whatever) like this:

foo
bar

then you are guaranteed that the command 'foo' will run and then the command 'bar' will run. If the command 'foo' works in the foreground (the usual situation) then the command 'bar' will not start executing until 'foo' has completed.
But if 'foo' is an alias for command that ends with an ampersand (&), that makes that command execute in the background and this means that the command 'bar' will start up right away. It sounds like this is what might be causing your trouble.
As someone else has said, please show us the exact commands you have in the script.

Quanto
08-12-2004, 09:40 AM
Hayne et al.,

Hayne you hit it on d'nose.

Here's that alias, and as you can see it has an ampersand for background which is convenient at command level but not in a script:

mac2win1zhtml cd /HTML/; rm *.*; wait; cd Batch; cp *.* /HTML/; wait; cd MACHTML; cp !:1 before.html; wait; doqtmctx !:1; cd full; cp !:1 /HTML/; wait; cp !:1 /Qtx_MAC_HTML/; wait; cd /HTML/; ls | wc; wait; grep '!:1' Sed_Batch_MAC_to_Win_HTML.txt >Sed_Batch_1z_MAC_to_Win_HTML.txt; m2u <Sed_Batch_1z_MAC_to_Win_HTML.txt >Sed_Batch_1z_MAC_to_Win_HTML.bat; wait; m2u <Sed_Cmd_MAC_to_Win_HTML.txt >Sed_Cmd_MAC_to_Win_HTML.bat; wait; tcsh -s <Sed_Batch_1z_MAC_to_Win_HTML.bat &; wait; cd Batch; pr -t bell

That's all one line. Notice next to last command, Sed_Batch... &. We suspect that is our offender.

That cd Batch uses that nice little MAC OS X Hints set of aliases which saves current dir as an alias of that directory as a 'set.' Recall how show gives reminder list of all your directory aliases. We use it alot, but be careful not to do save ''. Must do save 'alias.' Problem there if you do not: wipes all Sets.

When we get our G5 dualies, we plan to go to panther | tiger then and switch to bash | other. It would be nice to see a picture of multitasking multiterminal fg-bg strategies and behaviors including exec and other options. Need to study more and do it ourselves. Use to do real time embedded OSes and app's and that kind of pix was necessary as req's spec. prior to development of product.

You guys are great and we tell our community that. Hopefully you are getting some hits from our community.

Yes, we know, we should be doing Apple scripts, C, or Perl...but the aliases are so easy at this stage.

Thank you again for your rapid response team efforts.

Best,

Quanto :D

hayne
08-12-2004, 10:34 AM
Notice next to last command, Sed_Batch... &. We suspect that is our offender.
That is definitely the problem.

That cd Batch uses that nice little MAC OS X Hints set of aliases which saves current dir as an alias of that directory as a 'set.' Recall how show gives reminder list of all your directory aliases. We use it alot, but be careful not to do save ''. Must do save 'alias.' Problem there if you do not: wipes all Sets.

In a script, you really ought to avoid using any such shortcuts. You should spell out the full paths to all executables, etc. Short cuts are just for interactive use.
Interesting bug that you have discovered in the 'save' alias. I was the one who submitted that hint to macosxhints. (The idea originated long ago from a colleague of mine.) I'll see what I can do to prevent that bug.

Yes, we know, we should be doing Apple scripts, C, or Perl...but the aliases are so easy at this stage.

No need to use anything different than shell scripts (bash). All you need to do to fix your situation is to make a shell script that has something like this:

#!/bin/sh

the_file=$1

cd /HTML/
rm *.*
wait
cd Batch
cp *.* /HTML/
wait
cd MACHTML
cp $the_file before.html
wait
doqtmctx $the_file
cd full
cp $the_file /HTML/
wait
cp $the_file /Qtx_MAC_HTML/
wait
cd /HTML/
ls | wc
wait
grep "$the_file" Sed_Batch_MAC_to_Win_HTML.txt >Sed_Batch_1z_MAC_to_Win_HTML.txt
m2u <Sed_Batch_1z_MAC_to_Win_HTML.txt >Sed_Batch_1z_MAC_to_Win_HTML.bat
wait
m2u <Sed_Cmd_MAC_to_Win_HTML.txt >Sed_Cmd_MAC_to_Win_HTML.bat wait
tcsh -s <Sed_Batch_1z_MAC_to_Win_HTML.bat
wait
cd Batch
pr -t bell


but where you replace all of the aliases in the above by their full expansions.
Then put that script file into some convenient folder, make it executable, and then use that script instead of your alias. You will either need to specify the full path to that script, or else put it in a folder which is on your execution PATH. (See any Unix tutorial about setting up the PATH variable)

breen
08-12-2004, 11:51 AM
Quanto --

What exactly are you trying to accomplish here? If you can describe it we may be able to simplify the whole thing for you.

Breen

Quanto
08-12-2004, 08:22 PM
Hayne,

you wrote:

"I'll see what I can do to prevent that bug."

We use that little 'tool' constantly! It is great! Would appreciate a fix, but only have to be careful, which is fairly normal in unix.

We'll try your suggestion. Tend to stick w/what works since our other non-unix burdens are so high.

When we looked at our STDIN file containing individual command calls, each had a trailing &. :eek:

Breen,

What we are doing is pretty complex, but it distills to doing site-wide db mgmt with unix.

A major problem is that we did orig. site devel. on Win$Tel M$ NT4. We used long file names. We still FTP site to ISP from that environ, but do all maintenance on our wonderful PB G4.

mac2win1zhtml takes one MAC HTML page and turns it into a longfilename Win$Tel HTML fixing all links and anchors. We have another that does between 600-1000 pages (whole site)!

On top of that we do what we call "Quantum English Language Remediation" of all quantum co(n)mtexts in those pages. It's pretty complex. But what we have now allows us to do in less than a day what used to take weeks on a smaller site under NT4 with Office Pro.

We just use unix to do what a db would otherwise do.

We do not like CSS! Need more flexibility with many more fonts, etc.

Stop there to keep it fairly simple.

Breen, Hayne,

When and which OS X will support long filenames without truncation and encoding?

That has been our biggest disappointment with our MAC stuff.

Best,

Quanto (Doug).

kfaulhaber
08-12-2004, 09:00 PM
When and which OS X will support long filenames without truncation and encoding?

The OS and file system support long file names, however, some apps don't.

Sneeper
08-13-2004, 12:42 PM
I think Carbon apps, because it must be backwards compatible with OS 9, does not support long file names.

Other apps, including Cocao apps, should be fine.

Quanto
08-13-2004, 02:43 PM
kfaulhaber and sneeper,

Thank you for your input.

Guess our old PageMill is mostly d'problem.

Quanto.