The macosxhints Forums

The macosxhints Forums (http://hintsforums.macworld.com/index.php)
-   UNIX - Newcomers (http://hintsforums.macworld.com/forumdisplay.php?f=15)
-   -   Dumping file names with ls (http://hintsforums.macworld.com/showthread.php?t=98153)

cswenson 01-26-2009 11:53 AM

Dumping file names with ls
 
I'm not a unix newcomer (been using linux for 5 years), but I am a mac newcomer.

I've used the command ls *.(file_extension) > text.file (or some variant thereof) forever with no problem on linux. Yet, when I attempt the same thing on my mac I get a text.file with formated text (more/less says 'file may be a binary file), which really wrecks havoc when trying to parse lines from text.file.

Why is the text formatted? What do I need to do to get plain text dumped to text.file? I've tried searching for answers, but nothing obvious is coming up.

Thanks.

cwtnospam 01-26-2009 12:07 PM

What are you opening the file with, and have you tried this?

ls *.(file extension) > text.txt

cswenson 01-26-2009 12:13 PM

Doesn't matter
 
It doesn't matter what I open it with (it shouldn't matter either). What I want is unformatted text, but I get this type of stuff

Code:

~> ls *.img > files.dat
~> more files.dat
"files.dat" may be a binary file.  See it anyway?
ESC[0mESC[0msw00340895992ubb_sk.imgESC[0m
ESC[0msw00340895992um2_sk.imgESC[0m
ESC[0msw00340895992uuu_sk.imgESC[0m
ESC[0msw00340895992uvv_sk.imgESC[0m
ESC[0msw00340895992uw1_sk.imgESC[0m
ESC[0msw00340895992uw2_sk.imgESC[0m
ESC[0msw00340895992uwh_sk.imgESC[0m
ESC[m

What I want to see (and what I'm used to seeing on linux) is the following:

Code:

~> ls *.img > files.dat
~> more files.dat
sw00340895992ubb_sk.img
sw00340895992um2_sk.img
sw00340895992uuu_sk.img
sw00340895992uvv_sk.img
sw00340895992uw1_sk.img
sw00340895992uw2_sk.img
sw00340895992uwh_sk.img

(I copied the second output from the linux box sitting next to me after running the same command). Any thoughts?

hayne 01-26-2009 12:34 PM

Please copy & paste the following commands into a newly opened Terminal window (I'm assuming that you are using Terminal.app - is that correct?) and then press Return and then copy & paste the whole contents of the Terminal window back here so we can see:


/bin/ls *.img | /usr/bin/vis
/bin/ls *.img > files.txt
/usr/bin/hexdump -C files.txt

Hal Itosis 01-26-2009 12:45 PM

Perhaps ls is defined as a function or an alias:

type -a ls

?

cswenson 01-26-2009 01:05 PM

Here's the output:

Code:

~/UBS/test/uvot/image > /bin/ls *.img | /usr/bin/vis
sw00340895992ubb_sk.img
sw00340895992um2_sk.img
sw00340895992uuu_sk.img
sw00340895992uvv_sk.img
sw00340895992uw1_sk.img
sw00340895992uw2_sk.img
sw00340895992uwh_sk.img
/bin/ls *.img > files.txt
/usr/bin/hexdump -C files.txt~/UBS/test/uvot/image > /bin/ls *.img > files.txt
~/UBS/test/uvot/image > /usr/bin/hexdump -C files.txt
00000000  73 77 30 30 33 34 30 38  39 35 39 39 32 75 62 62  |sw00340895992ubb|
00000010  5f 73 6b 2e 69 6d 67 0a  73 77 30 30 33 34 30 38  |_sk.img.sw003408|
00000020  39 35 39 39 32 75 6d 32  5f 73 6b 2e 69 6d 67 0a  |95992um2_sk.img.|
00000030  73 77 30 30 33 34 30 38  39 35 39 39 32 75 75 75  |sw00340895992uuu|
00000040  5f 73 6b 2e 69 6d 67 0a  73 77 30 30 33 34 30 38  |_sk.img.sw003408|
00000050  39 35 39 39 32 75 76 76  5f 73 6b 2e 69 6d 67 0a  |95992uvv_sk.img.|
00000060  73 77 30 30 33 34 30 38  39 35 39 39 32 75 77 31  |sw00340895992uw1|
00000070  5f 73 6b 2e 69 6d 67 0a  73 77 30 30 33 34 30 38  |_sk.img.sw003408|
00000080  39 35 39 39 32 75 77 32  5f 73 6b 2e 69 6d 67 0a  |95992uw2_sk.img.|
00000090  73 77 30 30 33 34 30 38  39 35 39 39 32 75 77 68  |sw00340895992uwh|
000000a0  5f 73 6b 2e 69 6d 67 0a                          |_sk.img.|
000000a8
~/UBS/test/uvot/image >

Does this provide you a clue?

tlarkin 01-26-2009 01:14 PM

why not use the open command?

Code:

cd /path/to/directory

ls  *.img | open -t -f


Just tested it and it works for me.

hayne 01-26-2009 02:05 PM

Quote:

Originally Posted by cswenson (Post 515559)
Here's the output:

Code:

~/UBS/test/uvot/image > /bin/ls *.img | /usr/bin/vis
sw00340895992ubb_sk.img
sw00340895992um2_sk.img
sw00340895992uuu_sk.img
sw00340895992uvv_sk.img
sw00340895992uw1_sk.img
sw00340895992uw2_sk.img
sw00340895992uwh_sk.img
/bin/ls *.img > files.txt
/usr/bin/hexdump -C files.txt~/UBS/test/uvot/image > /bin/ls *.img > files.txt
~/UBS/test/uvot/image > /usr/bin/hexdump -C files.txt
00000000  73 77 30 30 33 34 30 38  39 35 39 39 32 75 62 62  |sw00340895992ubb|
00000010  5f 73 6b 2e 69 6d 67 0a  73 77 30 30 33 34 30 38  |_sk.img.sw003408|
00000020  39 35 39 39 32 75 6d 32  5f 73 6b 2e 69 6d 67 0a  |95992um2_sk.img.|
00000030  73 77 30 30 33 34 30 38  39 35 39 39 32 75 75 75  |sw00340895992uuu|
00000040  5f 73 6b 2e 69 6d 67 0a  73 77 30 30 33 34 30 38  |_sk.img.sw003408|
00000050  39 35 39 39 32 75 76 76  5f 73 6b 2e 69 6d 67 0a  |95992uvv_sk.img.|
00000060  73 77 30 30 33 34 30 38  39 35 39 39 32 75 77 31  |sw00340895992uw1|
00000070  5f 73 6b 2e 69 6d 67 0a  73 77 30 30 33 34 30 38  |_sk.img.sw003408|
00000080  39 35 39 39 32 75 77 32  5f 73 6b 2e 69 6d 67 0a  |95992uw2_sk.img.|
00000090  73 77 30 30 33 34 30 38  39 35 39 39 32 75 77 68  |sw00340895992uwh|
000000a0  5f 73 6b 2e 69 6d 67 0a                          |_sk.img.|
000000a8
~/UBS/test/uvot/image >

Does this provide you a clue?

That seems to show that everything is completely normal.
Look at that files.txt file with any text editor and you should see that it is as you would expect.
So I think the problem is with your 'ls' or 'more' commands - maybe (as Hal Itosis suggested) one or both of these is actually an alias or a function.
What do you get from:

type -a ls
type -a more

ataraxia 01-26-2009 02:05 PM

You are getting colorized output in your file for some reason. I have color turned on as well, but it doesn't do it while I write a file. Are you forcing ls colors somehow?

cswenson 01-26-2009 02:30 PM

Unfortunately everything is not normal. If I open the file in (fill in blank with favorite text editor, I tried several) I get this:

Code:

ESC[0mESC[0msw00340895992ubb_sk.imgESC[0m
ESC[0msw00340895992um2_sk.imgESC[0m
ESC[0msw00340895992uuu_sk.imgESC[0m
ESC[0msw00340895992uvv_sk.imgESC[0m
ESC[0msw00340895992uw1_sk.imgESC[0m
ESC[0msw00340895992uw2_sk.imgESC[0m
ESC[0msw00340895992uwh_sk.imgESC[0m
ESC[m

Something is definitely wrong. I originally realized there was a problem with I tried to parse the text file in idl. I'm trying to read in the file, line be line, as strings, but it can't be interpreted as a string with the extra formatting characters.

I'm really confused as to what's going on here.

cswenson 01-26-2009 02:40 PM

ataraxia: Yes, I do have have ls aliased to show colors, but if I disable the alias I still get the garbled output.

cwtnospam 01-26-2009 02:43 PM

Is your character set encoding Unicode (UTF-8)?

trevor 01-26-2009 02:44 PM

Quote:

Originally Posted by Hal Itosis (Post 515554)
Perhaps ls is defined as a function or an alias:

type -a ls

?

Quote:

Originally Posted by hayne (Post 515569)
So I think the problem is with your 'ls' or 'more' commands - maybe (as Hal Itosis suggested) one or both of these is actually an alias or a function.
What do you get from:

type -a ls
type -a more

Why have you not responded to Hal Itosis and hayne's queries? Please do so.

Trevor

hayne 01-26-2009 02:48 PM

Quote:

Originally Posted by cswenson (Post 515578)
Unfortunately everything is not normal. If I open the file in (fill in blank with favorite text editor, I tried several) I get this:

Code:

ESC[0mESC[0msw00340895992ubb_sk.imgESC[0m
ESC[0msw00340895992um2_sk.imgESC[0m
ESC[0msw00340895992uuu_sk.imgESC[0m
ESC[0msw00340895992uvv_sk.imgESC[0m
ESC[0msw00340895992uw1_sk.imgESC[0m
ESC[0msw00340895992uw2_sk.imgESC[0m
ESC[0msw00340895992uwh_sk.imgESC[0m
ESC[m


Sorry - which file is this?
I was speaking of the file "files.txt" that held the output of the '/bin/ls' command.
It looks like you are still looking at the output of your (alias for) 'ls' command.

The hexdump output you showed above makes it clear that the "files.txt" file is correct.

So, I repeat, the problem is with whatever alias you are using for 'ls'.

By the way, note that it is a bad idea to use the name of an existing command for an alias - i.e. to try to replace an existing command with an alias. You should use a different name for your alias - e.g. something like 'lsc', so that it is clear that this is not the standard 'ls' command.

tlarkin 01-26-2009 03:02 PM

When I need to out put something to a text file from terminal I typically use the open command, and since I dislike textedit over other text editors I use the -a switch followed by the application name.

example

Code:

tail -50 /var/log/system.log | open -a "TextWrangler.app"
That will output everything right into text wrangler and I can attach it to an email or what not and send it out.

hayne 01-26-2009 03:10 PM

Quote:

Originally Posted by tlarkin (Post 515588)
When I need to out put something to a text file from terminal I typically use the open command, and since I dislike textedit over other text editors I use the -a switch followed by the application name.

Unless you want to edit the text file, there is no reason to use 'open' to create the text document in an application.
The simple use of '>' to redirect the output to a file (as in the topic of this thread) is sufficient to create the text file wherever you choose.

cswenson 01-26-2009 03:10 PM

Problem solved!

I just talked to our system admin, and he said this wasn't the first problem that had been reported today involving commonly used command line tasks and he traced the problem back to a corrupt file that was improperly patched over the weekend on our mac server.

Apparently my issue was relatively tame in comparison to things that other people were dealing with.

Thanks everyone for the help.

hayne 01-26-2009 03:19 PM

Quote:

Originally Posted by cswenson (Post 515591)
I just talked to our system admin, and he said this wasn't the first problem that had been reported today involving commonly used command line tasks and he traced the problem back to a corrupt file that was improperly patched over the weekend on our mac server.

Maybe for the benefit of future readers (and to satisfy my curiosity), you could explain how this "mac server" was involved in the commands you showed above.
Is your home folder on this server (as opposed to being on the local Mac)?
Were you remotely logged into this server?
And why didn't the problem on this server affect the commands that I suggested (in post #4) which worked correctly?

tlarkin 01-26-2009 03:23 PM

Quote:

Originally Posted by hayne (Post 515590)
Unless you want to edit the text file, there is no reason to use 'open' to create the text document in an application.
The simple use of '>' to redirect the output to a file (as in the topic of this thread) is sufficient to create the text file wherever you choose.

I just like the pretty colors that textwrangler puts the fonts in and I like to choose where to save it, I keep different things in different directories and the GUI for me makes it easier.

It is just a preference is all.

cswenson 01-26-2009 03:33 PM

It was a combination of various factors:

IDL is run from the main server (that way we only need one license), so I was ssh-ed into the server and the problem resided on the server. I was using my own local machine terminal when I entered the commands you suggested in post #4 and therefore using my own machines /bin/ls, etc. For the time being I'm now able to create my files.dat, then ssh into the server, then run IDL (on my server home folder) using the now correct files.dat.

I guess that it is my own fault for not perfectly recreating the situation when you asked my to run those commands.

sox 01-27-2009 02:03 PM

Do this
 
here is the solution I use.

instead of

ls *.txt

add a slash:

\ls *.txt

it appears that ls is alaised somehow to run through a formatter. Using the slash causes it to use the unaliased generic ls.

hayne 01-27-2009 02:23 PM

Quote:

Originally Posted by sox (Post 515765)
it appears that ls is aliased somehow to run through a formatter

The "somehow" must be something that you configured (in the shell "dot" files - see the shell config section of this Unix FAQ)
or something that you installed.

Hal Itosis 01-27-2009 05:37 PM

Quote:

Originally Posted by sox (Post 515765)
Using the slash causes it to use the unaliased generic ls.

That's pretty neat (and news to me anyway).
I knew about the "command" command (for bypassing shell builtins and/or user-defined functions in scripts).

But why does the \backslash work?
Doesn't appear to be a keyword...
$ type \
>
[^C]
$ type \\
-bash: type: \: not found

$ type '\'
-bash: type: \: not found

--

Hmm, apparently the \ has the same effect as *quoting* the text.
So, typing...
'ls'
would have the same effect: bypass the alias and run the standard utility.

Still not clear why that works tho.

hayne 01-27-2009 06:39 PM

The page http://www.chiark.greenend.org.uk/~s...m/aliases.html (about special uses of aliases) explains:
Quote:

Originally Posted by above web page
It's worth bearing in mind that you can bypass any alias definition by prefixing it with a backslash: instead of starting your command with ‘find’, use ‘\find’. This works because alias lookup happens before backslash escapes are processed, so the shell actually looks for an alias called ‘\find’, which doesn't exist.


Hal Itosis 01-27-2009 07:12 PM

Interesting page!

I can relate to his "find" example there.
For a few scripts, i've even used this heading:
#!/bin/bash -f -

to kill globbing within the whole script... so that stuff like "*.txt" can be
passed to internal functions without expanding (if something in the cwd
happened to match).


All times are GMT -5. The time now is 10:17 PM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
Site design © IDG Consumer & SMB; individuals retain copyright of their postings
but consent to the possible use of their material in other areas of IDG Consumer & SMB.