It is currently Tue, 30 Nov 2021 18:35:34 GMT

Author Message
 reading files line-be-line?
Is there a unix utility that will return a file line-by-line? I'm trying
to do a loop like:

for somevariable in `cat somefile`
    #do something with $somevariable

but that loop would iterate once for each white-space-delimited token in
"somefile". I want it to iterate once for each line of somefile. What
could I replace the cat utility with?


 Fri, 09 Sep 2005 05:38:10 GMT   
 reading files line-be-line?

while read LINE; do
        echo "--${LINE}--"
done < file


cat file | while read LINE; do
        echo "--${LINE}--"


 Fri, 09 Sep 2005 05:47:02 GMT   
 reading files line-be-line?

Oooh, you're brave to post that example.  I'll be surprised if "useless
uses of cat gods' don't strike you down for that one!  :-)

David Means

Never frobnicate without first grokking.

 Fri, 09 Sep 2005 12:43:59 GMT   
 reading files line-be-line?
[This followup was posted to]

In article <>, Albert J.
( says...

while read somevariable
    # do something
done <somefile


Barry Kimelman
Winnipeg, Manitoba, Canada
email :

 Fri, 09 Sep 2005 22:25:48 GMT   
 reading files line-be-line?

There should be no problem, it clearly shows the useless use of cat.


 Sat, 10 Sep 2005 01:06:50 GMT   
 reading files line-be-line?

Anyway while-read loops are BUOS (bad use of shell), so nobody
would notice the UUOC.


 Sat, 10 Sep 2005 02:41:44 GMT   
 reading files line-be-line?

How else can you read <stdin> ?

William Park, Open Geometry Consulting, <>
Linux solution for data management and processing.

 Sat, 10 Sep 2005 03:42:09 GMT   
 reading files line-be-line?

Almost every text utility (read is one of them) can read its
standard output. "read" only reads one line at a time one
character at a time and performs further processing on it,
that's three good reasons for not using it in a shell script for
text processing.

Use and pipe utilities that read _all_ the input by chunks.

Running one (or several) utilities per line of input is really
bad practice.


 Sat, 10 Sep 2005 04:30:51 GMT   
 reading files line-be-line?

Well, reading character by character is not efficient.  But, how would
you re-write

    while read a b c; do
    done < file

using other utilities?

William Park, Open Geometry Consulting, <>
Linux solution for data management and processing.

 Sun, 11 Sep 2005 03:52:45 GMT   
 reading files line-be-line?

You mean its standard INput, don't you?

You have a good point when you discourage the practice of
executing an external command for every line of a file,
but the above just doesn't seem right.

Are you saying the shell doesn't use buffered file reads,
and the OS' VM system doesn't perform efficient large reads
from disk?

Tracing the /bin/sh on my Solaris 8 machine indicates otherwise.

:::::::::::::  Greg Andrews  :::::  :::::::::::::
     I have a map of the United States that's actual size.
                                -- Steven Wright

 Sun, 11 Sep 2005 06:17:58 GMT   
 reading files line-be-line?

awk '...' < file


utility1 | ... | utilityn

You may find the reading of this book useful:


 Sun, 11 Sep 2005 15:45:06 GMT   
 reading files line-be-line?

Yes, typo, sorry.

I say that read(1) reads one character at a time (one read(2)
system call per character). That doesn't mean that only one
character is read from disk when input is on a hard drive.

read does this because it has to read only one line, so it must
ensure it doesn't eat any additionnal byte once it has read the
trailing '\n'. The shell can't buffer it's input because
external utilities wouldn't have access to this "buffer".

~$ strace -e read sh -c 'read < a'
read(0, "#", 1)                         = 1
read(0, "!", 1)                         = 1
read(0, " ", 1)                         = 1
read(0, "/", 1)                         = 1
read(0, "b", 1)                         = 1
read(0, "i", 1)                         = 1
read(0, "n", 1)                         = 1
read(0, "/", 1)                         = 1
read(0, "s", 1)                         = 1
read(0, "h", 1)                         = 1
read(0, "\n", 1)                        = 1


 Sun, 11 Sep 2005 15:45:10 GMT   
 reading files line-be-line?

You don't answer the question: How do you read <stdin>?
You will need it if the script will be started by inetd and such.

I kind of like chuncks in record size, preferably subdivided into
fields. And with blank or spaces as field separator and EOL as record
separator, read does a fairly good job...

Some tools need to be run for every line. I frequently work with a list
of hostnames to do something on each host. Hard to avoid to run the
utility on each line...

I just like to say: Yes, `cat` can be regarded as useless however, it
adds some readability. I cannot follow you on the bad-use-of-shell...
(it's the first time I read about it)


 Mon, 12 Sep 2005 23:08:41 GMT   
 reading files line-be-line?

head -10
reads 10 lines (and maybe more as it may read by buffers) from
stdin, whatever stdin is.

while [ $c -lt 10 ]; do
  read line
  echo "$line"
  c=`expr $c + 1`

does the same except it may fork 30 processes and reads the
input one byte at a time, and strips leading and trailing blanks
and processes \ sequences (both in read and echo), may fail on
lines like "-n"...

That's what I call a BUOS.

awk '{print NR ": " $2}'

is better than:

while read line; do
  echo "$c: $line"
  c=`expr $c + 1`

awk does the same more reliably and more efficiently.


Yes, that's why I say while-read shell loops are /generally/ bad
practice. In some cases, it's useful.

If it's for text processing, I consider it BUOS, as there are
tools to perform text processing, and a shell is a tool designed
to run tools more than a programming language.


 Mon, 12 Sep 2005 23:51:48 GMT   
   [ 14 post ] 

Similar Threads

1. read from stdin while reading line by line a file

2. line by line reading of the lines in list

3. Reading a script file line by line.

4. Using read within a loop that parses a file line by line

5. read a file line by line in csh script

6. Read a file line after line

7. reading line by line from a file

8. reading a file line by line

9. read a file line by line

10. ksh script --reading file line by line?

Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software