It is currently Tue, 07 Dec 2021 02:58:01 GMT

Author Message
 redirecting stdout to file over NFS slows dramatically

Here's the experiment:
on an 880 (750MHz CPUs, 32GB memory)
I have a trivial C program with writes 80 character lines to a file.
It writes 10 *1024*1024 lines.  So that's 800 MB in the file.

The file (NFS) server is an E450.  4 CPUs 480MHz, 4 GB memory.  Doesn't
run memory-starved that I've ever noticed.  A gigabit connection
between them.  There might be trouble on the network, but I kind
of doubt it--but mostly, I haven't gotten to that question yet.
(The gigabit goes through some switches and a router.)

rusage reports about 15 seconds.  The elapsed time between date output
is anywhere from 45 to over 300 seconds.

If I rewrite "program" so that it reads:

Then, in this case, I do NOT see any slowdown.  That is, rusage
might say 28 seconds, the e.t. between date output is also 28 seconds.

In other words, if I redirect to stdout, to a file on an NFS-mounted
file system, the total wall clock time increases.  The fopen...
does NOT have this problem.  (I've got over 20 similar experiments.)
I've seen this with csh and sh.

Also, when the "slowed" experiment is going on, OTHER processes
have file access problems.  For example, vi can't save a file
over NFS until the 800 MB file is finished writing.  Oh, and
until the 45-300 second period is over, the actual size of
the file on the file server is growing.  The writing speed on the
file server is faster than the speed of the NFS write--that is,
the limiting factor is NOT disk speed on the file server.
So far, I have only seen this slowdown betweeen my 880
and my E450.  I've done this between other machines, but
the speed differential between the other machines is so big that
those experiments don't prove anything.

Any ideas about what's going on?

This came up because someone wrote a script which showed terrible
NFS performance (all they knew was that it took "too long")
but there wasn't an obvious use of stdout redirection.  So this
may not be limited to stdout and NFS.  Again, I haven't had
time to check that, yet.

Jay Scott               512-835-3553  
Head of Sun Support, Sr. Operating Systems Specialist
Applied Research Labs, Computer Science Div.                   S224
University of Texas at Austin

 Sat, 07 May 2005 04:53:41 GMT   
 redirecting stdout to file over NFS slows dramatically (Jay G. Scott) wrote in

Without reading your OP too awfully carefully, could it be that piping things
thru STDOUT is eliminating bufferring and going back to character oriented
I/O, whereas straight memory to disk I/O is being bufferred?  Character
oriented I/O to NFS would be sinfully slow since the buffers are single
character, and every one would have packet overhead involved.  I don't
rembmer how NFS deals with buffered vs. unbuffered operations so I'd be
interested if someone knows different than what I proposed.

-- Rob Prowel (A.K.A. da wiseguy)

-----= Posted via Newsfeeds.Com, Uncensored Usenet News =----- - The #1 Newsgroup Service in the World!
-----==  Over 80,000 Newsgroups - 16 Different Servers! =-----

 Sat, 07 May 2005 04:55:10 GMT   
 redirecting stdout to file over NFS slows dramatically
In article <arbk0l$>,
Jay G. Scott <> wrote:

program | dd of=yyy bs=1024k

I bet stdout is not buffered...


 Sat, 07 May 2005 06:27:46 GMT   
 redirecting stdout to file over NFS slows dramatically
In article <arbph2$>,

tried that.  what i found was that adding dd increased the
time measured by rusage, but nothing performed any better.
that is, rusage appears to have correctly added in the extra
cycles and time used by dd, but the actual writing of
the file on the NFS server was not affected.

rusage program > yyy
takes about 17 secs, NFS write completes in ~45 secs.

rusage program | dd of=yyy bs=1024k
takes about 22 secs, NFS write completes in ~45 secs.

So, if anything, we might be excessively buffering.

I take some comfort in seeing that adding dd took more
measured time, for some reason....  I suppose I feel
that way because I would expect overhead to be added,
and rusage concurs.

This does give me more info....  since I know things
went through dd.  I know from other experiments, that
redirecting to a local disk doesn't have this problem.
That means I have trouble either piping AND redirecting
from that program, OR dd with of= does something as
clumsy as my program (I doubt that).  OR it means
whatever is buffering/writing over NFS is having a problem.

Hmmm.  Since fopen... doesn't have the problem over
NFS, I think it must be redirecting and piping.

Anybody see something I don't?

Jay Scott               512-835-3553  
Head of Sun Support, Sr. Operating Systems Specialist
Applied Research Labs, Computer Science Div.                   S224
University of Texas at Austin

 Sat, 07 May 2005 07:09:25 GMT   
 redirecting stdout to file over NFS slows dramatically

I think if you do this, the dd is going to allocate a 1024k buffer
for blocks, then issue a read() that requests 1024k but gets a much
smaller amount of data (since the max size of I/O through a pipe
is like 8k or something), and then call write() with an amount much
smaller than 1024k.  In other words, I think it will write exclusively
partial blocks of the same size it read from the pipe (i.e. small).

Instead, you could try this (note the "obs" instead of "bs):

    rusage program | dd of=yyy obs=1024k

to set the output block size.

Actually, though, I have a hunch you'll get better performance if
you use a smaller size than 1024k, say 128k or 256k.

  - Logan

I'm currently looking for work as a Unix/Solaris
administrator, or Perl/C++/Java developer.  Resume

 Sat, 07 May 2005 11:39:59 GMT   
   [ 5 post ] 

Similar Threads

1. redirecting STDOUT of a bg process to STDOUT of a new xterm

2. Disk access dramatically slows down Linux

3. Problem redirecting stdout to pipes/files

4. Redirecting to a file AND stdout

5. curses problem: displaying to stdout when input redirected from a file

6. bash : how to redirect both stdout and stderr to append a file

7. Redirect stderr and stdout in two separate files?

8. Redirecting stdout and stderr to file as well as screen for Korn shell

9. Redirect stdout * stderr to one file in bourne sh using exec

10. redirecting stdout to two (or more) files

Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software