Re: Buffered reading of stream files

[Adam Glauser <adamglauser@xxxxxxxxx>]

Scott Klement wrote:
> Do you mean that you want to read your file using fgets()?   If so, 
> please be aware that fgets() can only be configured to read lines of 
> text delimited by either LF or CRLF.  It can't be used with any 
> arbitrary delimiter.
>
> if you really mean fread(), then I'd suggest setting up some loops and 
> benchmarking the diff.  Essentially, fread() and read() work the same -- 
> except that fread buffers the data.  Will it perform better than 
> buffering it yourself?  Hard to say without benchmarking...  and I 
> haven't done any.

Okay, so I got some code working with fread().  Code's still a bit 
rough, but in case any one is interested, here's what I found.

buffread0: http://code.midrange.com/9def61bedb.html
buffread1: http://code.midrange.com/1e70379662.html

I ran each five times in a row, reading a 5.98 MB file with two repeated 
 records of differing length.  Times in microseconds.

buffread0:		buffread1:
2373000			5799000
2368000			5831000
2368000			5810000
2367000			5718000
2352000			5737000

So it looks like either I'm doing something really dumb (likely) or 
fread() is quite a bit heavier than open() with roll-your-own buffering 
(less likely).

As Simon pointed out, I'm essentially having to double buffer due to not 
wanting to make assumptions about record length.  I suspect the results 
would be much closer if bufferread1 were to read a larger chunk, and 
handle the record delimiter finding differently.