BorlandTalk.com Forum Index BorlandTalk.com
Borland discussion newsgroups
 
Archives   FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

fgets() vs std::getline() performance
Goto page 1, 2, 3  Next
 
Post new topic   Reply to topic    BorlandTalk.com Forum Index -> C++ Builder (Language C++)
View previous topic :: View next topic  
Author Message
crhras
Guest





PostPosted: Sat Sep 16, 2006 7:01 pm    Post subject: fgets() vs std::getline() performance Reply with quote



I have noticed a big difference in performance between fgets() and
std::getline() using Borland Studio on Windows XP Pro. fgets() is much
faster. The performance problem seems to be with the compiler/OS because a
test program that I use doesn't exhibit the same speed difference with gcc++
on Linux and other OSs.

Here is a description of the environment with the test code I use and the
results I am getting.

The input data file used contains 3.5 million text records of varying
lengths terminated by '\n'.

I run the tests twice to make sure that caching is not responsible for the
time difference.

-----------------------------------
{
time_t start, end;
double dif;

std::string line;
std::ifstream in("c:\\Data\\IVData.csv");

time(&start);

while (in.getline(line, 1024))
{
}
time(&end);
in.close();

dif = difftime (end,start);
cxMemo1->Lines->Add((AnsiString)"Test 1 has taken " + dif + " seconds.");

FILE * fp;
char cline[512];
fp = fopen("c:\\Data\\IVData.CSV", "r");

time(&start);
while (fgets(cline, 512, fp) != NULL)
{
}
time(&end);
fclose(fp);

dif = difftime (end,start);
cxMemo1->Lines->Add((AnsiString)"Test 2 has taken " + dif + " seconds.");
}
-----------------------------------

Results :
Test 1 has taken 203 seconds.
Test 2 has taken 5 seconds.
Test 1 has taken 201 seconds.
Test 2 has taken 4 seconds.

Back to top
Hans Galema
Guest





PostPosted: Sat Sep 16, 2006 7:48 pm    Post subject: Re: fgets() vs std::getline() performance Reply with quote



crhras wrote:

Quote:
std::string line;
std::ifstream in("c:\\Data\\IVData.csv");
while (in.getline(line, 1024))
{
}

in.close();

Try this:

ifstream ifs(filename.c_str(), ios::in|ios::app);

std::string buffer;

while ( getline( ifs, buffer) )
{
}

ifs.close();


Hans.
Back to top
Clayton Arends
Guest





PostPosted: Sat Sep 16, 2006 8:46 pm    Post subject: Re: fgets() vs std::getline() performance Reply with quote

Please don't crosspost/multipost your messages. Click here for a complete
list of the newsgroup guidelines:

http://info.borland.com/newsgroups/guide.html

- Clayton
Back to top
crhras
Guest





PostPosted: Sat Sep 16, 2006 9:12 pm    Post subject: Re: fgets() vs std::getline() performance Reply with quote

"Clayton Arends" <nospam_claytonarends (AT) hotmail (DOT) com> wrote in message
news:450c1c37$1 (AT) newsgroups (DOT) borland.com...
Quote:
Please don't crosspost/multipost your messages. Click here for a complete
list of the newsgroup guidelines:

http://info.borland.com/newsgroups/guide.html

- Clayton


Sorry, I posted accidentally to the wrong newsgroup and tried to cancel the
post but wasn't able to.
Back to top
crhras
Guest





PostPosted: Sat Sep 16, 2006 9:33 pm    Post subject: Re: fgets() vs std::getline() performance Reply with quote

"Hans Galema" <notused (AT) notused (DOT) nl> wrote in message
news:450c104a$1 (AT) newsgroups (DOT) borland.com...
Quote:
crhras wrote:

std::string line;
std::ifstream in("c:\\Data\\IVData.csv");
while (in.getline(line, 1024))
{
}

in.close();

Try this:

ifstream ifs(filename.c_str(), ios::in|ios::app);

std::string buffer;

while ( getline( ifs, buffer) )
{
}

ifs.close();


Hans.

Thanks for your suggestion. I tried it and it helps (takes about 25% off of
the total time) but getline() still doesn't come even close to fgets(); I
am guessing that this is a flaw in the implementation of getline( ).
Back to top
Nate
Guest





PostPosted: Sun Sep 17, 2006 8:10 am    Post subject: Re: fgets() vs std::getline() performance Reply with quote

Do you get the same results with the order of the tests reversed?

Nate
Back to top
crhras
Guest





PostPosted: Sun Sep 17, 2006 7:22 pm    Post subject: Re: fgets() vs std::getline() performance Reply with quote

"Nate" <RNLockwoodButParseThis (AT) gmail (DOT) com> wrote in message
news:450cc2a2$1 (AT) newsgroups (DOT) borland.com...
Quote:
Do you get the same results with the order of the tests reversed?

Nate

Yes, since I run each of the tests twice (ie. test1 -> test2 -> test1 ->
test2),

I think that the order of the tests is effectively swapped, right ?
Back to top
Nate
Guest





PostPosted: Mon Sep 18, 2006 8:09 am    Post subject: Re: fgets() vs std::getline() performance Reply with quote

crhras wrote:

Quote:
Yes, since I run each of the tests twice (ie. test1 -> test2 -> test1 -
test2),

I think that the order of the tests is effectively swapped, right ?


Well, I guess so if the file is buffered to the same memory. I found
that for one of my programs that the second, third, etc. time that I ran
it it ran faster than the first time. I inferred that this was because
large chunks of the data were in memory - but I could be wrong. I
didn't see any time differences like yours, though.

Nate
Back to top
Chris Uzdavinis (TeamB)
Guest





PostPosted: Mon Sep 18, 2006 9:12 pm    Post subject: Re: fgets() vs std::getline() performance Reply with quote

"crhras" <crhras (AT) sbcglobal (DOT) net> writes:

Quote:
Thanks for your suggestion. I tried it and it helps (takes about 25% off of
the total time) but getline() still doesn't come even close to fgets(); I
am guessing that this is a flaw in the implementation of getline( ).

One difference is that getline writes to an std::basic_string object,
while fgets is writing into a fixed-size array that is declared on the
stack. Perhaps the real efficiency bottleneck is in the string class
and not in the getline() function itself? The string class is,
afterall, using dynamic memory.

--
Chris (TeamB);
Back to top
crhras
Guest





PostPosted: Tue Sep 19, 2006 8:10 am    Post subject: Re: fgets() vs std::getline() performance Reply with quote

"Chris Uzdavinis (TeamB)" <chris (AT) uzdavinis (DOT) com> wrote in message
news:86fyepkvcd.fsf (AT) explicit (DOT) atdesk.com...
Quote:
"crhras" <crhras (AT) sbcglobal (DOT) net> writes:

Thanks for your suggestion. I tried it and it helps (takes about 25% off
of
the total time) but getline() still doesn't come even close to fgets();
I
am guessing that this is a flaw in the implementation of getline( ).

One difference is that getline writes to an std::basic_string object,
while fgets is writing into a fixed-size array that is declared on the
stack. Perhaps the real efficiency bottleneck is in the string class
and not in the getline() function itself? The string class is,
afterall, using dynamic memory.

--
Chris (TeamB);

I would have no idea what the cause is but a 4000 percent difference in
performance seems out
of line to me and points to a flaw in the implementation of getline. Also,
when fgets is compared
to getline using gc++ on Linux there is some performance difference but
nothing to write home about.
Back to top
crhras
Guest





PostPosted: Wed Sep 20, 2006 6:51 am    Post subject: Re: fgets() vs std::getline() performance Reply with quote

Quote:

Exactly this had been explained to crhas over in
c.l.c++.m already.

Schobi


Schobi,

I don't know who you are but it seems that you are taking it
personally when I say that getline( )'s implementation is flawed. A lot
has been explained to me in this thread and I am grateful for
everyone's responses. It doesn't seem to me however, that the
explanation that "strings do more" than char buffers or getline( ) writes
into a string rather than a fixed size array should account for a
performance difference of 4000 percent.

Do you ?
Back to top
Chris Uzdavinis (TeamB)
Guest





PostPosted: Wed Sep 20, 2006 6:57 am    Post subject: Re: fgets() vs std::getline() performance Reply with quote

"crhras" <crhras (AT) sbcglobal (DOT) net> writes:

Quote:
I would have no idea what the cause is but a 4000 percent difference
in performance seems out of line to me and points to a flaw in the
implementation of getline. Also, when fgets is compared to getline
using gc++ on Linux there is some performance difference but nothing
to write home about.

Without profiling, it's hard to say where the time is spent. But it's
not a fair comparison between linux/g++ and windows/BCB. Different
os, different compiler (with different levels of optimizations... g++
probably much more aggressive), and last but not least, different
standard library implementations.

Dinkumware (BCB) has a great implementation, but I have heard it
expects the compiler to optimize away and inline things that Borland
might not be doing.

Also, I'm not making excuses for Borland, but based on what you have
said I do disagree with your conclusions that it is a flaw in
getline. It may later turn out you are right, but the supporting
arguments you are currently using are inadequate to draw any
conclusions about anything except that your program takes a lot
longer using getline.

--
Chris (TeamB);
Back to top
crhras
Guest





PostPosted: Wed Sep 20, 2006 8:10 am    Post subject: Re: fgets() vs std::getline() performance Reply with quote

"Chris Uzdavinis (TeamB)" <chris (AT) uzdavinis (DOT) com> wrote in message
news:86bqpbe1vc.fsf (AT) explicit (DOT) atdesk.com...
Quote:
"crhras" <crhras (AT) sbcglobal (DOT) net> writes:

I would have no idea what the cause is but a 4000 percent difference
in performance seems out of line to me and points to a flaw in the
implementation of getline. Also, when fgets is compared to getline
using gc++ on Linux there is some performance difference but nothing
to write home about.

Without profiling, it's hard to say where the time is spent. But it's
not a fair comparison between linux/g++ and windows/BCB. Different
os, different compiler (with different levels of optimizations... g++
probably much more aggressive), and last but not least, different
standard library implementations.

Dinkumware (BCB) has a great implementation, but I have heard it
expects the compiler to optimize away and inline things that Borland
might not be doing.

Also, I'm not making excuses for Borland, but based on what you have
said I do disagree with your conclusions that it is a flaw in
getline. It may later turn out you are right, but the supporting
arguments you are currently using are inadequate to draw any
conclusions about anything except that your program takes a lot
longer using getline.

--
Chris (TeamB);

Thanks Chris - good stuff.

Curt
Back to top
Duane Hebert
Guest





PostPosted: Thu Sep 21, 2006 2:28 am    Post subject: Re: fgets() vs std::getline() performance Reply with quote

"Ed Mulroy" <dont_email_me (AT) bitbuc (DOT) ket> wrote in message
news:4511448e (AT) newsgroups (DOT) borland.com...
Quote:
The machine I'm on right now only goes up to BCB6 and doesn't have BDS2006
on it.

I used the code as shown and, as BCB6 refuses the 'inline' of the function
containing the 'for' loop, commented out the word 'inline' so the warning
doesn't clutter up the screen capture.

Note that 'reserve' was not used.

The results were:

The 1.2M, 25,000 line test file took these times for getline/fgets:

122/20 dynamic linked debug on
36/20 dynamic linked debug off
129/21 static linked debug on
36/20 static linked debug off

So yours is more in line with expectations.
I imagine since you have BCB6 it's dinkumware.

Which BCB and which stl is the op using?
Back to top
Duane Hebert
Guest





PostPosted: Thu Sep 21, 2006 2:30 am    Post subject: Re: fgets() vs std::getline() performance Reply with quote

"Hendrik Schober" <SpamTrap (AT) gmx (DOT) de> wrote in message
news:45114521 (AT) newsgroups (DOT) borland.com...
Quote:
I just put a
line.reserve(1024)
into the 'test_getline()' function (first version,
without the output) and it didn't change all that
much.

I changed my optimization to be for speed
and the time was a bit less and reserve didn't
help as much.


FWIW, we do a lot of parsing of flat files
and I've never seen this as a bottleneck -
at least not more than you would expect
for file i/o.
Back to top
Post new topic   Reply to topic    BorlandTalk.com Forum Index -> C++ Builder (Language C++) All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
 


Powered by phpBB © 2001, 2006 phpBB Group
.