BorlandTalk.com Forum Index BorlandTalk.com
Borland discussion newsgroups
 
Archives   FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Strings

 
Post new topic   Reply to topic    BorlandTalk.com Forum Index -> Delphi Thirdparty Tools (General)
View previous topic :: View next topic  
Author Message
Allan Brocklehurst
Guest
PostPosted: Wed Jun 07, 2006 11:22 pm    Post subject: Strings Reply with quote


Hi;
Hi
stats : D7 XPpro - on a fast laptop lots of ram

What I am doing:

I'm reading through a text file , assigning the record to a variable
and parsing it into fields.
The records I'm reading in is of variable length.
In this case the record I'm reading into a variable is 10258 characters.
Problem is that I'm only getting a max of 4096 characters into the variable.
if I define the variable as String but an additional 12000+ characters
are added to the end

Code
var line :string;

Readln(TempFile, Line);

Any help would be greatly appreciated

Allan Brocklehurst
Back to top
Ed Vander Hoek
Guest
PostPosted: Wed Jun 07, 2006 11:59 pm    Post subject: Re: Strings Reply with quote


On 2006.06.07 11:22, Allan Brocklehurst wrote:
Quote:
In this case the record I'm reading into a variable is 10258 characters.
Problem is that I'm only getting a max of 4096 characters into the
variable.

Sounds like a buffer size is being exceeded somewhere. I know that
Windows defaults to using 4096 as the buffer size for COM ports - I
don't know if it also applies to text files. (Com ports are opened with
the same Windows API CreateFile function that is used to open files.)

The following code works fine under Delphi 7 and Windows 2003:
procedure TForm1.Button1Click(Sender: TObject);
var
f: TextFile;
s: string;
begin
assignFile(f,'test.txt');
reset(f);
readln(f,s);

memo.Lines.Add('Length of first line is ' + inttostr(length(s)) );
memo.Lines.Add(s);
closeFile(f);
end;

I don't know how big your files are, but an alternative approach is to
read the entire file at once. eg:
function ReadStringFromFile(const fileName: string): string;
var
f: File;
size: integer;
begin
assignFile(f,fileName);
reset(f,1);
try
size := FileSize(f);
SetLength(result,size);
BlockRead(f,result[1],size);
finally
CloseFile(f);
end;
end;


--
Ed Vander Hoek
web: www.getawaymaps.com
email: ed.vanderhoek at ...
Gastown Command Line Parser
Back to top
Peter Morris [Droopy eyes
Guest
PostPosted: Thu Jun 08, 2006 12:32 am    Post subject: Re: Strings Reply with quote


Quote:
In this case the record I'm reading into a variable is 10258 characters.
Problem is that I'm only getting a max of 4096 characters into the
variable.

This sentence suggests that your problem is you are reading a 10258 char
line but Length(Line) returns 4096?


Quote:
if I define the variable as String but an additional 12000+ characters
are added to the end

I have no idea what this means.

Can you post the read code? The definition of the variables + the loop
statement + the read statement + the line of code where you check the
length. I presume you are checking the length immediately after the line is
read and not after other code has executed which may alter the value?


--
Pete
====
Audio compression components, DIB graphics controls, ECO extensions,
FastStrings
http://www.droopyeyes.com

My blog
http://blogs.slcdug.org/petermorris/
Back to top
Clinton R. Johnson
Guest
PostPosted: Thu Jun 08, 2006 1:39 am    Post subject: Re: Strings Reply with quote
Stop using the OLD (very old) IO routines that use ReadLn And Writeln.
They are limited (such as their input buffer size which you've run
across) You need to start learning about streams for IO and even file
support for native objects like TStringList.

In this case, however, you could just load the whole file into a
TStringList and get the length for the first filename.

Function GetFirstLineLength(FileName : String) : Integer;

Var
S : TStringList;

Begin
S := TStringList.Create;
Try
S.LoadFromFile(FileName);
Result := Length(S[0])
Except
REsult := 0;
End;
S.Free;
End;
OF course, you could do more advanced checking instead of the QnD I did
here, or you could start loading data in through a TFileStream and
seeking forward until you find a #13 or #10, counting how many chrs
went before it.

** WARNING ** TStringList does a few funny things if it runs across a
null in a file it is reading, so custom routines that use streams can
often be more useful.

You could also try :


Function GetFirstLineLength(FileName : String) : Integer;

Var
M : TMemoryStream;
I : Integer;
P : PChar;

Begin
M := TMemoryStream.Create;
Try
M.LoadFromFile(FileName);
P := PChar(M.Memory);
Result := 0;
While (Result<M.Size) And (Not (P[Result] In [#13,#10])) Do
Inc(Result);
Except
Result := 0;
End;
M.Free;
End;

There are other, endless ways to go about it, but don't use ReadLn with
TextFile, they just didn't scale properly from the dos days.
Back to top
Peter Morris [Droopy eyes
Guest
PostPosted: Thu Jun 08, 2006 8:12 am    Post subject: Re: Strings Reply with quote
Quote:
Begin
M := TMemoryStream.Create;
Try
M.LoadFromFile(FileName);
P := PChar(M.Memory);
Result := 0;
While (Result<M.Size) And (Not (P[Result] In [#13,#10])) Do
Inc(Result);
Except
Result := 0;
End;
M.Free;
End;

The [] would make that really slow. Best to read into a buffer in (x)MB
chunks and get the string out of it.
Back to top
Peter Morris [Droopy eyes
Guest
PostPosted: Fri Jun 09, 2006 1:14 pm    Post subject: Re: Strings Reply with quote
Quote:
The [] would make that really slow. Best to read into a buffer in (x)MB
chunks and get the string out of it.

Sorry, that was really to vague.

What I mean is this

1) Read ?MB
2) Use a PByte to point to the first byte
3) Increment PChar until PByte^ = #13
4) Set a string to the length of @PByte - @FirstChar
5) Use MOVE to copy the data from the data to the string

I'd probably use a circular buffer to save memory, there should be a few
around on the web somewhere. I think I have one somewhere too if you can't
find one.



--
Pete
====
Audio compression components, DIB graphics controls, ECO extensions,
FastStrings
http://www.droopyeyes.com

My blog
http://blogs.slcdug.org/petermorris/
Back to top
Display posts from previous:   
Post new topic   Reply to topic    BorlandTalk.com Forum Index -> Delphi Thirdparty Tools (General) All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2006 phpBB Group
SEO toolkit © 2004-2006 webmedic.