BorlandTalk.com

Osmosis · Guest

I found the following function on a newsgroup and it works pretty good.
The only disadvantage is that it's pretty slow for larger files.
Are there any thoughts on how to improve the speed of this procedure ?

procedure HexDisplay(Filename: string; mmodump: trichedit);
var
byteFile: file of Byte;
loop: Integer;
sizeInBytes: Integer;
line: string;
line2: string;
data: byte;
begin
mmoDump.lines.clear;

assignFile(byteFile, FileName);
reset(byteFile);
sizeInBytes := fileSize(byteFile);
loop := 0;

if (sizeInBytes = 0) then
mmoDump.lines.add('File is zero size');

while (loop < sizeInBytes) do
begin
if (loop mod 16 = 0) then
begin
if trim(line) <> '' then
mmoDump.lines.Add(line + ' : ' + line2);
line := intToHex(loop, Cool

+ ' : ';
line2 := '';
end;
read(byteFile, data);

line := line + intToHex(data, 2) + ' ';

if (data in [32..126, 128..255]) then
line2 := line2 + chr(data)
else
line2 := line2 + '.';

inc(loop);
end;

if trim(line) <> '' then
begin
mmoDump.lines.add(line + ' : ' + line2);
end;

closeFile(byteFile);
end;

The result should be something like this:
000114B0 : FC BE FD BD FE BC FF BB 01 BC BB FA BA FD B9 FE :
ü¾ý½þ¼ÿ».¼»úºý¹þ
000114C0 : B8 FB B7 03 B6 B7 B6 B6 FD B5 01 B4 B5 FC B4 FD :
¸û·.¶·¶¶ýµ.´µü´ý
000114D0 : B3 01 B2 B3 FD B2 00 B1 FE B2 FE B1 01 B0 B1 FD :
³.²³ý².±þ²þ±.°±ý
000114E0 : B0 01 AF B0 FC AF FE AE FF AF FE AE FE AD 00 AE : °.¯°ü¯þ®ÿ¯þ®þ.®
000114F0 : EC AD AD AD A8 E6 FE E5 00 E6 FC E5 F8 E4 02 E3 : ì¨æþå.æüåøä.ã
00011500 : E4 E4 FE E3 FB E2 01 E1 E2 FE E1 FD E0 FF E0 FF :
ääþãûâ.áâþáýàÿàÿ
00011510 : DF 02 E0 DF DF FE DE 02 DF DE DE FB DD FD DC FD :
ß.àßßþÞ.ßÞÞûÝýÜý
00011520 : DB FD DA 02 D9 DA DA FE D9 FE D8 FF D7 00 D8 FE :
ÛýÚ.ÙÚÚþÙþØÿ×.Øþ
00011530 : D7 FD D6 05 D5 D6 D5 D5 D4 D5 FE D4 FC D3 FE D2 :
×ýÖ.ÕÖÕÕÔÕþÔüÓþÒTh

Thanks in advance,
Peter

Jeremy Collins · Guest

Osmosis wrote:

Duncan McNiven · Guest

On Sun, 15 Feb 2004 10:55:57 GMT, "Osmosis" <spam.nebel (AT) hotmail (DOT) com> wrote:

Osmosis · Guest

Thanks for the input.

I need a richedit, because I want to highlight some part of the text
eventually (in RTF).
I want to use it for a search program that can handle any file (any
filesize).

AlanGLLoyd · Guest

In article <3wMXb.2264$al6.226401 (AT) phobos (DOT) telenet-ops.be>, "Osmosis"
<spam.nebel (AT) hotmail (DOT) com> writes:

Osmosis · Guest

Sounds like an interesting idea.
But easy wouldn't exactly be my choice of words ;-)

Dragon Lord · Guest

If you want real improvement though, do this. If you CAN, load all the file
at once, this allows you to read it, keep a copy in memory, and close it so
other files can resume processing on it if needed. If not, trying loading in
16K chunks, as this is good place for a block align with the hard drive.
Most hard drives have a 4K block size in 512 byte chunks but some have 2, 8,
16 and sometimes 32. This will also result less WinAPI calls. I did notice
that the original code was using Read. Back in my old TP days, which is
what delphi is based off of, I used the old blockread command, but I still
favor the FileStream method. The only problem with this code, is I would
change the fmShareExclusive, to fmShareDenyWrite. This allows other files
to read but not write, in case another program needs to read from it while
your using it.

Jeremy

a quick write up that will take any "stream".

procedure BinarytoMemo(Memo1: TMemo; Stream: TStream);
var
X: Array of Byte;
S: String;
I: Integer;
Count2: Integer;
TmpByte: Byte;
begin
SetLength(X, Stream.Size);
Stream.Position := 0;
Stream.Read(X[0], Length(X));
Memo1.Lines.Clear;
Memo1.Lines.BeginUpdate;
I := 0;
while( I < Length(X)) do
begin
S := IntToHex(I, Cool

+' : ';
For Count2 := 0 to 15 do
if (I + Count2 < Length(X)) then
S := S + IntToHex(X[Count2 + I],2)
else
S := S + ' ';
S := S + ' ';
For Count2 := 0 to 15 do
begin
if (I + Count2 < Length(X)) then
begin
TmpByte := X[Count2 + I];
case TmpByte of
$20..$7E:
S := S + Chr(TmpByte);
else
S := S + '.';
end;
end
else
S := S + ' ';
end;
Memo1.Lines.Add(S);
Inc(I,$10);
end;
Memo1.Lines.EndUpdate;
end;

You could easily change TMemo to a TRichEdit, or even provide an override
for a TRichEdit.

interface

procedure BinaryToHex(Memo1: TMemo; Stream: TStream); override;
procedure BinaryToHex(Memo1: TRichEdit; Stream: TStream); override;

implementation

procedure BinarytoMemo(Lines: TStrings; Stream: TStream);
var
X: Array of Byte;
S: String;
I: Integer;
Count2: Integer;
TmpByte: Byte;
begin
SetLength(X, Stream.Size);
Stream.Position := 0;
Stream.Read(X[0], Length(X));
Lines.Clear;
Lines.BeginUpdate;
I := 0;
while( I < Length(X)) do
begin
S := IntToHex(I, Cool

+' : ';
For Count2 := 0 to 15 do
if (I + Count2 < Length(X)) then
S := S + IntToHex(X[Count2 + I],2)
else
S := S + ' ';
S := S + ' ';
For Count2 := 0 to 15 do
begin
if (I + Count2 < Length(X)) then
begin
TmpByte := X[Count2 + I];
case TmpByte of
$20..$7E:
S := S + Chr(TmpByte);
else
S := S + '.';
end;
end
else
S := S + ' ';
end;
Lines.Add(S);
Inc(I,$10);
end;
Lines.EndUpdate;
end;

procedure BinaryToHex(Memo1: TMemo; Stream: TStream);
begin
BinaryToHex(Memo1.Lines, Stream);
end;

procedure BinaryToHex(Memo1: TRichEdit; Stream: TStream);
begin
BinaryToHex(Memo1.Lines, Stream);
end;

Although, I personally use this for debuging purposes, not as a full blown
application. This works even speedy for 7 meg executables.

Jeremy

"Duncan McNiven"

Duncan McNiven · Guest

On Sun, 15 Feb 2004 15:47:11 GMT, "Osmosis" <spam.nebel (AT) hotmail (DOT) com> wrote:

Duncan McNiven · Guest

On Sun, 15 Feb 2004 12:59:35 -0500, "Dragon Lord" <invalid (AT) joke (DOT) com> wrote:

Dragon Lord · Guest

Yeah, that code would be slower, but 2 small code changes would make it
faster than yours, also good to know. Assigning to Memo.Text is SLOWER than
Assigning to RichEdit.Text, don't ask me why. Loading an 8 meg text file
into a stringlist, then assigning to Text, Memo takes 17 seconds, RichEdit,
4 seconds. I'm not sure what the underlying code is doing, as they pass
everything via messages. I could find out though, if anyone is interested.
On that note, taking out one If that gets recursively called and changing
the parameter of TStrings, from a TMemoStrings, to a TStringList, changes
the running code from 50 seconds to 5.2 seconds for a 2.8 megs file on my
machine, your code runs at 7.9 seconds, at least on my machine. 40 seconds
of that time was spent in TMemoStrings.Add

I rewrote my procedure completely from scratch this time and just for the
sake of argument, i rewrote my new procedure to either read all(BinaryToHex)
or read in 16 byte chunks(BinaryToHex2). Here is the code.

procedure HexDisplay(Filename: string; mmodump : TRichEdit); // Note Changed
this to TMemo when needed
const
MAX_LINE_LENGTH = 16;
var
Src : TFileStream;
OneSrcLine : array [1 .. MAX_LINE_LENGTH] of Byte;
OneHexLine : String;
ASCII_Line : String;
ThisLineLength : Integer;
iChr : Integer;
iOffset : Integer;
tmpOutput : TStrings;
begin
iOffset := 0;
mmodump.Clear;
tmpOutput := TStringList.Create;
try
Src := TFileStream.Create(FileName,fmOpenRead OR fmShareExclusive);
try
if (Src.Size = 0) then
mmoDump.lines.add('File is zero size')
else
begin
while Src.Position < Src.Size do
begin
ThisLineLength := Src.Read(OneSrcLine,MAX_LINE_LENGTH);
OneHexLine := '';
ASCII_Line := '';
for iChr := 1 to ThisLineLength do
begin
OneHexLine := OneHexLine + IntToHex(OneSrcLine[iChr],2) + ' ';
if (OneSrcLine[iChr] in [32..126, 128..255]) then
ASCII_Line := ASCII_Line + chr(OneSrcLine[iChr])
else
ASCII_Line := ASCII_Line + '.';
end;
tmpOutput.Add(intToHex(iOffset, Cool

+ ' : ' + OneHexLine + ' : ' +
ASCII_Line);
inc(iOffset,ThisLineLength);
end;
end;
finally
Src.Free;
end;
mmodump.Text := tmpOutput.Text;
finally
tmpOutput.Free;
end;
end;

procedure BinarytoMemo(Lines: TStrings; Stream: TStream);
var
X: String;
S,T: String;
I: Integer;
J: Integer;
Count2: Integer;
begin
SetByteArrayLength(X, Stream.Size);
Stream.Position := 0;
Stream.Read(X[1], Length(X));
Lines.Clear;
Lines.BeginUpdate;
I := 1;
J := Length(X) - $10;
SetLength(T,62);
FillChar(T[1],62,$20);
T[10] := ':';
T[46] := ':';
while( I < J) do
begin
S := IntToHex(I-1, Cool

;
Move(S[1],T[1], Cool

;
For Count2 := 0 to 15 do
begin
S := IntToHex(Ord(X[I+Count2]),2);
Move(S[1],T[(Count2*2)+13],2);
if X[I+Count2] in [' '..'~'] then
T[Count2+48] := X[I+Count2]
else
T[Count2+48] := '.';
end;
Inc(I,$10);
Lines.Add(T);
end;
FillChar(T[13],$20,$20);
FillChar(T[48],$10,$20);
S := IntToHex(I-1, Cool

;
Move(S[1],T[1], Cool

;
For Count2 := I to Length(X)-1 do
begin
S := IntToHex(Ord(X[Count2]),2);
Move(S[1],T[((Count2-I)*2)+13],2);
if Ord(X[Count2]) in [$20..$7E] then
T[Count2-I+48] := X[Count2]
else
T[Count2-I+48] := '.';
end;
Lines.Add(T);
Lines.EndUpdate;
end;

procedure BinarytoMemo2(Lines: TStrings; Stream: TStream);
var
X: Array[0..15] of Char;
S,T: String;
I, R: Integer;
Count2: Integer;
begin
SetByteArrayLength(X, Stream.Size);
Stream.Position := 0;
Lines.Clear;
Lines.BeginUpdate;
SetLength(T,62);
FillChar(T[1],62,$20);
T[10] := ':';
T[46] := ':';
I := 0;
while( Stream.Position < Stream.Size ) do
begin
R := Stream.Read(X[0],$10);
S := IntToHex(I, Cool

;
Move(S[1],T[1], Cool

;
For Count2 := 0 to R-1 do
begin
S := IntToHex(Ord(X[Count2]),2);
Move(S[1],T[(Count2*2)+13],2);
if X[Count2] in [' '..'~'] then
T[Count2+48] := X[Count2]
else
T[Count2+48] := '.';
end;
S := ' ';
for Count2 := R to 15 do
begin
Move(S[1],T[(Count2*2)+13],2);
T[Count2+48] := ' ';
end;
Inc(I,$10);
Lines.Add(T);
end;
Lines.EndUpdate;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
X, Y: Int64;
MyFile: TFileStream;
MyStrings: TStringList;
begin
if OpenDialog1.Execute then
begin
ListBox1.Items.Clear;
X := GetClockTicks;
HexDisplay(OpenDialog1.FileName, RichEdit1);
Y := GetClockTicks;
ListBox1.Items.Add(IntToStr(Y-X));

X := GetClockTicks;
MyFile := TFileStream.Create(OpenDialog1.FileName,fmOpenRead or
fmShareDenyWrite);
MyStrings := TStringList.Create;
BinaryToMemo(MyStrings, MyFile);
RichEdit1.Text := MyStrings.Text;
MyFile.Free;
Y := GetClockTicks;
ListBox1.Items.Add(IntToStr(Y-X));

X := GetClockTicks;
MyFile := TFileStream.Create(OpenDialog1.FileName,fmOpenRead or
fmShareDenyWrite);
MyStrings := TStringList.Create;
BinaryToMemo2(MyStrings, MyFile);
RichEdit1.Text := MyStrings.Text;
MyFile.Free;
Y := GetClockTicks;
ListBox1.Items.Add(IntToStr(Y-X));

end;
end;

And here are the results

File tested - "Blue Gender - Ending2.avi" Size: 8,179,712 bytes

RichEdit With no word wrap and fixed width font

Yours Mine Mine2
Run1 22953ms 8702ms 9704ms
Run2 after1 22332ms 8672ms 9794ms
Run3 fresh 23093ms 8762ms 9287ms

Memo with no word wrap and fixed width font

Yours Mine Mine2
Run1 53346ms 25557ms 27800ms
Run2 after1 52115ms 25547ms 28851ms
Run3 fresh 52536ms 25116ms 29082ms

Memo with word wrap and variable width font

Yours Mine Mine2
Run1 87095ms 64012ms 78553ms
Run2 after1 88187ms 63611ms 78042ms
Run3 fresh 106283ms 64062ms 78793ms

RichEdit with word wrap and variable width font

Yours Mine Mine2
Run1 20300ms 8552ms 9734ms
Run2 after1 20589ms 8713ms 10064ms
Run3 fresh 20349ms 8612ms 8923ms

Run1 = Fresh start
Run2 = run right after run1 without exiting
Run3 = Fresh Start after Run2

As you can see, there is a difference in the components you use, AS WELL as
if you set the component to no wrap with fixed width font. I'm guessing
that the Memo component does formating on line change, and RichEdit does
formating on Painting. Also, you can see a different in Mine(Reading all)
and Mine2(reading in 16 byte chunks). I'm sure you would also see an
improvement doing 4K, 8K, or even 16K chunks at a time. I could also make
it faster by replacing the IntToHex with a ByteToHex, which would be much
faster

Jeremy

----- Original Message -----
From: "Duncan McNiven" Newsgroups: comp.lang.pascal.delphi.misc
Sent: Sunday, February 15, 2004 3:30 PM
Subject: Re: File Hex View : how to improve this procedure ?

Osmosis · Guest

I'm already using a custom scanfile function to find files that contain a
certain text.
This function uses blockread and it's blazing fast.

The display of the text needs to be almost instantaneous, so the only way is
to use a virtual stringgrid, as Alan suggested.
But the trichedit needs to be aware of length of the text because the
scrollbars need to be displayed correctly.

The purpose of my program (this part of the program) is to be a generic file
viewer. Binary files need to be displayed in HEX values and ASCII files
need to be displayed as plain text.

I still have to figure out how to implement this virtual stringgrid stuff,
but any ideas are welcome.

Dragon Lord · Guest

"Osmosis" <spam.nebel (AT) hotmail (DOT) com> wrote

Osmosis · Guest

Well, it would be nice if the user could select a part of the file and copy
that to the clipboard.
The part that can be selected could be bigger than the number of screen
lines, of course.

It looks to me like there has to be some kind of text buffer, that can grow
dynamically depending on the amount of selected text.

Dragon Lord · Guest

wouldn't the same be true of RichEdit though?

Jeremy

"Osmosis" <spam.nebel (AT) hotmail (DOT) com> wrote

Osmosis · Guest

What I mean is that in the method you're proposing, the richedit would only
hold the number of characters that are on the screen,
even if you scroll up or down. But if you want to select a larger part of
text, this needs to handled somewhere else, in another memory location.

Or am I seeing this wrong ?

"Dragon Lord" <invalid (AT) joke (DOT) com> wrote