 |
BorlandTalk.com
Borland discussion newsgroups
|
| View previous topic :: View next topic |
| Author |
Message |
Gabriel Corneanu
Guest
|
 Posted: Mon Oct 03, 2005 12:22 pm Post subject: Object pooling |
 |
|
Hi all,
I read some time ago that a memory manager can't be much faster than FastMM.
And that an object (class) pooling mechanism might be a good way to improve
speed (Eric Grange).
I (almost) always used object pooling when it was really necessary. But I
was curios if a generic object pooling makes a difference, so I implemented
a simple (much simpler than I thought) pooling at TObject level.
I redirected NewInstance and FreeInstance (is there a way to modify directly
system.pas unit?) to my implementations. Here I store the destroyed objects
to a pool (an array of objects for each (size div 4)) and return them back
when new ones are created.
Surprisingly, it worked almost from the first try. However, we still need to
call InitInstance and CleanupInstance (or is it just needed the first/last
time??) and what we really save is just GetMem and FreeMem calls.
Does anyone think that this make sense? If this can go further, I will
gladly publish my fist implementation here.
Regards,
Gabriel
|
|
| Back to top |
|
 |
Martin James
Guest
|
| Posted: Mon Oct 03, 2005 1:21 pm Post subject: Re: Object pooling |
|
|
"Gabriel Corneanu" <gabrielcorneanu (AT) nospam (DOT) yahoo.com> wrote
| Quote: | Hi all,
I read some time ago that a memory manager can't be much faster than
FastMM.
And that an object (class) pooling mechanism might be a good way to
improve
speed (Eric Grange).
I (almost) always used object pooling when it was really necessary. But I
was curios if a generic object pooling makes a difference, so I
implemented
a simple (much simpler than I thought) pooling at TObject level.
I redirected NewInstance and FreeInstance (is there a way to modify
directly
system.pas unit?) to my implementations. Here I store the destroyed
objects
to a pool (an array of objects for each (size div 4)) and return them back
when new ones are created.
Surprisingly, it worked almost from the first try. However, we still need
to
call InitInstance and CleanupInstance (or is it just needed the first/last
time??) and what we really save is just GetMem and FreeMem calls.
Does anyone think that this make sense? If this can go further, I will
gladly publish my fist implementation here.
|
I have only used object pooling for objects that have a high turnover rate
and/or are very expensive to continually create/destroy, eg. buffers,
sockets. Many of these objects are bound to OS kernel objects or contain
other allocated resources that I want to re-use, and I certainly would not
want CleanupInstance /InitInstance run on them every time!
If I was to go down your path and pool all TObject descendants, there is the
problem of deciding on a good initial pool size and the occasional
maintenance of the pool. A pool that starts out empty will be intially
slow, and a non-maintained pool could eat up a lot of memory if some
infrequent operation creates and then frees a lot of a particular class.
Rgds,
Martin
|
|
| Back to top |
|
|
Eric Grange
Guest
|
| Posted: Mon Oct 03, 2005 1:42 pm Post subject: Re: Object pooling |
|
|
| Quote: | Does anyone think that this make sense? If this can go further, I will
gladly publish my fist implementation here.
|
Yes, it makes sense 
What your implementation achieved is only, as you said, a GetMem/FreeMem
dedicated to objects, so you're not saving as much as you could (and
maybe not always as much as using FastMM could).
What I meant was to pool ready-to-use objects, ie. not just the space
storage GetMem/FreeMem level, but also some InitInstance stuff.
InitInstance does 2 things:
- setup VMT and interface pointers
- reset the rest of the object's data to zero
When pooling, what you'll want to avoid/simplify is the setup part, so
you need to pool pre-setup objects, so that all you may have to do when
allocating a new object is the reset.
FinalizeInstance is a bit more complicated, as it basicly cleanups all
automatically-dynamically allocated fields (strings, dynamic arrays...).
This is something you can probably be more efficient about than the RTL
code, but it's something you can't really skip that isn't trivial.
Then comes a little bit of complexity: since you preserve setup, you
have to have a pool per object CLASS, rather than per object size, so
the mechanism that associates ClassType (Self in InitInstance) to its
pool has to be real fast.
So, a better place to start might be NewInstance, which is a virtual
class function... which could thus theoretically be somehow dynamically
implemented, redirected, and with a "hardcoded" pool reference in that
dynamic implementation.
Eric
|
|
| Back to top |
|
|
Eric Grange
Guest
|
| Posted: Mon Oct 03, 2005 1:53 pm Post subject: Re: Object pooling |
|
|
| Quote: | If I was to go down your path and pool all TObject descendants, there is the
problem of deciding on a good initial pool size and the occasional
maintenance of the pool. A pool that starts out empty will be intially
slow, and a non-maintained pool could eat up a lot of memory if some
infrequent operation creates and then frees a lot of a particular class.
|
Might be possible to avoid by:
a) running your app in 'profiling' mode, where classes that are
instanciated and freed 'a lot' are identified, then only these ones are
pooled (a bit like making pools manually, except it would happen
automagically).
b) not pooling whole objects: leave pooling to FastMM and concentrate on
caching/accelerating/simplifying XxxInstance methods.
....or a bit of both.
Classes without dynamically allocated fields (string etc.) or that
expose a lot of interfaces should be the ones to benefit most.
Of course that won't be able to compete in special situations like the
one you described in your first paragraph.
Eric
|
|
| Back to top |
|
|
Martin James
Guest
|
| Posted: Mon Oct 03, 2005 2:23 pm Post subject: Re: Object pooling |
|
|
| Quote: |
Might be possible to avoid by:
a) running your app in 'profiling' mode, where classes that are
instanciated and freed 'a lot' are identified, then only these ones are
pooled (a bit like making pools manually, except it would happen
automagically).
|
Yeah, maybee. I'm not sure whether there would be much advantage over a
'normal' high-prformance memory manager, which woule be needed anyway
because of the misery of long AnsiStrings.
| Quote: | b) not pooling whole objects: leave pooling to FastMM and concentrate on
caching/accelerating/simplifying XxxInstance methods.
|
Yes - I read your other post. I did not think of attaching the class VMT at
runtime and so reducing the object to a straightforward contiguous
allocation.
The more I think about this, the more it sounds like a 'general-purpose'
bucketing memory manager, but with 'intelligent' bucket sizes.
| Quote: | ...or a bit of both.
Classes without dynamically allocated fields (string etc.) or that
expose a lot of interfaces should be the ones to benefit most.
|
Yes - those strings :(
| Quote: | Of course that won't be able to compete in special situations like the
one you described in your first paragraph.
|
No if I could only get rid of the strings in these classes...<g>
Rgds,
Martin
|
|
| Back to top |
|
|
Eric Grange
Guest
|
| Posted: Mon Oct 03, 2005 3:12 pm Post subject: Re: Object pooling |
|
|
| Quote: | Yeah, maybee. I'm not sure whether there would be much advantage over a
'normal' high-prformance memory manager.
|
Cf. sampling profiler results posted sometime ago in the attachments:
when allocating/freeing TObject, only 15% of the time is spent in
FastMM, and about 75% in XxxInstance and related object
initialization/finalization overhead (when using Borland MM, the MM is
the #1 contention by far).
In other words, once the MM bottleneck is removed, you start hitting the
object initialization/finalization overhead in SysUtils. Yet, it isn't
as spectacular a bottleneck as the MM, nor as far reaching.
| Quote: | Yes - those strings 
|
All things could come together in the end: if you improve object
allocation/deallocation, you might as well give a look at string/array
alloc/dealloc, and there may be synergies between the two...
Quite a bit of complexity also goes into cleaning up interface
references, yet many objects don't have any of these, which means a few
useless hoops that could be avoided.
Another option would be to "compile" a finalizer per class, current RTL
implementation looks up RTTI information to know what has to be freed
automagically, since this is static per object, you could theoretically
compile a code (dynamically) dedicated to freeing TSomeObject ressources
(specifically that class), eliminating loops and branchings in the process.
Eric
|
|
| Back to top |
|
|
Thaddy
Guest
|
| Posted: Mon Oct 03, 2005 4:37 pm Post subject: Re: Object pooling |
|
|
Eric Grange wrote:
| Quote: | If I was to go down your path and pool all TObject descendants, there
is the
problem of deciding on a good initial pool size and the occasional
maintenance of the pool. A pool that starts out empty will be intially
slow, and a non-maintained pool could eat up a lot of memory if some
infrequent operation creates and then frees a lot of a particular class.
|
__________
{*****************************************************************************}
{
}
{ Objects on the Stack
}
{
}
{ Copyright (c) 2001 Robert R. Marsh, S.J.
}
{ & the British Province of the Society of Jesus
}
{
}
{
}
{ If you like this code and find yourself using it then please
}
{ consider making a donation to your favorite charity. I would
}
{ also be pleased if you would acknowledge me in any projects
}
{ that make use of them.
}
{
}
{ This unit is supplied as is. The author disclaims all
warranties, }
{ expressed or implied, including, without limitation, the
warranties }
{ of merchantability and of fitness for any purpose.
}
{
}
{ The author assumes no liability for damages, direct or
}
{ consequential, which may result from use of this code.
}
{
}
{ [email]stack (AT) rmarsh (DOT) com[/email]
}
{ http://www.rmarsh.com
}
{
}
{*****************************************************************************}
(*
Very fast object-allocation on the stack
*)
unit StackObjects;
interface
uses
sysutils;
type
EStackError = type Exception;
type
TStackStatus = record
Capacity: Cardinal; // total size of reserved space
Available: Cardinal; // amount of space still available
end;
function StackStatus: TStackStatus;
// Initialize the stack memory manager for the current
// procedure. <size> bytes will be available in all.
// Exceeding this limit will raise EStackError.
procedure UseStack(Size: Cardinal);
// Restore the default memory manager.
procedure UnUseStack;
// ObjectsOnly is true by default and only TObject
// descendants are allocated on the stack. Other
// allocations (e.g., strings, memory, dynamic arrays)
// remain on the heap. If ObjectsOnly is set to false
// ALL allocations are made on the stack and you
// should be VERY careful to make sure reference-counted
// variables like strings are finalized before the
// standard memory manager is reinstated.
var
ObjectsOnly: Boolean = True;
implementation
var
OldMM: TMemoryManager;
IsNewMM: Boolean = False;
var
// global pointer to the reserved stack space
StackMemory: Pointer = nil;
// offset from the start of StackMemory
StackOffset: Cardinal = 0;
// size of StackMemory block
StackLimit: Cardinal = 0;
// Returns a pointer into an unused portion of the reserved stack,
// effectively allocating memory.
function StackGetMem(Size: Integer): Pointer;
var
Caller: Cardinal;
begin
// get the address of the calling routine
asm
mov eax,[ebp+8]
mov Caller,eax
end;
// if the caller is not in TObject we use the heap
if ObjectsOnly and ((Caller > Cardinal(@TObject.InstanceSize)) or
(Caller < Cardinal(@TObject.ClassType))) then
begin
Result := OldMM.GetMem(Size);
end
else
begin
// return a pointer into the reserved stack
Result := Pointer(Cardinal(StackMemory) + StackOffset);
Inc(StackOffset, Size);
if StackOffset > StackLimit then
begin
UnUseStack;
raise EStackError.Create('stack allocation exceeded');
end;
end;
end;
// Deallocates previously allocated memory. In fact, stack
// memory is not freed since it automatically disappears
// when pointer goes out of scope.
function StackFreeMem(P: Pointer): Integer;
var
Caller: Cardinal;
begin
// get the address of the calling routine
asm
mov eax,[ebp+8]
mov Caller,eax
end;
// if the caller is not in TObject we use the heap
if ObjectsOnly and ((Caller > Cardinal(@TObject.InstanceSize)) or
(Caller < Cardinal(@TObject.ClassType))) then
begin
Result := OldMM.FreeMem(P);
end
else
begin
// do nothing except report success
Result := 0;
end;
end;
// Reallocation is achieved by returning a fresh pointer.
function StackReallocMem(P: Pointer; Size: Integer): Pointer;
var
Caller: Cardinal;
begin
// get the address of the calling routine
asm
mov eax,[ebp+12] // note 12 and not 8 since we have two parameters
mov Caller,eax
end;
// if the caller is not in TObject we use the heap
if ObjectsOnly and ((Caller > Cardinal(@TObject.InstanceSize)) or
(Caller < Cardinal(@TObject.ClassType))) then
begin
Result := OldMM.ReallocMem(P, Size);
end
else
begin
// get a fresh pointer
Result := StackGetMem(Size);
// and copy the contents of old to new ...
// the size parameter is wrong but Realloc
// only guarantees the old contents anyway
Move(P^, Result^, Size);
end;
end;
var
NewMM: TMemoryManager = (GetMem: StackGetMem; FreeMem: StackFreeMem;
ReallocMem: StackReallocMem);
// just swap the memory managers
procedure SetStackMemory;
begin
if not IsNewMM then
begin
GetMemoryManager(OldMM);
SetMemoryManager(NewMM);
IsNewMM := True;
StackOffset := 0;
end;
end;
// based on StackAlloc in grids.pas -- thank you Borland!
procedure UseStack(Size: Cardinal); register;
asm
POP ECX // return address
MOV EDX, ESP
ADD EAX, 3
AND EAX, not 3 // round up to keep ESP dword aligned
MOV StackLimit, EAX // remember the size
CMP EAX, 4092
JLE @@2
@@1:
SUB ESP, 4092
PUSH EAX // make sure we touch guard page, to grow stack
SUB EAX, 4096
JNS @@1
ADD EAX, 4096
@@2:
SUB ESP, EAX
MOV EAX, ESP // function result = low memory address of block
MOV StackMemory, EAX // set the global pointer
PUSH EDX // save original SP, for cleanup
MOV EDX, ESP
SUB EDX, 4
PUSH EDX // save current SP, for sanity check (sp = [sp])
PUSH ECX // return to caller
CALL SetStackMemory // swap the managers
end;
procedure UnUseStack;
begin
if IsNewMM then
begin
SetMemoryManager(OldMM);
IsNewMM := False;
end;
end;
function StackStatus: TStackStatus;
begin
Result.Capacity := StackLimit;
Result.Available := StackLimit - StackOffset;
end;
end.
|
|
| Back to top |
|
|
Thaddy
Guest
|
| Posted: Mon Oct 03, 2005 4:41 pm Post subject: Re: Object pooling |
|
|
Just to Honour Robert Marsh:
His code works and is much faster in allocating and de-allocating,
especially small, objects.
Also look at the code in grods.pas (look for stackalloc), that contains
rather nifty code that does about the same (although not coupled with a
memory manager).
Regards,
Thaddy
|
|
| Back to top |
|
|
Gabriel Corneanu
Guest
|
| Posted: Mon Oct 03, 2005 7:15 pm Post subject: Re: Object pooling |
|
|
"Eric Grange" <egrangeNO (AT) SPAMglscene (DOT) org> wrote
| Quote: | Does anyone think that this make sense? If this can go further, I will
gladly publish my fist implementation here.
Then comes a little bit of complexity: since you preserve setup, you have
to have a pool per object CLASS, rather than per object size, so the
mechanism that associates ClassType (Self in InitInstance) to its pool has
to be real fast.
So, a better place to start might be NewInstance, which is a virtual
class function... which could thus theoretically be somehow dynamically
implemented, redirected, and with a "hardcoded" pool reference in that
dynamic implementation.
Eric
|
Hi,
I have now implemented a per class pooling.
It seems to work very well for a normal object (just create/free a TButton
in a loop), but for TInterfacedObject (as expected, because extra logic in
AfterConstruction...).
Maybe flags can be passed to the pool class to specify which steps (like
InitInstance, CleanupInstance) can be skipped.
I would appreciate comments about it.
BTW, it's posted to the attachments group.
Regards,
Gabriel
|
|
| Back to top |
|
|
Gabriel Corneanu
Guest
|
| Posted: Tue Oct 04, 2005 5:42 am Post subject: Re: Object pooling |
|
|
There is a problem.
Because my redirected functions have to use asm, they are not properly
optimized and therefore slower.
I have to write asm for the whole function.
Gabriel
|
|
| Back to top |
|
|
Atmapuri
Guest
|
| Posted: Tue Oct 04, 2005 12:10 pm Post subject: Re: Object pooling |
|
|
Hi!
| Quote: | Another option would be to "compile" a finalizer per class, current RTL
implementation looks up RTTI information to know what has to be freed
automagically, since this is static per object, you could theoretically
compile a code (dynamically) dedicated to freeing TSomeObject ressources
(specifically that class), eliminating loops and branchings in the
process.
|
Auto-magic wont do here. The cost of the "clean up" of a pooled object
is directly added to the cost of its "set up". The clean up must be
PoolDestroy method which reinitializes the properties/fields changed back to
their default values and only the user knows what those are.
This is different from Destroy, which simply cleans up the memory
contained.
Regards!
Atmapuri.
|
|
| Back to top |
|
|
adem
Guest
|
| Posted: Tue Oct 04, 2005 3:17 pm Post subject: Re: Object pooling |
|
|
Gabriel,
| Quote: | Does anyone think that this make sense? If this can go further,
I will gladly publish my fist implementation here.
|
I have tried the code you posted (in the attachments group) on
new version of TVirtualTree (by MikeL) that I am working on (on
and off). My version uses TVirtualNode as TObject as opposed to
the original that used record structure.
All this paragraph is to tell you that the new VT looks like a
good candidate to test out your object pool idea. VT histroically
prides itself in creatingt large sums of nodes instantly.
The new version, however, turned out to be twice as slow.
That is until I tried it with your Object Pool idea. VT is
basically as fast as it was before and I like it a lot.
There is however one catch: I get an 'inalid pointer' exception
in TClassPool.Destroy.
I am almost 100 % certain that it is not from my code --I have
checked everything (in the past and now) with AQTime, MemCheck
and FastMM4: no leaks.
Could it be due to something in your code or some incompatibility
that I should be aware of.
I can send post my sources in attachments or you can get a copy
from
nntp://news.mustangpeak.net/delphi-gems.support.attachments/312
In any case, I think Object Pooling is a great idea.
Cheers,
Adem
|
|
| Back to top |
|
|
Eric Grange
Guest
|
| Posted: Tue Oct 04, 2005 3:20 pm Post subject: Re: Object pooling |
|
|
| Quote: | PoolDestroy method which reinitializes the properties/fields changed back to
their default values and only the user knows what those are.
|
It wouldn't be safe to pool after the constructor has occured, and
before it has occured, fields are set zero. So if you pool objects
after InitInstance but before Constructor, what you have to do is only
cleanup everything back to zero when returning the object to the pool.
| Quote: | This is different from Destroy, which simply cleans up the memory
contained.
|
I'm not sure to follow you there.
Destroy is user code that cleans up user-managed objects, handles and
references that where instantatiated during the life of the object
(after Constructor), but destroy doesn't free or cleanup anything more,
it passes execution to FinalizeInstance, which cleans up strings,
dynamic arrays, interfaces refs, etc. and that's after that step that
you could return your object to a pool, not before.
However, that finalization sequence is executed a fixed-per-class
cleanup sequence, that is dependant only from the class.
To pool an object after constructor/before destructor requires knowledge
of what that object does, knowledge which only the developper has, so
you can't really provide generic RTL pooling at that level.
Eric
|
|
| Back to top |
|
|
Eric Grange
Guest
|
| Posted: Tue Oct 04, 2005 4:04 pm Post subject: Re: Object pooling |
|
|
Maybe instead of reimplementing InitInstance (which is complex),
it would be possible just invoke it once per class, copy the result
to some buffer, then reuse that for subsequent fastInitInstance?
So per-class, in addition to the pool, we have a template of size
InstanceSize that is copied over to initialize a new instance
(the System.TObject.InitInstance is thus executed only once
for all pooled classes).
Ideally, we don't need to have the whole InstanceSize in the template,
only the object header, the rest can be set to zero as usual.
....additionnally, this mechanism could be made to work for all classes,
pooled or not, with limited memory overhead... I wonder if the per-class
pool would be that necessary afterward, relying of FastMM's own pools
might be enough, it's not like the object header represent a huge amount
of data to copy from their template.
Eric
|
|
| Back to top |
|
|
Eric Grange
Guest
|
| Posted: Tue Oct 04, 2005 4:27 pm Post subject: Re: Object pooling |
|
|
Templating the instance yelds speedups even with a slow move, some extra
hoops and deactivated pooling (I hacked your unit :p) so I'm hopeful
there are gains to be made. Got TInterfacedObject working after some
After/Before virtual methods tinkering, it should be possible to detect
those situations cleanly however, so I'm not worried.
Since there are gains even without pooling, I'll try finding time to
look into that direction alone, pooling can always come back in later.
Eric
|
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
