BorlandTalk.com Borland discussion newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  Performance of Bold with different Delphi versions - D2006 a          BorlandTalk.com Forum Index -> Delphi Model Driven Architecture (General) View previous topic :: View next topic   Author Message Rolf Lampa [RIL] Guest Posted: Thu Aug 31, 2006 9:10 pm    Post subject: Performance of Bold with different Delphi versions - D2006 a How... can it be? I run a (very) simple test by creating 283873 Bold objects and stored them in Interbase (default, from original D7A install). All tests started from "scratch" (clean empty database). I'm (fairly) sure of that I had the same compiler settings in all three cases below (optimizations ON, assertions, etc). I got the following results (from the my inbuilt log) for threee different versions of D: D2005/SP2,U3: 283873 objects, 08:39 sec, 0.00183 sec/obj D2006/SP2,--: 283873 objects, 11:23 sec, 0.00241 sec/obj D7A__/SP2,--: 283873 objects, 11:56 sec, 0.00252 sec/obj 2000 objects/batch was "imported" (AND unloaded) sequentially (to keep memory usage low). BoldObject Locators remained in memory though. Wasn't D2006 supposed to be fast on W32 apps? Test running on the same database (Interbase from the D7A install). Bold Memory Manager switched with {$DEFINE BOLD_DISABLEMEMORYMANAGER} in project file. OK, this was a very limited test as it only created some objects, inserted strings into tree string fields + one big text chunk into a BoldBlob field, but anyway. Hm, what is it that makes D2006 to perform this "modest"? Regards, // Rolf Lampa Back to top Jan Nordén (Borland) Guest Posted: Fri Sep 01, 2006 8:11 am    Post subject: Re: Performance of Bold with different Delphi versions - D20 Rolf Lampa [RIL] wrote: Quote: How... can it be? I run a (very) simple test by creating 283873 Bold objects and stored them in Interbase (default, from original D7A install). All tests started from "scratch" (clean empty database). I'm (fairly) sure of that I had the same compiler settings in all three cases below (optimizations ON, assertions, etc). I got the following results (from the my inbuilt log) for threee different versions of D: D2005/SP2,U3: 283873 objects, 08:39 sec, 0.00183 sec/obj D2006/SP2,--: 283873 objects, 11:23 sec, 0.00241 sec/obj D7A__/SP2,--: 283873 objects, 11:56 sec, 0.00252 sec/obj 2000 objects/batch was "imported" (AND unloaded) sequentially (to keep memory usage low). BoldObject Locators remained in memory though. Wasn't D2006 supposed to be fast on W32 apps? Test running on the same database (Interbase from the D7A install). Bold Memory Manager switched with {$DEFINE BOLD_DISABLEMEMORYMANAGER} in project file. OK, this was a very limited test as it only created some objects, inserted strings into tree string fields + one big text chunk into a BoldBlob field, but anyway. Hm, what is it that makes D2006 to perform this "modest"? Regards, // Rolf Lampa You didn't by any chance compile the D2006 version of ECO with debugging and/or assertions enabled? -- Jan Nordén Architect, ECO Team Back to top Rolf Lampa [RIL] Guest Posted: Fri Sep 01, 2006 5:13 pm    Post subject: Re: Performance of Bold with different Delphi versions - D20 Jan Nordén (Borland) wrote: Quote: Rolf Lampa [RIL] wrote: How... can it be? ... D2005/SP2,U3: 283873 objects, 08:39 sec, 0.00183 sec/obj D2006/SP2,--: 283873 objects, 11:23 sec, 0.00241 sec/obj D7A__/SP2,--: 283873 objects, 11:56 sec, 0.00252 sec/obj 2000 objects/batch was "imported" (AND unloaded) sequentially (to keep memory usage low). BoldObject Locators remained in memory though. ... Hm, what is it that makes D2006 to perform this "modest"? You didn't by any chance compile the D2006 version of ECO with debugging and/or assertions enabled? Yes, they all had these Debugging options set like this: Debug information = ON, Local symbols = ON, Reference info = ON, Definitions only = ON Assertions = ON. Use debug DCUs = OFF Regards, // Rolf Lampa Back to top Rolf Lampa [RIL] Guest Posted: Fri Sep 01, 2006 5:22 pm    Post subject: Re: Performance of Bold with different Delphi versions - D20 Rolf Lampa [RIL] wrote: Quote: Jan Nordén (Borland) wrote: You didn't by any chance compile the D2006 version of ECO with debugging and/or assertions enabled? Yes, <... I also ran the tests from the IDE, and THAT may not have been so smart since the IDE's may have different impact on the execution times... ? I'll try again, also with all the debugging stuff = OFF, and running only the compiled exe file. // Rolf Lampa Back to top Peter Morris [Droopy eyes Guest Posted: Fri Sep 01, 2006 5:46 pm    Post subject: Re: Performance of Bold with different Delphi versions - D20 But did you build Bold from the source in D7 and D2005 too? Try building them all with asserts + debug info off and recomparing. Back to top Rolf Lampa [RIL] Guest Posted: Fri Sep 01, 2006 6:58 pm    Post subject: Re: Performance of Bold with different Delphi versions - D20 Peter Morris [Droopy eyes software] wrote: Quote: But did you build Bold from the source in D7 and D2005 too? No, I built separately for each D with its own distro of Bold. Quote: Try building them all with asserts + debug info off and recomparing. Yup, new numbers are on its way, just wait a minute (or two). BTW, is the D2006 memory manager mainly dealing with optimizing string handling? If so then I know why it doesn't perform better; I use HyperStrings AND banned the usage of System.Delete(). (Which made my app run 10x + faster. WHY THIS TEST In this run I create some 280.000 objects. But I aim to create some 3.4 million objects from another import source, but seemingly it will take a weekend for D2006 to accomplish that. Well, that's my problem, I will perform the big import at least two or three times a week (automated task), which is what the app is made for. A can sense problems ahead though... Regards, // Rolf Lampa Back to top Rolf Lampa [RIL] Guest Posted: Fri Sep 01, 2006 7:28 pm    Post subject: Re: Performance of Bold with different Delphi versions - D20 Rolf Lampa [RIL] wrote: Quote: Jan Nordén (Borland) wrote: Rolf Lampa [RIL] wrote: How... can it be? ... D2006/SP2,--: 283873 objects, 11:23 sec, 0.00241 sec/obj D2005/SP2,U3: 283873 objects, 08:39 sec, 0.00183 sec/obj D7A__/SP2,--: 283873 objects, 11:56 sec, 0.00252 sec/obj 2000 objects/batch was "imported" (AND unloaded) sequentially (to keep memory usage low). BoldObject Locators remained in memory though. ... Hm, what is it that makes D2006 to perform this "modest"? You didn't by any chance compile the D2006 version of ECO with debugging and/or assertions enabled? I built all the test executables with the Bold sources for each D version respectively. [Test round 2] Same settings as above except for no IDE. Results: D2006/SP2,--: 283873 objects, 11:26 sec, 0.00242 sec/obj (diff: 00:03, 0.00001 from the above) D2005/SP2,U3: (skipped) D7A__/SP2,--: (skipped) [Test round 3] No IDE, and NO debug info (all grouped debug options set to OFF). Results: D2006/SP2,--: 283873 objects, 12:31 sec, 0.00265 sec/obj (diff 01:08, 0.00024) D2005/SP2,U3: 283873 objects, 12:19 sec, 0.00260 sec/obj (diff 03:40, 0.00142) D7A__/SP2,--: 283873 objects, 12:11 sec, 0.00258 sec/obj (diff 00:15, 0.00006) For some reason the test doesnt run any faster with debug OFF. It could be due to environmental differences between the tests (like memory defragmentation etc?). I also tested to run the test with D2005 from the IDE (debug off), just to compare in the same "environment", but I get essentially identical numbers, or, 10 sec LESS from the IDE...). No hotfixes installed for D2006. I also noticed that the estimated avg execution times increases as the test proceeds. The test typically begins with an estimated time of around 65% of the final total time, and seemingly the increase of the avg time is linear. All in all: ---------- D7A and D2005 still beats D2006 in this third round. Q: Can it be that Bold simply is so stable and mature that it's immune against any attempts to induce any new "artificial improvements"?... :) Regards, // Rolf Lampa Back to top Jesper Hogstrom (Borland) Guest Posted: Mon Sep 04, 2006 1:59 pm    Post subject: Re: Performance of Bold with different Delphi versions - D20 Rolf Lampa [RIL] wrote: Quote: Q: Can it be that Bold simply is so stable and mature that it's immune against any attempts to induce any new "artificial improvements"?... Hiya Rolf, I haven't looked into this in any detail, but one major speed boost for D2006 was the introduction of FastMM. Since Bold installs its own MM we may inadvertently have innoculated the Bold sources against FastMM. You mentioned you did compile with BOLD_DISABLEMEMORYMANAGER. Can you perform any tests to see that you really really didn't use our own MM? --Jesper Back to top Rolf Lampa [RIL] Guest Posted: Tue Sep 05, 2006 12:45 am    Post subject: Re: Performance of Bold with different Delphi versions - D20 Jesper Hogstrom (Borland) wrote: Quote: Rolf Lampa [RIL] wrote: Q: Can it be that Bold simply is so stable and mature that it's immune against any attempts to induce any new "artificial improvements"?... :) Hiya Rolf, I haven't looked into this in any detail, but one major speed boost for D2006 was the introduction of FastMM. Since Bold installs its own MM we may inadvertently have innoculated the Bold sources against FastMM. You mentioned you did compile with BOLD_DISABLEMEMORYMANAGER. Can you perform any tests to see that you really really didn't use our own MM? Hi, Yes, perhaps it isn't enough to define it only in the project file. I'll give it a try asap. Regards, // Rolf Lampa Back to top Rolf Lampa [RIL] Guest Posted: Tue Sep 05, 2006 4:36 am    Post subject: Re: Performance of Bold with different Delphi versions - D20 Rolf Lampa [RIL] wrote: Quote: Jesper Hogstrom (Borland) wrote: Rolf Lampa [RIL] wrote: Q: Can it be that Bold simply is so stable and mature that it's immune against any attempts to induce any new "artificial improvements"?... :) Hiya Rolf, I haven't looked into this in any detail, but one major speed boost for D2006 was the introduction of FastMM. Since Bold installs its own MM we may inadvertently have innoculated the Bold sources against FastMM. You mentioned you did compile with BOLD_DISABLEMEMORYMANAGER. Can you perform any tests to see that you really really didn't use our own MM? Now I defined the variable in the Project > Options > Directories/Conditional defines field. This time it seems like D2006 was a little bit faster. But only a little, and when comparing to earlier results I suspect that the deviation in results (from earlier runs) is due to differences in the windows environment only: D2006: Total time: 00:10:39, Sec/obj=0,00225 D2005: Total time: 00:11:30, Sec/obj=0,00243 Earlier results was: D2006: Total time: 00:12:31, Sec/obj 0,00265 D2005: Total time: 00:12:19, Sec/obj 0,00260 I would have expected seeing D2006 spend significantly lesser time on this than it actually does. Hope I didn't destroy someones day with this. As said, the test does the following: It scans a file (stream), reads 2K blocks of xml string data (huuuuge files), extracts "raw tags" directly from strings (no resizing of blocks or strings = extremely fast execution). Then it: 0. Starts a (Bold) transaction. 1. Creates batches (2000) of objects of one type (with 2 string attributes and one blob field). 2. Inserts the strings into the attributes. 3. Commits the (bold) transaction. 3. Saves the batch to IB database (6.5). Well, thats about it. Logging is performed only at every batch so it shouldn't disturb. My only guess is that my app is so smart in dealing with the strings that the FastMM of D2006 simply doesn't make any difference. ("not so smart string handling" really really can slow your app down*). Any other suggestions regarding the basic "Bold activities" involved in such simple operations? I like the stable growling of the Bold app running though, but I'm not looking forward to the 3.4 million objects test... (well, it has succeeded once already, but it takes tiiiiime). Regards, // Rolf Lampa * In my app I avoid all unnecessary resizing of strings or read blocks from file streams, and instead keeps only indexes of "start and ends" of my (logical bounds of) data. I also consistently used const declared method signatures etc. All this speeded up the file & string handling of the app more than 10 times compared to my initial "standard" solution. And this kind of stuff (the shuffling and alloc and realloc of memory for strings) might be what Memory Managers struggles with the most in ordinary applications(?), while in this app it won't have to bother much since I'm only spinning integers to keep track of where my data (strings/read blocks) starts and ends. I also read a 6.5GB file from which I import some 3.4 million objects, so I had no choice but to go "low level" like this with the strings&files. Back to top Rolf Lampa [RIL] Guest Posted: Tue Sep 05, 2006 6:31 am    Post subject: Re: Performance of Bold with different Delphi versions - D20 Rolf Lampa [RIL] wrote: Quote: ...As said, the test does the following: It scans a file (stream), reads 2K blocks of xml string data (huuuuge files), extracts "raw tags" directly from strings (no resizing of blocks or strings = extremely fast execution). Then it: 0. Starts a (Bold) transaction. 1. Creates batches of objects (2000) of one type (with 2 string attributes and one blob field). 2. Inserts the strings into the attributes. 3. Commits the (bold) transaction. 4. Saves the batch to IB database (6.5). Oh, and before starting a new batch of 2000 objects it *unloads* the objects saved in #4. This also, I assume, calms down the need for allocation / reallocation of memory on behalf of Bolds part of the operations, as the iteration essentially uses the same space in memory over and over again... Hm. Perhaps my app simply doesn't produce very much extra memory management for the FastMM to handle after all? I was hoping to cut some 50% (or more) execution time with D2006... Regards, // Rolf Lampa Back to top Jan Nordén (Borland) Guest Posted: Wed Sep 06, 2006 1:28 am    Post subject: Re: Performance of Bold with different Delphi versions - D20 Rolf Lampa [RIL] wrote: Quote: Rolf Lampa [RIL] wrote: ...As said, the test does the following: It scans a file (stream), reads 2K blocks of xml string data (huuuuge files), extracts "raw tags" directly from strings (no resizing of blocks or strings = extremely fast execution). Then it: 0. Starts a (Bold) transaction. 1. Creates batches of objects (2000) of one type (with 2 string attributes and one blob field). 2. Inserts the strings into the attributes. 3. Commits the (bold) transaction. 4. Saves the batch to IB database (6.5). Oh, and before starting a new batch of 2000 objects it unloads the objects saved in #4. This also, I assume, calms down the need for allocation / reallocation of memory on behalf of Bolds part of the operations, as the iteration essentially uses the same space in memory over and over again... Hm. Perhaps my app simply doesn't produce very much extra memory management for the FastMM to handle after all? I was hoping to cut some 50% (or more) execution time with D2006... Regards, // Rolf Lampa Hi Rolf. I haven't read everything here, but if you are comparing FastMM to the Bold memory manager, I suspect you won't see much speed improvement, In fact, I think FastMM may be even by slightly slower, but better at reusing memory. However, if you are comparing FastMM to the default Delphi MM, I.e. BOLD_DISABLEMEMORYMANAGER defined in both cases, then I would indeed expect an improvement. -- Jan Nordén Architect, ECO Team Back to top Rolf Lampa [RIL] Guest Posted: Wed Sep 06, 2006 2:42 am    Post subject: Re: Performance of Bold with different Delphi versions - D20 Jan Nordén (Borland) wrote: Quote: However, if you are comparing FastMM to the default Delphi MM, I.e. BOLD_DISABLEMEMORYMANAGER defined in both cases, then I would indeed expect an improvement. Hi Jan, If I understood you correctly this is what I did (by implication) by defining BOLD_DISABLEMEMORYMANAGER for both D2006 (= MM handled by FastMM) and D2005 (= MM handled by default MM)? For now I have the feeling that it's a dead end to try to gain better speed through D2006 on this one. == What I'll try next is to try take benefit from the initially fairly low "estimated total time" (starting at about 60% of the final execution time) by "reigniting" the Bold system between each batch of imported objects. I've never tried this trick before but I assume that by deactivating the BoldSystem then all the BoldObjectLocators would be freed from memory also? My assumption is that the increasing number of BoldLocators play a significant role in causing the avg execution times for objects to increase during import(?). Restarting the BoldSystem would possibly cause some cost in the other end, like increasing the overhead for having to reallocate memory repeatedly for the BoldSystem(?). If so then perhaps FastMM would make a difference. Another benefit with closing the BoldSystem would be that it requires storing the index for "Last Imported Object" between batches, because the import process would have to terminate entirely (since it's managed by a BoldObject). Knowing the index of the last imported object I will be able to resume the whole import process again, starting from the LastImportedObject + 1. Such a "Resume" feature could actually become a "savior" for very big import files (millions of objects) if I can't find any other way to significantly speed this process up. Hm, this means I could even perform import as OnIdle tasks... Perhaps it's not all that bad after all. :) Regards, // Rolf Lampa Back to top Peter Morris [Droopy eyes Guest Posted: Wed Sep 06, 2006 8:11 am    Post subject: Re: Performance of Bold with different Delphi versions - D20 Hi Rolf Are you also unloading objects? I suspect that you might see an improvement when you reallocate memory rather than just allocate it. -- Pete Blessed are the geek, for they shall public class GeekEarth : Earth {} ==== Audio compression components, DIB graphics controls, ECO extensions, FastStrings http://www.droopyeyes.com ==== Back to top Rolf Lampa [RIL] Guest Posted: Wed Sep 06, 2006 6:28 pm    Post subject: Re: Performance of Bold with different Delphi versions - D20 Peter Morris [Droopy eyes software] wrote: Quote: Are you also unloading objects? I suspect that you might see an improvement when you reallocate memory rather than just allocate it. Hi Pete, Yes, I unload after each batch (of 2000 objects). But BoldObjectLocators will remain in memory even if unloading the objects. Having lots of Locators invoked seems to impact performance quite significantly, a lesson learned from another project. Dunno why so though. Regards, // Rolf Lampa Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         BorlandTalk.com Forum Index -> Delphi Model Driven Architecture (General) All times are GMT Page 1 of 1   Jump to: Select a forum C++ Builder Newsgroups----------------C++ BuilderC++ Builder (ActiveX)C++ Builder (Announcements)C++ Builder (Command Line Tools)C++ Builder DatabasesC++ Builder Databases (ADO)C++ Builder Databases (Desktop)C++ Builder Databases (InterBase Express)C++ Builder Databases (Multi-Tier)C++ Builder Databases (SQL Servers)C++ Builder (Graphics)C++ Builder (IDE)C++ Builder (Internet)C++ Builder (Internet Socket)C++ Builder (Internet Web)C++ Builder (Jobs)C++ Builder (Language)C++ Builder (Language C++)C++ Builder (Compatibility with MFC)C++ Builder (Native API)C++ Builder (Non-Technical)C++ Builder (Students)C++ Builder (Third Party Tools)C++ Builder (Upgrade)C++ Builder (VCL)C++ Builder (VCL Components Usage)C++ Builder (VCL Components Development)C++ Builder (VCL Usage)C++ Builder (WinAPI)C++ Builder Installation Delphi Newsgroups----------------Delphi (General)Delphi ActiveX Controls UsageDelphi ActiveX Controls DevelopmentDelphi As400Delphi DatabasesDelphi Databases (ADO)Delphi Databases (dbExpress)Delphi Databases (Desktop)Delphi Databases (InterBase Express)Delphi Databases (Multi-Tier)Delphi Databases (SQL Servers)Delphi DeploymentDelphi GraphicsDelphi IDEDelphi InternationalizationDelphi InternetDelphi Internet Isapi-WebBrokerDelphi Internet WebSnapDelphi Internet WinsockDelphi JobsDelphi Language BASMDelphi Language ObjectPascalDelphi Model Driven Architecture (General)Delphi Model Driven Architecture (Third Party)Delphi MultimediaDelphi Native APIDelphi Non-TechnicalDelphi ObjectPascalDelphi OLE AutomationDelphi OO designDelphi OpenToolsAPIDelphi Reporting-ChartingDelphi RTLDelphi StudentsDelphi TeamSourceDelphi Thirdparty ToolsDelphi Thirdparty Tools (General)Delphi Thirdparty Tools (IntraWeb)Delphi Thirdparty Tools (RaveReports)Delphi UpgradeDelphi VCLDelphi VCL ComponentsDelphi VCL Components UsingDelphi VCL Components WritingDelphi WebServices SOAPDelphi WinAPIDelphi XMLDelphi Installationcomp.lang.pascal.delphi.components.misccomp.lang.pascal.delphi.components.usagecomp.lang.pascal.delphi.components.writingcomp.lang.pascal.delphi.databasescomp.lang.pascal.delphi.misccomp.sources.delphi JBuilder Newsgroups----------------JBuilderJBuilder AnnouncementsJBuilder Applet IssuesJBuilder As400JBuilder CompilerJBuilder CORBA/RMIJBuilder DatabasesJBuilder DB SwingJBuilder DebuggerJBuilder DeploymentJBuilder DocumentationJBuilder EnterpriseJBuilder HandheldJBuilder IDEJBuilder JavaJBuilder JavaAPIJBuilder Java LanguageJBuilder Java SwingJBuilder Java BeansJBuilder Java Beans UsageJBuilder Java Beans DevelopmentJBuilder JBCLJBuilder JDataStoreJBuilder JobsJBuilder LinuxJBuilder MacintoshJBuilder & MobileSetJBuilder MultiLingual AppsJBuilder Non-TechnicalJBuilder OpenToolsJBuilder Servlets (JSP)JBuilder StudentsJBuilder Team DevelopmentJBuilder Thirdparty ToolsJBuilder UpgradeJBuilder Installation InterBase Newsgroups----------------InterBaseInterBase Bug DiscussionInterBase GeneralInterBase IBConsoleInterBase OpenSourceInterBase ReplicationInterBase SQLInterBase ThirdpartyInterBase Installation Miscellaneous Borland Newsgroups----------------AppCenterAppServerEnteraVisiBroker  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.