BorlandTalk.com Borland discussion newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  New polls on Yahoo Groups Goto page 1, 2, 3  Next          BorlandTalk.com Forum Index -> Delphi Language BASM View previous topic :: View next topic   Author Message Bruce McGee Guest Posted: Wed Jan 18, 2006 2:31 am    Post subject: New polls on Yahoo Groups Please take a minute to vote on the new polls on Yahoo Groups. If you aren't sure of the implications, please post your comments here. The questions are: 1) Some individual benchmarks and validations run for a very long time. In some cases, days. Should very long running tests be stopped after a certain amount of time? This question is specifically for the memory manager challenge but would apply to others. 2) Should Delphi 2006 be added to the 2005 memory manager challenge close to the time the challenge is scheduled to finish? -- Regards, Bruce McGee Glooscap Software Back to top Dennis Guest Posted: Wed Jan 18, 2006 6:52 am    Post subject: Re: New polls on Yahoo Groups Hi Bruce I think that the most important question is: Should benchmarks be run as validations (for validation purpose) on D6-D2005? This is our major point of disagreement as I see it. Best regards Dennis Kjaer Christensen Back to top Dennis Guest Posted: Wed Jan 18, 2006 7:02 am    Post subject: Re: New polls on Yahoo Groups Hi Bruce Quote: 1) Some individual benchmarks and validations run for a very long time. In some cases, days. Should very long running tests be stopped after a certain amount of time? This question is specifically for the memory manager challenge but would apply to others. I voted "Yes but make the upper limit 2 days". I previously argued and argue again that any MM that is able to run through the benchmarks and validations on all OS's, CPU's, Compilers etc are valid. We have no rules that any MM or function must be faster than any given limit. In the real world: DKCMM is awfully slow but passes. It can run for days. Nexus locks up in one subbenchmark. All other MM's either pass or fail within a very short time. We have do use some judgement on DKCMM and Nexus. Quote: 2) Should Delphi 2006 be added to the 2005 memory manager challenge close to the time the challenge is scheduled to finish? No the rules for the 2005 competition states that D6-D2005 are baseline compilers for validation and D2005 is baseline for benchmarks. This rule is carved in stone and should not be discussed after all deadlines have been passed. It is documented on the voting history paged which Dennis L has removed from the site. "close to the time the challenge is scheduled to finish". Deadlines have all been passed. Question is bad. Regards Dennis C Back to top Dennis Guest Posted: Wed Jan 18, 2006 7:20 am    Post subject: Re: New polls on Yahoo Groups Hi D2006 will be added as a compiler for validation for the 2006 competition. D2006 will replace D2005 as baseline for benchmarking for the 2006 competition. The 2006 competition started at 00:00 1/1 2006, but we are still trying to finish the 2005 competition. Deadline for finishing benchmarking and validation and spreadsheets and everything for the 2005 competition was at 7/1-2006. This deadline has been passed by 11 days meaning that we have been unable to settle the competition in 2.57 as much time as we normally use to settle it. We will not be able to start real work on the 2006 competition until we have settled the 2005 competition. Starting to work ahead by releasing new B&V's and new functions would add to much noise into a process that we can hardly handle anyway. For this reason I am holding back the CompareMem B&V 2.0 (and the next release of my fantastic MM ). We also cannot start testing the latest Nexus MM until we have finished testing the old one. It is sad to be in this state. Best regards Dennis Kjaer Christensen Back to top Dennis Guest Posted: Wed Jan 18, 2006 7:36 am    Post subject: Re: New polls on Yahoo Groups Hi Bruce and Community (which includes Bruce (and me )))) I think we should have a poll about this Quote: Should benchmarks be run as validations (for validation purpose) on D6-D2005? Best regards Dennis Kjaer Christensen Back to top Avatar Zondertau Guest Posted: Wed Jan 18, 2006 8:01 am    Post subject: Re: New polls on Yahoo Groups Quote: 2) Should Delphi 2006 be added to the 2005 memory manager challenge close to the time the challenge is scheduled to finish? There is no option "Validate on all compilers (i.e. D6 - D2006) and benchmark only on the FastCode target compiler (IIRC D2005)". Unfortunately i cannot edit the poll since this will remove existing votes. -- The Fastcode Project: http://www.fastcodeproject.org/ Back to top Dennis Guest Posted: Wed Jan 18, 2006 8:06 am    Post subject: Re: New polls on Yahoo Groups Hi Avatar I think that this specific poll is bogus because we have rules that tells exactly what to do. It is a very bad idea to change rules after deadline. Best regards Dennis Kjaer Christensen Back to top Bruce McGee Guest Posted: Wed Jan 18, 2006 10:05 am    Post subject: Re: New polls on Yahoo Groups Dennis wrote: Quote: Hi Bruce 1) Some individual benchmarks and validations run for a very long time. In some cases, days. Should very long running tests be stopped after a certain amount of time? This question is specifically for the memory manager challenge but would apply to others. I voted "Yes but make the upper limit 2 days". I previously argued and argue again that any MM that is able to run through the benchmarks and validations on all OS's, CPU's, Compilers etc are valid. We have no rules that any MM or function must be faster than any given limit. In the real world: DKCMM is awfully slow but passes. It can run for days. Nexus locks up in one subbenchmark. All other MM's either pass or fail within a very short time. We have do use some judgement on DKCMM and Nexus. I voted that an individual test should be stopped in a matter of hours. If a memory manager takes hours or days to run a single test, then it should be failed. Keep in mind that there are 46 individual benchmarks and 27 validations that are run. My testing literally runs for days on fast machines almost exclusively because of a few very slow MMs and one lock up. Which brings up the other reason I want to have some kind of reasonable limit. How do you distinguish between a very very long test and a lock up? Waiting 2 days to find out seems excessive. Quote: 2) Should Delphi 2006 be added to the 2005 memory manager challenge close to the time the challenge is scheduled to finish? No the rules for the 2005 competition states that D6-D2005 are baseline compilers for validation and D2005 is baseline for benchmarks. This rule is carved in stone and should not be discussed after all deadlines have been passed. It is documented on the voting history paged which Dennis L has removed from the site. "close to the time the challenge is scheduled to finish". Deadlines have all been passed. Question is bad. I added this question because you argued first that Delphi 2006 validations needed to be run (this is why ValidateAll.bat includes it) and then that benchmarks needed to be run against D2006. This was before the deadline. Recently I asked about benchmarks on the other compilers and I thought you said they should also be run. I apologise if I misunderstood and you were talking about after the 2005 competition. I'm happy with the original rule (3 sets of validations and one set of benchmarks). If you are as well, I'll change the question to reflect that it will be for the next test. -- Regards, Bruce McGee Glooscap Software Back to top Bruce McGee Guest Posted: Wed Jan 18, 2006 10:05 am    Post subject: Re: New polls on Yahoo Groups Dennis wrote: Quote: Hi Bruce and Community (which includes Bruce (and me )))) I think we should have a poll about this Should benchmarks be run as validations (for validation purpose) on D6-D2005? Best regards Dennis Kjaer Christensen We don't disagree that benchmarks also serve as validations. We only disagree on how many need to be run. This is what I intended with th current poll. Only the D2005 benchmarks will be used in the totals, but any benchmarks on other compilers would serve as validations to prove that the benchmarks work as expected. Should I change the question to make this more clear? -- Regards, Bruce McGee Glooscap Software Back to top Bruce McGee Guest Posted: Wed Jan 18, 2006 10:07 am    Post subject: Re: New polls on Yahoo Groups Avatar Zondertau wrote: Quote: 2) Should Delphi 2006 be added to the 2005 memory manager challenge close to the time the challenge is scheduled to finish? There is no option "Validate on all compilers (i.e. D6 - D2006) and benchmark only on the FastCode target compiler (IIRC D2005)". Unfortunately i cannot edit the poll since this will remove existing votes. This is what I meant by the second option. It should be something more like "same as #1 except also run validations on D2006". It may be a moot point for the 2005 competition, though. See Dennis' recent posts on the subject. -- Regards, Bruce McGee Glooscap Software Back to top Bruce McGee Guest Posted: Wed Jan 18, 2006 10:07 am    Post subject: Re: New polls on Yahoo Groups Dennis wrote: Quote: Hi Avatar I think that this specific poll is bogus because we have rules that tells exactly what to do. It is a very bad idea to change rules after deadline. Best regards Dennis Kjaer Christensen Dennis, you were the one pushing to run benchmarks and validations on more compilers than was included in the original rule. Did you change your mind or did I misunderstand you? -- Regards, Bruce McGee Glooscap Software Back to top Dennis Guest Posted: Wed Jan 18, 2006 10:55 am    Post subject: Re: New polls on Yahoo Groups Hi I think that voters do not know what they are voting about in the "Should Delphi 2006 be added to the 2005 memory manager challenge close to the time the challenge is scheduled to finish?" poll. Those who vote "Yes. Run benchmarks and validations against all compilers." probably think it makes sense to validate and benchmark against D2006. It does but not for the 2005 competition. D2006 will/should be added for the 2006 competition. Those that vote yes also vote that we should benchmark and validate against D1, D2, D3, D4 and D5. This is not a good idea for many reasons discussed properly prior to the 2/7 voting. The poll does not mention that we are polling about a rule for the 2005 competition. The rule is described at the voting history page http://www.fastcode.dk/fastcodeproject/fastcodeproject/67.htm 2/7 2005 Compiler Baseline Voting Use as baseline for validation 1)D5-D2005 2)D6-D2005 8 votes: Dennis, Bruce, Francois, Fikret, Joe, Eric Grange, Michael Stum, John O'Harrow 3)D7-D2005 4)D2005 Use as baseline for benchmarking 1)D5 2)D6 3)D7 4)D2005 9 Votes: Dennis, Bruce, Francois, Fikret, Joe, Eric Grange, Michael Stum, Avatar, John O'Harrow The voting history page is unfortunately not present at our new site. Best regards Dennis Kjaer Christensen Back to top Dennis Guest Posted: Wed Jan 18, 2006 11:01 am    Post subject: Re: New polls on Yahoo Groups Hi All Quote: 1) Some individual benchmarks and validations run for a very long time. In some cases, days. Should very long running tests be stopped after a certain amount of time? This question is specifically for the memory manager challenge but would apply to others. This "Yes. Make the upper limit a matter of hours" is in position to win. This means that my MM is ruled out. I have spent many ours on developing the MM and now you are going to change the rules after deadline and effectively stealing my 50 points. Do you think this is fair? Am I allowed to run my MM through to a pass and report these results and will it enable me to claim the 50 points? I am expecting answers from these people: bmcgee@... roberto@... pierre@... knightkiller@... marius@... Regards Dennis C Back to top Bruce McGee Guest Posted: Wed Jan 18, 2006 11:27 am    Post subject: Re: New polls on Yahoo Groups Dennis wrote: Quote: This "Yes. Make the upper limit a matter of hours" is in position to win. This means that my MM is ruled out. I have spent many ours on developing the MM and now you are going to change the rules after deadline and effectively stealing my 50 points. Do you think this is fair? Am I allowed to run my MM through to a pass and report these results and will it enable me to claim the 50 points? I am expecting answers from these people: bmcgee@... roberto@... pierre@... knightkiller@... marius@... Regards Dennis C I think it makes sense to start using this rule in the 2006 competition. -- Regards, Bruce McGee Glooscap Software Back to top Jouni Turunen Guest Posted: Wed Jan 18, 2006 11:46 am    Post subject: Re: New polls on Yahoo Groups Dennis wrote: Quote: Hi All 1) Some individual benchmarks and validations run for a very long time. In some cases, days. Should very long running tests be stopped after a certain amount of time? This question is specifically for the memory manager challenge but would apply to others. This "Yes. Make the upper limit a matter of hours" is in position to win. This means that my MM is ruled out. I have spent many ours on developing the MM and now you are going to change the rules after deadline and effectively stealing my 50 points. Do you think this is fair? Am I allowed to run my MM through to a pass and report these results and will it enable me to claim the 50 points? I am expecting answers from these people: bmcgee@... roberto@... pierre@... knightkiller@... marius@... Hi Dennis, I think that nobody wants to take away your 50 points. IMHO only purpose in this discussion is to improve situation : to get large amount of validations runs done in reasonable time, fully automated. Here's an example. I've had validation running on my AMD XP. D7 DKCIA32MM regular validation took about 30 minutes (not bad). My AMD has been running D6 DKCIA32MM regular validation for 6 hours and I don't know if its stuck or just slow. I don't want to take away your 50 points but it's true that without slow DKCIA32MM I would have validation results there completed earlier. For me it sounds perfectly fair if you run DKCIA32MM validations and benchmarks and the rest of us concentrate on the others. I voted 2 days but I'd be much happier if I don't need to wait for something that long. Like you mentioned, 2005 deadline was days ago. Regards, Jouni -- The Fastcode Project: http://www.fastcodeproject.org/ Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         BorlandTalk.com Forum Index -> Delphi Language BASM All times are GMT Goto page 1, 2, 3  Next Page 1 of 3   Jump to: Select a forum C++ Builder Newsgroups----------------C++ BuilderC++ Builder (ActiveX)C++ Builder (Announcements)C++ Builder (Command Line Tools)C++ Builder DatabasesC++ Builder Databases (ADO)C++ Builder Databases (Desktop)C++ Builder Databases (InterBase Express)C++ Builder Databases (Multi-Tier)C++ Builder Databases (SQL Servers)C++ Builder (Graphics)C++ Builder (IDE)C++ Builder (Internet)C++ Builder (Internet Socket)C++ Builder (Internet Web)C++ Builder (Jobs)C++ Builder (Language)C++ Builder (Language C++)C++ Builder (Compatibility with MFC)C++ Builder (Native API)C++ Builder (Non-Technical)C++ Builder (Students)C++ Builder (Third Party Tools)C++ Builder (Upgrade)C++ Builder (VCL)C++ Builder (VCL Components Usage)C++ Builder (VCL Components Development)C++ Builder (VCL Usage)C++ Builder (WinAPI)C++ Builder Installation Delphi Newsgroups----------------Delphi (General)Delphi ActiveX Controls UsageDelphi ActiveX Controls DevelopmentDelphi As400Delphi DatabasesDelphi Databases (ADO)Delphi Databases (dbExpress)Delphi Databases (Desktop)Delphi Databases (InterBase Express)Delphi Databases (Multi-Tier)Delphi Databases (SQL Servers)Delphi DeploymentDelphi GraphicsDelphi IDEDelphi InternationalizationDelphi InternetDelphi Internet Isapi-WebBrokerDelphi Internet WebSnapDelphi Internet WinsockDelphi JobsDelphi Language BASMDelphi Language ObjectPascalDelphi Model Driven Architecture (General)Delphi Model Driven Architecture (Third Party)Delphi MultimediaDelphi Native APIDelphi Non-TechnicalDelphi ObjectPascalDelphi OLE AutomationDelphi OO designDelphi OpenToolsAPIDelphi Reporting-ChartingDelphi RTLDelphi StudentsDelphi TeamSourceDelphi Thirdparty ToolsDelphi Thirdparty Tools (General)Delphi Thirdparty Tools (IntraWeb)Delphi Thirdparty Tools (RaveReports)Delphi UpgradeDelphi VCLDelphi VCL ComponentsDelphi VCL Components UsingDelphi VCL Components WritingDelphi WebServices SOAPDelphi WinAPIDelphi XMLDelphi Installationcomp.lang.pascal.delphi.components.misccomp.lang.pascal.delphi.components.usagecomp.lang.pascal.delphi.components.writingcomp.lang.pascal.delphi.databasescomp.lang.pascal.delphi.misccomp.sources.delphi JBuilder Newsgroups----------------JBuilderJBuilder AnnouncementsJBuilder Applet IssuesJBuilder As400JBuilder CompilerJBuilder CORBA/RMIJBuilder DatabasesJBuilder DB SwingJBuilder DebuggerJBuilder DeploymentJBuilder DocumentationJBuilder EnterpriseJBuilder HandheldJBuilder IDEJBuilder JavaJBuilder JavaAPIJBuilder Java LanguageJBuilder Java SwingJBuilder Java BeansJBuilder Java Beans UsageJBuilder Java Beans DevelopmentJBuilder JBCLJBuilder JDataStoreJBuilder JobsJBuilder LinuxJBuilder MacintoshJBuilder & MobileSetJBuilder MultiLingual AppsJBuilder Non-TechnicalJBuilder OpenToolsJBuilder Servlets (JSP)JBuilder StudentsJBuilder Team DevelopmentJBuilder Thirdparty ToolsJBuilder UpgradeJBuilder Installation InterBase Newsgroups----------------InterBaseInterBase Bug DiscussionInterBase GeneralInterBase IBConsoleInterBase OpenSourceInterBase ReplicationInterBase SQLInterBase ThirdpartyInterBase Installation Miscellaneous Borland Newsgroups----------------AppCenterAppServerEnteraVisiBroker  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.