BorlandTalk.com Borland discussion newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  OPF Batch Processing Goto page 1, 2  Next          BorlandTalk.com Forum Index -> Delphi OO design View previous topic :: View next topic   Author Message preston Guest Posted: Thu Oct 06, 2005 3:14 pm    Post subject: OPF Batch Processing I saw this brought up about a month ago, but it's still clear as mud to me so I thought I'd bring it up again. The question was if you're using OPF how do you perform an operation that requires updating many objects, a customer balance for example. I see the following solution: 1. Write a query that does it. "Update customers set balance = (select sum(amount) from invoices where customerID = :CustoemrID". 2. Load each customer object using OPF and calls it's specific update balance method. It's obvious the first is going to be an order of magnitude faster than the second, but it really doesn't fit the OPF model. Ultimately, the question is how do you get something like that to fit the OPF model? An example would be great :) Preston Back to top Jim Cooper Guest Posted: Thu Oct 06, 2005 3:56 pm    Post subject: Re: OPF Batch Processing Quote: It's obvious the first is going to be an order of magnitude faster than the second, but it really doesn't fit the OPF model. It can fit the OPF model, it just depends what classes you have. The simplistic way to model things with objects is to just have classes corresponding to nouns in a requirement, hence your implicit assumption that you will have a Customer class. However, other things often need to be modelled by classes too, like relationships between objects, or specific actions. An example of the latter might be a class that handled batch customer balance updates. There is then the question of whether an OPF can use a customised query or not. Some autogenerate all their SQL, in which case you might have a problem, but most I've seen allow some sort of customisation at that level. Cheers, Jim Cooper __________________________________________ Jim Cooper [email]jcooper (AT) tabdee (DOT) ltd.uk[/email] Tabdee Ltd http://www.tabdee.ltd.uk TurboSync - Connecting Delphi to your Palm __________________________________________ Back to top preston Guest Posted: Thu Oct 06, 2005 4:14 pm    Post subject: Re: OPF Batch Processing Quote: There is then the question of whether an OPF can use a customised query or not. Some autogenerate all their SQL, in which case you might have a problem, but most I've seen allow some sort of customisation at that level. You hit the nail on the head. I thought the goal of OPF was to hide the database. Wouldn't writing a custom SQL script defeat that? .... I just learned about ECO and was reading "What is ECO anyway" article on Borland's site. In the article they do something similar. The example was to show how ECO recurses through all objects to calculate a simple sum. Sounds much slower than a simple select. Or does ECO do some magic that makes it very fast. Preston Back to top Bob Dawson Guest Posted: Thu Oct 06, 2005 5:00 pm    Post subject: Re: OPF Batch Processing "preston" wrote Quote: You hit the nail on the head. I thought the goal of OPF was to hide the database. Wouldn't writing a custom SQL script defeat that? Not at all. Hiding the database means hiding it from layers forward of the connection level, not imposing any particular way for the connection level to go about doing its job. While most OPF writers implement custom SQL generators in the connection level, that's certainly not the only way to proceed. bobD Back to top Jim Cooper Guest Posted: Thu Oct 06, 2005 5:55 pm    Post subject: Re: OPF Batch Processing Quote: You hit the nail on the head. I thought the goal of OPF was to hide the database. Wouldn't writing a custom SQL script defeat that? No, of course not The goal is to hide the database from the UI and application/business logic, not to completely ignore the fact that there is a database. With an OPF you end up with different logical (and sometimes physical) layers. IME there is a need to tweak the data access layer from time to time (for me that's usually to make reports run at a reasonable speed, but the batch update scenario would be another case. I just haven't had to do that). Quote: Sounds much slower than a simple select. Or does ECO do some magic that makes it very fast. Sorry, I don't know enough to say. Cheers, Jim Cooper __________________________________________ Jim Cooper [email]jcooper (AT) tabdee (DOT) ltd.uk[/email] Tabdee Ltd http://www.tabdee.ltd.uk TurboSync - Connecting Delphi to your Palm __________________________________________ Back to top preston Guest Posted: Thu Oct 06, 2005 6:49 pm    Post subject: Re: OPF Batch Processing Jim or Bob, Can you give a quick list of classes and what they might do in order to accomplish batch processing. I'm wondering where in the framework you would put it. Would it still be part of the customer's class or would it be a new class like customerBacthProcessing? Preston Back to top Bob Dawson Guest Posted: Fri Oct 07, 2005 3:46 am    Post subject: Re: OPF Batch Processing "preston" wrote Quote: I'm wondering where in the framework you would put it. Would it still be part of the customer's class or would it be a new class like customerBatchProcessing? In OO terms, when I think of a batch I'm really thinking of a collection--so in my view batch updates would be methods of class-specific collections: JimsEmps := TEmployeeCollection.Create; JimsEmps:Load('BySupervisor',[Jim.Id]); JimsEmps:GrantRaise('BySupervisor',[Jim.Id]); Internal logic of the GrantRaise method would be would be something like create internal helper object set its properties to the selection criteria of the collection save the helper object <trickery> since my framework allows custom SQL to be associated with objects, the save method of the helper object would actually invoke an appropriate update query or stored proc, which uses the selection criteria as params </trickery> free the helper object iterate the collection making the corresponding change Basically what this does is use the internal helper object as the host for your batch update query, and gives the collection responsibility for exposing the use of this object as a method of the collection. bobD Back to top Joanna Carter [TeamB] Guest Posted: Fri Oct 07, 2005 8:24 am    Post subject: Re: OPF Batch Processing "Bob Dawson" <RBDawson (AT) prodigy (DOT) net> a écrit dans le message de news: 4345f1e9$1 (AT) newsgroups (DOT) borland.com... Quote: In OO terms, when I think of a batch I'm really thinking of a collection--so in my view batch updates would be methods of class-specific collections: JimsEmps := TEmployeeCollection.Create; JimsEmps:Load('BySupervisor',[Jim.Id]); JimsEmps:GrantRaise('BySupervisor',[Jim.Id]); ..... Basically what this does is use the internal helper object as the host for your batch update query, and gives the collection responsibility for exposing the use of this object as a method of the collection. I would agree with Bob's approach to this problem. However, I still find myself arguing about the "what ifs" involved when updating the salary triggers a validity check which has to verufy that the employee has had less than 10 sick days in the past year. So whilst I agree that you could add methods to your collection classes, I also feel that the better way to cope with this is to provide an object querying syntax as part of the compiler as in Microsoft's new Linq ideas. I always thought that OCL was a good idea, but lacking in its execution in Bold/ECO as it relied on passing strings around. OCL also implies a language for specifying constraints and not queries, so I am not happy with the thought that this "language" has morphed from its original intent without changing its name. I would like to see a syntax something like this : TEmployee.Where(TEmployee.SickLeave <= 10).Update(TEmployee.Salary += 10%) The intent is that you first specify the type upon which the operation is to executed, then the type has an operation like "Where" which "returns" a collection; this collection then has an operation like "Update" which allows us to pass in a property and an expression. If it were possible, the properties passed in to the operations should be checkable at compile time as being a valid part of the type to which they are being applied. This kind of "OQL" expression would be easier to translate to auto-generated SQL. What does anyone else think ? Joanna -- Joanna Carter [TeamB] Consultant Software Engineer Back to top Jim Cooper Guest Posted: Fri Oct 07, 2005 9:35 am    Post subject: Re: OPF Batch Processing Quote: I'm wondering where in the framework you would put it. Would it still be part of the customer's class or would it be a new class like customerBacthProcessing? Almost certainly not in the customer class, since it is not to do with one customer. You could either put it in a customer collection class, as per Bob's example, or you might want a separate class. To some extent this will depend on whether there are other similar operations, extra methods needed, whether the update is only for customers, or whether the same update might also be required for suppliers, say, and so on. It might also depend on how your OPF lets you add custom data layer operations. However, these days I don't get too uptight about those choices, as I'm comfortable with being able to refactor it later. Cheers, Jim Cooper __________________________________________ Jim Cooper [email]jcooper (AT) tabdee (DOT) ltd.uk[/email] Tabdee Ltd http://www.tabdee.ltd.uk TurboSync - Connecting Delphi to your Palm __________________________________________ Back to top Anders Ivner Guest Posted: Fri Oct 07, 2005 11:37 am    Post subject: Re: OPF Batch Processing "preston" <oracle (AT) xmission (DOT) com> wrote Quote: I just learned about ECO and was reading "What is ECO anyway" article on Borland's site. In the article they do something similar. The example was to show how ECO recurses through all objects to calculate a simple sum. Sounds much slower than a simple select. Or does ECO do some magic that makes it very fast. While ECO does in fact do some magic to speed things up (it will try to load many objects at once), a simple select will indeed be faster than first loading the objects and then calculating the sum on the loaded objects. Then again, if the objects are already loaded, ECO will perform the operation without querying the database at all. That should be faster :) /Anders Back to top Bob Dawson Guest Posted: Fri Oct 07, 2005 2:07 pm    Post subject: Re: OPF Batch Processing "Joanna Carter [TeamB]" wrote Quote: However, I still find myself arguing about the "what ifs" involved when updating the salary triggers a validity check which has to verify that the employee has had less than 10 sick days in the past year. Yes, essentially this is one core of the design problem. While it quite possible to accomodate group/batch updates in an OPF, from a 'how to code it' perspective, it remains that doing so to some extent constitutes altering objects 'behind their back' and so compromises object sovereignty. To a certain extent this objection is answered by the design of the collection: that is, Joanna's sick days exception would have to be incorporated into the SQL that loads the collection, so that the operation is preknown to be valid for all items. For example, we could say JimsEmps := TUpdateableSalaryEmployeeCollection.Create; JimsEmps:Load('BySupervisor',[Jim.Id]); JimsEmps:GrantPercentRaise(5); But our load here is much more complicated--the collection query may need to know how to select only those employees who --meet time in grade requirement --are not in any personnel hold or probation status that precludes favorable action --are not retiring if raises immediately before retirement are precluded --aren't over their time-off allowance --etc. Presumeably these criteria are themselves datapoints, and the Collection query ends up being a fairly complex stored proc that satisifes the selection rules. Note two things: this path potentially leads to many, many custom collection classes, and much business logic has been moved from the object layer into the database. Batch updates in the face of complex business rules, then, provide us with a variety of unpleasant alternatives: (1) embed the business rules as collection princples of selection. (2) double code the rules--in the object layer and in the database (either the SQL update statement itself, or via triggers or some other technique) (3) makes sure all affected objects are in memory, and abort the update unless those objects all agree to the change (4) make sure no affected objects are in memory, and remove this particular alteration from the object model entirely --- I mentioned that handling the selection principles is 'one core of the design problem.' The other is maintaining state consistency between the in-memory object hive and the database. This has two manifestations: first, alerting the object that its db state has changed, and second, even supposing we double code the business rules for raise-eligibility, it remains possible for a dirty object in memory to come to a different conclusion about its eligibility than the query or stored proc comes to using its trace in the database. Quote: This kind of "OQL" expression would be easier to translate to auto-generated SQL. Would make a lot of things easier, certainly. Though don't think it answers that final conundrum. bobD Back to top Bob Dawson Guest Posted: Fri Oct 07, 2005 2:10 pm    Post subject: Re: OPF Batch Processing "Jim Cooper" wrote Quote: However, these days I don't get too uptight about those choices, as I'm comfortable with being able to refactor it later. Jim--you have a paper or talk planned on 'writing for refactoring'? bobD Back to top Jim Cooper Guest Posted: Fri Oct 07, 2005 2:34 pm    Post subject: Re: OPF Batch Processing Quote: Jim--you have a paper or talk planned on 'writing for refactoring'? I would if I could It's a topic I've often thought about, but it's hard to come up with examples, or indeed sometimes to pin down exactly what the difference is writing code that way. It's something that's easier to point out in a code review or suchlike. I've started to make a note when I find a relevant example, so maybe one day. I don't think it'll be a talk at a US BorCon though, it seems those days are gone :( Cheers, Jim Cooper __________________________________________ Jim Cooper [email]jcooper (AT) tabdee (DOT) ltd.uk[/email] Tabdee Ltd http://www.tabdee.ltd.uk TurboSync - Connecting Delphi to your Palm __________________________________________ Back to top Iman L Crawford Guest Posted: Fri Oct 07, 2005 3:28 pm    Post subject: Re: OPF Batch Processing Jim Cooper <jcooper (AT) tabdee (DOT) ltd.uk> wrote in news:43468708$1 @newsgroups.borland.com: Quote: I don't think it'll be a talk at a US BorCon though, it seems those days are gone Is there going to be OODesign birds of a feather? -- Iman Back to top preston Guest Posted: Fri Oct 07, 2005 3:32 pm    Post subject: Re: OPF Batch Processing Quote: I mentioned that handling the selection principles is 'one core of the design problem.' The other is maintaining state consistency between the in-memory object hive and the database. This has two manifestations: first, alerting the object that its db state has changed, and second, even supposing we double code the business rules for raise-eligibility, it remains possible for a dirty object in memory to come to a different conclusion about its eligibility than the query or stored proc comes to using its trace in the database. In the J2EE world isn't this something that the application server deals with? I would assume that ECO has some sort of server side process that does the same thing. Does it not? Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         BorlandTalk.com Forum Index -> Delphi OO design All times are GMT Goto page 1, 2  Next Page 1 of 2   Jump to: Select a forum C++ Builder Newsgroups----------------C++ BuilderC++ Builder (ActiveX)C++ Builder (Announcements)C++ Builder (Command Line Tools)C++ Builder DatabasesC++ Builder Databases (ADO)C++ Builder Databases (Desktop)C++ Builder Databases (InterBase Express)C++ Builder Databases (Multi-Tier)C++ Builder Databases (SQL Servers)C++ Builder (Graphics)C++ Builder (IDE)C++ Builder (Internet)C++ Builder (Internet Socket)C++ Builder (Internet Web)C++ Builder (Jobs)C++ Builder (Language)C++ Builder (Language C++)C++ Builder (Compatibility with MFC)C++ Builder (Native API)C++ Builder (Non-Technical)C++ Builder (Students)C++ Builder (Third Party Tools)C++ Builder (Upgrade)C++ Builder (VCL)C++ Builder (VCL Components Usage)C++ Builder (VCL Components Development)C++ Builder (VCL Usage)C++ Builder (WinAPI)C++ Builder Installation Delphi Newsgroups----------------Delphi (General)Delphi ActiveX Controls UsageDelphi ActiveX Controls DevelopmentDelphi As400Delphi DatabasesDelphi Databases (ADO)Delphi Databases (dbExpress)Delphi Databases (Desktop)Delphi Databases (InterBase Express)Delphi Databases (Multi-Tier)Delphi Databases (SQL Servers)Delphi DeploymentDelphi GraphicsDelphi IDEDelphi InternationalizationDelphi InternetDelphi Internet Isapi-WebBrokerDelphi Internet WebSnapDelphi Internet WinsockDelphi JobsDelphi Language BASMDelphi Language ObjectPascalDelphi Model Driven Architecture (General)Delphi Model Driven Architecture (Third Party)Delphi MultimediaDelphi Native APIDelphi Non-TechnicalDelphi ObjectPascalDelphi OLE AutomationDelphi OO designDelphi OpenToolsAPIDelphi Reporting-ChartingDelphi RTLDelphi StudentsDelphi TeamSourceDelphi Thirdparty ToolsDelphi Thirdparty Tools (General)Delphi Thirdparty Tools (IntraWeb)Delphi Thirdparty Tools (RaveReports)Delphi UpgradeDelphi VCLDelphi VCL ComponentsDelphi VCL Components UsingDelphi VCL Components WritingDelphi WebServices SOAPDelphi WinAPIDelphi XMLDelphi Installationcomp.lang.pascal.delphi.components.misccomp.lang.pascal.delphi.components.usagecomp.lang.pascal.delphi.components.writingcomp.lang.pascal.delphi.databasescomp.lang.pascal.delphi.misccomp.sources.delphi JBuilder Newsgroups----------------JBuilderJBuilder AnnouncementsJBuilder Applet IssuesJBuilder As400JBuilder CompilerJBuilder CORBA/RMIJBuilder DatabasesJBuilder DB SwingJBuilder DebuggerJBuilder DeploymentJBuilder DocumentationJBuilder EnterpriseJBuilder HandheldJBuilder IDEJBuilder JavaJBuilder JavaAPIJBuilder Java LanguageJBuilder Java SwingJBuilder Java BeansJBuilder Java Beans UsageJBuilder Java Beans DevelopmentJBuilder JBCLJBuilder JDataStoreJBuilder JobsJBuilder LinuxJBuilder MacintoshJBuilder & MobileSetJBuilder MultiLingual AppsJBuilder Non-TechnicalJBuilder OpenToolsJBuilder Servlets (JSP)JBuilder StudentsJBuilder Team DevelopmentJBuilder Thirdparty ToolsJBuilder UpgradeJBuilder Installation InterBase Newsgroups----------------InterBaseInterBase Bug DiscussionInterBase GeneralInterBase IBConsoleInterBase OpenSourceInterBase ReplicationInterBase SQLInterBase ThirdpartyInterBase Installation Miscellaneous Borland Newsgroups----------------AppCenterAppServerEnteraVisiBroker  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.