BorlandTalk.com Borland discussion newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  Implementation question          BorlandTalk.com Forum Index -> comp.lang.pascal.delphi.misc View previous topic :: View next topic   Author Message Darius Blaszijk Guest Posted: Thu Sep 01, 2005 11:17 pm    Post subject: Implementation question For an application I'm developing I need to store data from a file in a matrix. The matrix is 2-D and can be any size (also sparse, although never much). The data itself can be real or string or a combination of the both. The reading part of the app. I would like to make available to libraries, so in future different file formats can be added. My question is, what kind of datatype to declare to make the code as portable as possible (communication between library and app). I first had in mind of using an "array of array of string", but I'm afraid this could take a lot of memory away and it will definetly not portable enough for use in C libraries. Currently I'm thinking of creating a linked list with row entries and column entries. But still the datatype is a question for me. Should I use string or array of char or something else?? Perhaps some of you have experience with this type of implementation and can give me some pointers. Darius Blaszijk Back to top Andreas Koch Guest Posted: Fri Sep 02, 2005 6:52 am    Post subject: Re: Implementation question Darius Blaszijk wrote: Quote: My question is, what kind of datatype to declare to make the code as portable as possible (communication between library and app). Inside Delphi i'd use an TObjectList of TObjectList of Class{myReal;myString}, but you won't find that portable (you are talking about non-delphi apps, right?) So perhaps use an one dimensional array of byte - and do all the structures "by hand". Possibly RLE encoded to safe space, and an index for the start addresses of all rows if you need quick access. Should avoid most compability headaches. Back to top J French Guest Posted: Fri Sep 02, 2005 12:55 pm    Post subject: Re: Implementation question On Fri, 02 Sep 2005 01:17:47 +0200, "Darius Blaszijk" <dhkblaszyk (AT) zeelandnet (DOT) nl> wrote: Quote: For an application I'm developing I need to store data from a file in a matrix. The matrix is 2-D and can be any size (also sparse, although never much). Just keep it as plain text - CSV or tab delineated Back to top Bruce Roberts Guest Posted: Fri Sep 02, 2005 3:05 pm    Post subject: Re: Implementation question "Darius Blaszijk" <dhkblaszyk (AT) zeelandnet (DOT) nl> wrote Quote: For an application I'm developing I need to store data from a file in a matrix. The matrix is 2-D and can be any size (also sparse, although never much). The data itself can be real or string or a combination of the both. The reading part of the app. I would like to make available to libraries, so in future different file formats can be added. My question is, what kind of datatype to declare to make the code as portable as possible (communication between library and app). Don't share the datastructure between library and app. Provide methods for referencing the data. Quote: I first had in mind of using an "array of array of string", but I'm afraid this could take a lot of memory away and it will definetly not portable enough for use in C libraries. Using a couple of extra MB RAM isn't worth worrying about. If you wan't portability, provide code to access the data structure and return simple common types like pChar, longInt, etc. Quote: Currently I'm thinking of creating a linked list with row entries and column entries. But still the datatype is a question for me. Should I use string or array of char or something else?? You may find an array of array of variant suitable. Unless the structure is really sparse linked lists are not worth the trouble. The Orpheus component collection includes a tSparseArray. You can get the library from sourceforge. Back to top Darius Blaszijk Guest Posted: Fri Sep 02, 2005 4:49 pm    Post subject: Re: Implementation question Quote: My question is, what kind of datatype to declare to make the code as portable as possible (communication between library and app). Perhaps it's possible to use a TStream to stream data from the DLL to the application, but is it portable to C(++)?? Darius Blaszijk Back to top Darius Blaszijk Guest Posted: Fri Sep 02, 2005 4:52 pm    Post subject: Re: Implementation question Quote: Just keep it as plain text - CSV or tab delineated So how would you see this?? The DLL reads for isntance Excel file format, converts it to CSV then the data is handed over to the app and the app decodes it back to some Type it can handle and manipulate?? Don't you think there would be unnescesary overhead?? Beside that you need to equip all DLL's with CSV support. Perhaps I misinterpret you, so please give me more details. Darius Back to top Darius Blaszijk Guest Posted: Fri Sep 02, 2005 4:59 pm    Post subject: Re: Implementation question On Fri, 02 Sep 2005 11:05:20 +0000, Bruce Roberts wrote: Quote: Don't share the datastructure between library and app. Provide methods for referencing the data. So how do you see that in more detail? The lib reads the data and with a method you can access each cell one by one?? Or do you stream the data to the app?? By passing a pointer for example to the stream. Quote: Using a couple of extra MB RAM isn't worth worrying about. If you wan't portability, provide code to access the data structure and return simple common types like pChar, longInt, etc. How about TStream? Quote: You may find an array of array of variant suitable. Unless the structure is really sparse linked lists are not worth the trouble. The Orpheus component collection includes a tSparseArray. You can get the library from sourceforge. Looking into that. Thanks. Back to top Darius Blaszijk Guest Posted: Fri Sep 02, 2005 5:06 pm    Post subject: Re: Implementation question On Fri, 02 Sep 2005 08:52:31 +0200, Andreas Koch wrote: Quote: So perhaps use an one dimensional array of byte - and do all the structures "by hand". Possibly RLE encoded to safe space, and an index for the start addresses of all rows if you need quick access. Should avoid most compability headaches. Sounds like forward star. It can do irregular array's, but not sparse. But as said normally the data is not very sparse, so It could be used pretty efficient. Darius Back to top Andreas Koch Guest Posted: Fri Sep 02, 2005 5:36 pm    Post subject: Re: Implementation question Darius Blaszijk wrote: Quote: Possibly RLE encoded to safe space, and an index for the start addresses of all rows if you need quick access. Should avoid most compability headaches. Sounds like forward star. It can do irregular array's, but not sparse. Sure it can handle sparse arrays with RLE. [End of Line] [end of line] [end of line] [end of line] [end of line] [empty x 40 ] 1 Thats an sparse array with just 1 entry. Seems pretty compact in comparison to the 6x41 vars you would need for an full arry. And you could even use some multiple-line-end-of-lines. Back to top Darius Blaszijk Guest Posted: Fri Sep 02, 2005 6:37 pm    Post subject: Re: Implementation question On Fri, 02 Sep 2005 19:36:51 +0200, Andreas Koch wrote: Quote: Sure it can handle sparse arrays with RLE. I agree with you, although I find that RLE wil not be that efficient, considering the complexity of the data. There would be few runs of similar bytes, resulting in bad performance of RLE. Besides, the size of the data will typically never exceed a few 100Kb. So not sure if encoding is absolutely nescesary. But perhaps I can add support for it later and develop a SDK for different languages. Darius Back to top J French Guest Posted: Sat Sep 03, 2005 7:59 am    Post subject: Re: Implementation question On Fri, 02 Sep 2005 18:52:58 +0200, "Darius Blaszijk" <dhkblaszyk (AT) zeelandnet (DOT) nl> wrote: Quote: Just keep it as plain text - CSV or tab delineated So how would you see this?? The DLL reads for isntance Excel file format, converts it to CSV then the data is handed over to the app and the app decodes it back to some Type it can handle and manipulate?? Don't you think there would be unnescesary overhead?? Beside that you need to equip all DLL's with CSV support. Perhaps I misinterpret you, so please give me more details. Now I'm getting more of an idea of what you are after. Bruce is spot on about storing the data in the DLL and handing it out to the main App via Functions - on request and Cell by Cell - eg: the main App does not store a copy of the data For simplicity I would wrap that as an Object in the main App - but make the Object use StdCall functions/procedures to talk to the DLL How you store the data internally inside the DLL is down to you - mostly it depends on the sparcity of the matrix I've long decided that Strings are the most flexible way of storing data - as for CSV support - you get that from TStrings.CommaText - although it is pretty easy to implement oneself Back to top Bruce Roberts Guest Posted: Sun Sep 04, 2005 2:22 am    Post subject: Re: Implementation question "Darius Blaszijk" <dhkblaszyk (AT) zeelandnet (DOT) nl> wrote Quote: On Fri, 02 Sep 2005 11:05:20 +0000, Bruce Roberts wrote: Don't share the datastructure between library and app. Provide methods for referencing the data. So how do you see that in more detail? The lib reads the data and with a method you can access each cell one by one?? Or do you stream the data to the app?? By passing a pointer for example to the stream. There are a number of different approaches and it really depends on exactly what you want to achieve and what you have to work with. Lets supose, since I don't know, that you simply want to be able to release different file format recognizers and that the app is to store and manipulate all of the data. In this case I would likely define a set of standard routines that the app would call to gather the data. Each dll might therefore provide something like the following function EndOfData : boolean; function OpenDataSource (sourceName : pChar) : boolean; procedure ReadRecord (buffer : pChar; buffLength : cardinal); procedure CloseDataSource; ReadRecord could simply place the data in the array of byte, (notice that its allocated by the caller), in some predetermined format. IOW all dll would place data in the array in the same format, regardless of how it was read. An alternative to ReadRecord might be something like procedure ReadColumn (buffer : pChar; buffLength : cardinal; var endOfRow : boolean); Or even function ColumnCount : cardinal; procedure ReadColumn (colNo : cardinal; buffer : pChar; buffLength : cardinal); procedure NextRow; Lots of options. The thing to note is that the routines provide structured access to the data and they do so using only very basic types. Quote: How about TStream? No reason why you couldn't use a stream, if that is what you really want to do. Back to top Darius Blaszijk Guest Posted: Sun Sep 04, 2005 9:19 am    Post subject: Re: Implementation question On Sat, 03 Sep 2005 22:22:44 +0000, Bruce Roberts wrote: Quote: Lots of options. The thing to note is that the routines provide structured access to the data and they do so using only very basic types. Ok Bruce, I get the picture. Guess that TStream is not nescesary in this setup, because it doesn't add much or eliminates complexity. If I stream all data to my app then I still need to untangle the entire stream to columns and rows. Just like I would do using your suggestion, but in that case I already have nice exported functions. All I need is to add the function: "function EndOfRow : boolean;" and then I can read data using the following scheme; OpenDataSource; while not EndOfData do while not EndOfRow do MyStructure.Add(ReadRecord); CloseDataSource; It's simple and portable... Thank you for helping me out here Darius Blaszijk Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         BorlandTalk.com Forum Index -> comp.lang.pascal.delphi.misc All times are GMT Page 1 of 1   Jump to: Select a forum C++ Builder Newsgroups----------------C++ BuilderC++ Builder (ActiveX)C++ Builder (Announcements)C++ Builder (Command Line Tools)C++ Builder DatabasesC++ Builder Databases (ADO)C++ Builder Databases (Desktop)C++ Builder Databases (InterBase Express)C++ Builder Databases (Multi-Tier)C++ Builder Databases (SQL Servers)C++ Builder (Graphics)C++ Builder (IDE)C++ Builder (Internet)C++ Builder (Internet Socket)C++ Builder (Internet Web)C++ Builder (Jobs)C++ Builder (Language)C++ Builder (Language C++)C++ Builder (Compatibility with MFC)C++ Builder (Native API)C++ Builder (Non-Technical)C++ Builder (Students)C++ Builder (Third Party Tools)C++ Builder (Upgrade)C++ Builder (VCL)C++ Builder (VCL Components Usage)C++ Builder (VCL Components Development)C++ Builder (VCL Usage)C++ Builder (WinAPI)C++ Builder Installation Delphi Newsgroups----------------Delphi (General)Delphi ActiveX Controls UsageDelphi ActiveX Controls DevelopmentDelphi As400Delphi DatabasesDelphi Databases (ADO)Delphi Databases (dbExpress)Delphi Databases (Desktop)Delphi Databases (InterBase Express)Delphi Databases (Multi-Tier)Delphi Databases (SQL Servers)Delphi DeploymentDelphi GraphicsDelphi IDEDelphi InternationalizationDelphi InternetDelphi Internet Isapi-WebBrokerDelphi Internet WebSnapDelphi Internet WinsockDelphi JobsDelphi Language BASMDelphi Language ObjectPascalDelphi Model Driven Architecture (General)Delphi Model Driven Architecture (Third Party)Delphi MultimediaDelphi Native APIDelphi Non-TechnicalDelphi ObjectPascalDelphi OLE AutomationDelphi OO designDelphi OpenToolsAPIDelphi Reporting-ChartingDelphi RTLDelphi StudentsDelphi TeamSourceDelphi Thirdparty ToolsDelphi Thirdparty Tools (General)Delphi Thirdparty Tools (IntraWeb)Delphi Thirdparty Tools (RaveReports)Delphi UpgradeDelphi VCLDelphi VCL ComponentsDelphi VCL Components UsingDelphi VCL Components WritingDelphi WebServices SOAPDelphi WinAPIDelphi XMLDelphi Installationcomp.lang.pascal.delphi.components.misccomp.lang.pascal.delphi.components.usagecomp.lang.pascal.delphi.components.writingcomp.lang.pascal.delphi.databasescomp.lang.pascal.delphi.misccomp.sources.delphi JBuilder Newsgroups----------------JBuilderJBuilder AnnouncementsJBuilder Applet IssuesJBuilder As400JBuilder CompilerJBuilder CORBA/RMIJBuilder DatabasesJBuilder DB SwingJBuilder DebuggerJBuilder DeploymentJBuilder DocumentationJBuilder EnterpriseJBuilder HandheldJBuilder IDEJBuilder JavaJBuilder JavaAPIJBuilder Java LanguageJBuilder Java SwingJBuilder Java BeansJBuilder Java Beans UsageJBuilder Java Beans DevelopmentJBuilder JBCLJBuilder JDataStoreJBuilder JobsJBuilder LinuxJBuilder MacintoshJBuilder & MobileSetJBuilder MultiLingual AppsJBuilder Non-TechnicalJBuilder OpenToolsJBuilder Servlets (JSP)JBuilder StudentsJBuilder Team DevelopmentJBuilder Thirdparty ToolsJBuilder UpgradeJBuilder Installation InterBase Newsgroups----------------InterBaseInterBase Bug DiscussionInterBase GeneralInterBase IBConsoleInterBase OpenSourceInterBase ReplicationInterBase SQLInterBase ThirdpartyInterBase Installation Miscellaneous Borland Newsgroups----------------AppCenterAppServerEnteraVisiBroker  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.