BorlandTalk.com Borland discussion newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  Slow Goto page 1, 2, 3  Next          BorlandTalk.com Forum Index -> C++ Builder (Language C++) View previous topic :: View next topic   Author Message Cenk Guest Posted: Fri Feb 24, 2006 3:03 pm    Post subject: Slow hi, i have app. 30 billion data in sql server db and i write them into a txt file.But its too time consuming, is there a faster way to do this? thanks Back to top Jonathan Benedicto Guest Posted: Fri Feb 24, 2006 3:03 pm    Post subject: Re: Slow Cenk wrote: Quote: i have app. 30 billion data in sql server db and i write them into a txt file.But its too time consuming, is there a faster way to do this? Can we see your code ? Jonathan Back to top Cenk Guest Posted: Fri Feb 24, 2006 3:03 pm    Post subject: Re: Slow String tmp1,tmp2,tmp3,tmp4,tmp5,tmp6,tmp7,tmp8,tmp9; int count; TQuery *CountQuery; CountQuery= new TQuery (this); CountQuery->DatabaseName = "Raw"; CountQuery->SQL->Add ("select COUNT(bid) from [EUR/USD]"); CountQuery->Open (); count = CountQuery->FieldByName ("COLUMN1")->AsInteger; CountQuery->SQL->Clear(); CountQuery->SQL->Add ("select [fTime],[fID],CAST([fName] As varchar(12)) As fName,[bid],[ask],[high],[low],[bidTrend],[askTrend] from [EUR/USD]"); CountQuery->Open (); ofstream writefile ("eurusdfull.txt",ios::app); if (writefile.is_open()) { for (int i = 0;i<count ;i++) { tmp1 = IntToStr(CountQuery->FieldByName ("fID")->AsInteger); tmp2 = CountQuery->FieldByName ("fName")->AsString; tmp3 = FloatToStr(CountQuery->FieldByName ("bid")->AsFloat); tmp4 = FloatToStr(CountQuery->FieldByName ("ask")->AsFloat); tmp5 = FloatToStr(CountQuery->FieldByName ("high")->AsFloat); tmp6 = FloatToStr(CountQuery->FieldByName ("low")->AsFloat); tmp7 = FloatToStr(CountQuery->FieldByName ("bidTrend")->AsInteger); tmp8 = FloatToStr(CountQuery->FieldByName ("askTrend")->AsInteger); tmp9 = DateTimeToStr(CountQuery->FieldByName ("fTime")->AsDateTime); writefile << tmp9.c_str()<<"\t"<<tmp1.c_str()<<"\t"<<tmp2.c_str()<<"\t"<<tmp3.c_str()<<"\t"<<tmp4.c_str()<<"\t"<<tmp5.c_str()<<"\t"<<tmp6.c_str()<<"\t"<<tmp7.c_str()<<"\t"<<tmp8.c_str()<<"\n"; CountQuery->Next (); } writefile.close(); CountQuery->Close (); CountQuery->Free (); } else cout << "Unable to open file"; Back to top Cenk Guest Posted: Fri Feb 24, 2006 4:03 pm    Post subject: Re: Slow this is the original code, actually i m trying to find a better one.its 34 mins and still continuing. Back to top Alan Bellingham Guest Posted: Fri Feb 24, 2006 4:03 pm    Post subject: Re: Slow "Cenk" <cenk1536 (AT) yahoo (DOT) com> wrote: Quote: file size is 2,5 GB and row size is app 34 billions You've got 13 rows per byte of database?! Alan Bellingham -- ACCU Conference 2006 - 19-22 April, Randolph Hotel, Oxford, UK Back to top Jonathan Benedicto Guest Posted: Fri Feb 24, 2006 4:03 pm    Post subject: Re: Slow Cenk wrote: Quote: this is the original code, actually i m trying to find a better one.its 34 mins and still continuing. I pasted code below my post with the file writing stripped out of your code. How fast does that run ? Jonathan Back to top John Blackburn Guest Posted: Fri Feb 24, 2006 4:03 pm    Post subject: Re: Slow "Cenk" <cenk1536 (AT) yahoo (DOT) com> wrote in message news:43ff226c$1 (AT) newsgroups (DOT) borland.com... Quote: i m talking about 34,000,000 of data! bytes or records ? Back to top Jonathan Benedicto Guest Posted: Fri Feb 24, 2006 4:03 pm    Post subject: Re: Slow Alan Bellingham wrote: Quote: You've got 13 rows per byte of database?! Thats excellent compression, I'd better get that code and patent it :-) Jonathan Back to top Cenk Guest Posted: Fri Feb 24, 2006 4:03 pm    Post subject: Re: Slow Quote: You've got 13 rows per byte of database?! So What?? Back to top Cenk Guest Posted: Fri Feb 24, 2006 4:04 pm    Post subject: Re: Slow i need solution not laughing Back to top Jonathan Benedicto Guest Posted: Fri Feb 24, 2006 4:04 pm    Post subject: Re: Slow Cenk wrote: Quote: well i could not try your code because mine is still going & going! Ctrl+Alt+Delete and kill it :-) Jonathan Back to top Cenk Guest Posted: Fri Feb 24, 2006 4:04 pm    Post subject: Re: Slow i m talking about 34,000,000 of data! Back to top Jonathan Benedicto Guest Posted: Fri Feb 24, 2006 4:04 pm    Post subject: Re: Slow Cenk wrote: How fast does this code run compared to the original code: ? String tmp1,tmp2,tmp3,tmp4,tmp5,tmp6,tmp7,tmp8,tmp9; int count; TQuery *CountQuery; CountQuery= new TQuery (this); CountQuery->DatabaseName = "Raw"; CountQuery->SQL->Add ("select COUNT(bid) from [EUR/USD]"); CountQuery->Open (); count = CountQuery->FieldByName ("COLUMN1")->AsInteger; CountQuery->SQL->Clear(); CountQuery->SQL->Add ("select [fTime],[fID],CAST([fName] As varchar(12)) As fName,[bid],[ask],[high],[low],[bidTrend],[askTrend] from [EUR/USD]"); CountQuery->Open (); for ( int i = 0; i < count; ++i ) { tmp1 = IntToStr(CountQuery->FieldByName ("fID")->AsInteger); tmp2 = CountQuery->FieldByName ("fName")->AsString; tmp3 = FloatToStr(CountQuery->FieldByName ("bid")->AsFloat); tmp4 = FloatToStr(CountQuery->FieldByName ("ask")->AsFloat); tmp5 = FloatToStr(CountQuery->FieldByName ("high")->AsFloat); tmp6 = FloatToStr(CountQuery->FieldByName ("low")->AsFloat); tmp7 = FloatToStr(CountQuery->FieldByName ("bidTrend")->AsInteger); tmp8 = FloatToStr(CountQuery->FieldByName ("askTrend")->AsInteger); tmp9 = DateTimeToStr(CountQuery->FieldByName ("fTime")->AsDateTime); CountQuery->Next (); } CountQuery->Close (); CountQuery->Free (); Jonathan Back to top Cenk Guest Posted: Fri Feb 24, 2006 4:04 pm    Post subject: Re: Slow file size is 2,5 GB and row size is app 34 billions Back to top John Blackburn Guest Posted: Fri Feb 24, 2006 4:04 pm    Post subject: Re: Slow "Cenk" <cenk1536 (AT) yahoo (DOT) com> wrote in message news:43ff1e32$1 (AT) newsgroups (DOT) borland.com... Quote: this is the original code, actually i m trying to find a better one.its 34 mins and still continuing. by "30 billion data", do you mean 30 billion records or 30 GByte database file size and, if the latter, how many records are we talking about ? Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         BorlandTalk.com Forum Index -> C++ Builder (Language C++) All times are GMT Goto page 1, 2, 3  Next Page 1 of 3   Jump to: Select a forum C++ Builder Newsgroups----------------C++ BuilderC++ Builder (ActiveX)C++ Builder (Announcements)C++ Builder (Command Line Tools)C++ Builder DatabasesC++ Builder Databases (ADO)C++ Builder Databases (Desktop)C++ Builder Databases (InterBase Express)C++ Builder Databases (Multi-Tier)C++ Builder Databases (SQL Servers)C++ Builder (Graphics)C++ Builder (IDE)C++ Builder (Internet)C++ Builder (Internet Socket)C++ Builder (Internet Web)C++ Builder (Jobs)C++ Builder (Language)C++ Builder (Language C++)C++ Builder (Compatibility with MFC)C++ Builder (Native API)C++ Builder (Non-Technical)C++ Builder (Students)C++ Builder (Third Party Tools)C++ Builder (Upgrade)C++ Builder (VCL)C++ Builder (VCL Components Usage)C++ Builder (VCL Components Development)C++ Builder (VCL Usage)C++ Builder (WinAPI)C++ Builder Installation Delphi Newsgroups----------------Delphi (General)Delphi ActiveX Controls UsageDelphi ActiveX Controls DevelopmentDelphi As400Delphi DatabasesDelphi Databases (ADO)Delphi Databases (dbExpress)Delphi Databases (Desktop)Delphi Databases (InterBase Express)Delphi Databases (Multi-Tier)Delphi Databases (SQL Servers)Delphi DeploymentDelphi GraphicsDelphi IDEDelphi InternationalizationDelphi InternetDelphi Internet Isapi-WebBrokerDelphi Internet WebSnapDelphi Internet WinsockDelphi JobsDelphi Language BASMDelphi Language ObjectPascalDelphi Model Driven Architecture (General)Delphi Model Driven Architecture (Third Party)Delphi MultimediaDelphi Native APIDelphi Non-TechnicalDelphi ObjectPascalDelphi OLE AutomationDelphi OO designDelphi OpenToolsAPIDelphi Reporting-ChartingDelphi RTLDelphi StudentsDelphi TeamSourceDelphi Thirdparty ToolsDelphi Thirdparty Tools (General)Delphi Thirdparty Tools (IntraWeb)Delphi Thirdparty Tools (RaveReports)Delphi UpgradeDelphi VCLDelphi VCL ComponentsDelphi VCL Components UsingDelphi VCL Components WritingDelphi WebServices SOAPDelphi WinAPIDelphi XMLDelphi Installationcomp.lang.pascal.delphi.components.misccomp.lang.pascal.delphi.components.usagecomp.lang.pascal.delphi.components.writingcomp.lang.pascal.delphi.databasescomp.lang.pascal.delphi.misccomp.sources.delphi JBuilder Newsgroups----------------JBuilderJBuilder AnnouncementsJBuilder Applet IssuesJBuilder As400JBuilder CompilerJBuilder CORBA/RMIJBuilder DatabasesJBuilder DB SwingJBuilder DebuggerJBuilder DeploymentJBuilder DocumentationJBuilder EnterpriseJBuilder HandheldJBuilder IDEJBuilder JavaJBuilder JavaAPIJBuilder Java LanguageJBuilder Java SwingJBuilder Java BeansJBuilder Java Beans UsageJBuilder Java Beans DevelopmentJBuilder JBCLJBuilder JDataStoreJBuilder JobsJBuilder LinuxJBuilder MacintoshJBuilder & MobileSetJBuilder MultiLingual AppsJBuilder Non-TechnicalJBuilder OpenToolsJBuilder Servlets (JSP)JBuilder StudentsJBuilder Team DevelopmentJBuilder Thirdparty ToolsJBuilder UpgradeJBuilder Installation InterBase Newsgroups----------------InterBaseInterBase Bug DiscussionInterBase GeneralInterBase IBConsoleInterBase OpenSourceInterBase ReplicationInterBase SQLInterBase ThirdpartyInterBase Installation Miscellaneous Borland Newsgroups----------------AppCenterAppServerEnteraVisiBroker  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.