BorlandTalk.com Borland discussion newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  ICS UDP: Receive buffer limitations?          BorlandTalk.com Forum Index -> Delphi Internet Winsock View previous topic :: View next topic   Author Message Stefan Paege Guest Posted: Mon Sep 15, 2003 1:31 pm    Post subject: ICS UDP: Receive buffer limitations? Hi, I'm using the ICS TWSocket component in UDP mode for quite some time now and with good success. It is used from within a DLL, which receives UDP datagrams and transports these datagrams to 3rd party software that uses our DLL. At one site the customer is reporting missing datagrams. I know about the possible problems with UDP in this regard, but on a LAN with only one segment, no routers, only 3 NICs, and low network load it should be pretty reliable. OK, all my tests did not show any problems, even with much higher network load. So I visited the problem site and saw that the 3rd party software that uses our DLL often eats up all available CPU cycles for some seconds. Their fault, not mine. But I thought this might be worth testing. I wrote a small test app that calls Sleep(60000) at the press of a button in order to simulate an app blocking for some time. During this blocking period I sent lots of UDP datagrams over the LAN. And as I expected after the blocking was over the data was shown in the testapp's memo. No problems with the buffering of the datagrams. So everything is fine? No, because I see missing datagrams when I sent more than 342 datagrams during the blocking period. If I do only the first 342 UDP packages are delivered later. As 342 is not a "magic number" I thought I'd take the size of the transferred data into account. Each package has 24 bytes of information. 342 x 24 = 8208, that is 8K (8192 byte) and 16 bytes remaining. So my question is: Is there any buffer in either TWSocket, Winsock or elsewhere in the chain that is limited to 8K or 342 list entries or whatever? Any ideas apprechiated. Thanks -- elektronik-labor Carls GmbH & Co. KG Stefan Paege Kontakt: +49 (0)5973 9497-23 Fax: +49 (0)5973 9497-19 Back to top Martin James Guest Posted: Mon Sep 15, 2003 1:47 pm    Post subject: Re: ICS UDP: Receive buffer limitations? "Stefan Paege" <paege (AT) el-carls (DOT) de> wrote Quote: Hi, I'm using the ICS TWSocket component in UDP mode for quite some time now and with good success. It is used from within a DLL, which receives UDP datagrams and transports these datagrams to 3rd party software that uses our DLL. At one site the customer is reporting missing datagrams. I know about the possible problems with UDP in this regard, but on a LAN with only one segment, no routers, only 3 NICs, and low network load it should be pretty reliable. OK, all my tests did not show any problems, even with much higher network load. So I visited the problem site and saw that the 3rd party software that uses our DLL often eats up all available CPU cycles for some seconds. Their fault, not mine. But I thought this might be worth testing. I wrote a small test app that calls Sleep(60000) at the press of a button in order to simulate an app blocking for some time. During this blocking period I sent lots of UDP datagrams over the LAN. And as I expected after the blocking was over the data was shown in the testapp's memo. No problems with the buffering of the datagrams. Quote: So my question is: Is there any buffer in either TWSocket, Winsock or elsewhere in the chain that is limited to 8K or 342 list entries or whatever? I would imagine so. 60 seconds is a*long* time to not-service-a-socket for. ICS probably has an 8k buffer. This is not an unreasonable size, FP will know more. 60 secs is too long to sieze up an app. If it's single-threaded, 100ms is too long, unless you're really into hourglasses. Besides, sleep(x) is not the same as using x CPU. Rgds, Martin Back to top Francois Piette Guest Posted: Mon Sep 15, 2003 1:55 pm    Post subject: Re: ICS UDP: Receive buffer limitations? Quote: So my question is: Is there any buffer in either TWSocket, Winsock or elsewhere in the chain that is limited to 8K or 342 list entries or whatever? There is a buffer in winsock. And there is an API to set the buffer size: setsockopt with SO_RCVBUF option. You can call this API function using TWSocket.Handle once the socket is opened. btw: whatever size you set for the buffer, you'll have some limit to the number of datagram you can receive. UDP is by design an unreliable protocol and you will at one time loose datagrams. Your application should be designed with that fact in mind. -- Contribute to the SSL Effort. Visit http://overbyte.delphicenter.com/eng/ssl.html -- [email]francois.piette (AT) overbyte (DOT) be[/email] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be "Stefan Paege" <paege (AT) el-carls (DOT) de> a écrit dans le message de news:3f65bf21$1 (AT) newsgroups (DOT) borland.com... Quote: Hi, I'm using the ICS TWSocket component in UDP mode for quite some time now and with good success. It is used from within a DLL, which receives UDP datagrams and transports these datagrams to 3rd party software that uses our DLL. At one site the customer is reporting missing datagrams. I know about the possible problems with UDP in this regard, but on a LAN with only one segment, no routers, only 3 NICs, and low network load it should be pretty reliable. OK, all my tests did not show any problems, even with much higher network load. So I visited the problem site and saw that the 3rd party software that uses our DLL often eats up all available CPU cycles for some seconds. Their fault, not mine. But I thought this might be worth testing. I wrote a small test app that calls Sleep(60000) at the press of a button in order to simulate an app blocking for some time. During this blocking period I sent lots of UDP datagrams over the LAN. And as I expected after the blocking was over the data was shown in the testapp's memo. No problems with the buffering of the datagrams. So everything is fine? No, because I see missing datagrams when I sent more than 342 datagrams during the blocking period. If I do only the first 342 UDP packages are delivered later. As 342 is not a "magic number" I thought I'd take the size of the transferred data into account. Each package has 24 bytes of information. 342 x 24 = 8208, that is 8K (8192 byte) and 16 bytes remaining. So my question is: Is there any buffer in either TWSocket, Winsock or elsewhere in the chain that is limited to 8K or 342 list entries or whatever? Any ideas apprechiated. Thanks -- elektronik-labor Carls GmbH & Co. KG Stefan Paege Kontakt: +49 (0)5973 9497-23 Fax: +49 (0)5973 9497-19 Back to top tcpip Guest Posted: Mon Sep 15, 2003 3:25 pm    Post subject: Re: ICS UDP: Receive buffer limitations? It's by UDP design. UDP do not guarantee delivery. Rgds Quote: So everything is fine? No, because I see missing datagrams when I sent more Back to top Stefan Paege Guest Posted: Mon Sep 15, 2003 4:01 pm    Post subject: Re: ICS UDP: Receive buffer limitations? Francois and Martin, thanks for your help. Some more ammunition I can shoot at the users of my DLL at the problem site ;-) -- elektronik-labor Carls GmbH & Co. KG Stefan Paege Kontakt: +49 (0)5973 9497-23 Fax: +49 (0)5973 9497-19 Back to top Martin James Guest Posted: Mon Sep 15, 2003 11:42 pm    Post subject: Re: ICS UDP: Receive buffer limitations? "tcpip" <donotspam (AT) me (DOT) com> wrote Quote: It's by UDP design. UDP do not guarantee delivery. Rgds So everything is fine? No, because I see missing datagrams when I sent more I suppose one of the reasons it doesn't guarantee delivery is no built-in flow control mechanims, unike TCP. I've never noticed any datagram loss on intranets but I'm not hammering it like your DLL comms. Rgds, Martin Back to top Francois Piette Guest Posted: Tue Sep 16, 2003 7:17 am    Post subject: Re: ICS UDP: Receive buffer limitations? Quote: It's by UDP design. UDP do not guarantee delivery. So everything is fine? No, because I see missing datagrams when I sent more I suppose one of the reasons it doesn't guarantee delivery is no built-in flow control mechanims, unike TCP. I've never noticed any datagram loss on intranets but I'm not hammering it like your DLL comms. You can easily loose datagram on an intranet (LAN) when the LAN is composed of different sped segments. For example a gigabit backbone and FastEthernet (100 Mbps) horizontal cabling. You can have a lot of datagrams comming from the gigabit side to the FastEthernet side. At some point, there are way too much datagrams for the switch or router which begin to drop packets. This never occur with TCP because of the ACK mechanism. And even if it occurs with TCP, it is automatically corrected and the application doesn't see anything. -- Contribute to the SSL Effort. Visit http://overbyte.delphicenter.com/eng/ssl.html -- [email]francois.piette (AT) overbyte (DOT) be[/email] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be Back to top Martin James Guest Posted: Tue Sep 16, 2003 12:40 pm    Post subject: Re: ICS UDP: Receive buffer limitations? "Francois Piette" <francois.piette (AT) overbyte (DOT) be> wrote Quote: It's by UDP design. UDP do not guarantee delivery. So everything is fine? No, because I see missing datagrams when I sent more I suppose one of the reasons it doesn't guarantee delivery is no built-in flow control mechanims, unike TCP. I've never noticed any datagram loss on intranets but I'm not hammering it like your DLL comms. You can easily loose datagram on an intranet (LAN) when the LAN is composed of different sped segments. For example a gigabit backbone and FastEthernet (100 Mbps) horizontal cabling. You can have a lot of datagrams comming from the gigabit side to the FastEthernet side. At some point, there are way too much datagrams for the switch or router which begin to drop packets. Hmm.. I'll remember that one. Luckily, I've never worked across a system like this. Thanks - good tip. This Quote: never occur with TCP because of the ACK mechanism. And even if it occurs with TCP, it is automatically corrected and the application doesn't see anything. Yes. I only explicitly use UDP for non-essential stuff. On the internet, about one in 30 datagrams seems to disappear. Rgds, Martin Back to top SocketSlave Guest Posted: Fri Sep 19, 2003 2:15 pm    Post subject: Re: ICS UDP: Receive buffer limitations? There is another typical problem with UDP that is datagrams can arrive in a different order to the order they were sent in. I think this happens when a datagram could take more that one route to the destination and some routes are inevitably quicker than others. Some datagrams take the quicker route and arrive before others, even if they were sent after. I guess this happens over the internet. However on a LAN that is not subdivided, has no routers I think this could never happen. Can anyone confirm this? "Martin James" <mjames_falcon (AT) dial (DOT) pipex.com> wrote Quote: "tcpip" <donotspam (AT) me (DOT) com> wrote in message news:3f65d9e6 (AT) newsgroups (DOT) borland.com... It's by UDP design. UDP do not guarantee delivery. Rgds So everything is fine? No, because I see missing datagrams when I sent more I suppose one of the reasons it doesn't guarantee delivery is no built-in flow control mechanims, unike TCP. I've never noticed any datagram loss on intranets but I'm not hammering it like your DLL comms. Rgds, Martin Back to top tcpip Guest Posted: Fri Sep 19, 2003 3:48 pm    Post subject: Re: ICS UDP: Receive buffer limitations? Hi Martin, Quote: There is another typical problem with UDP that is datagrams can arrive in a different order to the order they were sent in. .... sent after. I guess this happens over the internet. However on a LAN that is not subdivided, has no routers I think this could never happen. Can anyone confirm this? That happens even on the plane LAN. The network interface driver or intermediate switcher, hub, or router can change the UDP packets order to utilize the media bandwidth to fill the pipe more efficiently or simple by mistake in hash/queue algorithm. I had once some nasty problem with gigabit driver pulling from the kernel queue the smaller packets to fill the buffer hole before send to the wire over the one IRP call, - too smart. Rgds, Igor. Back to top SocketSlave Guest Posted: Fri Sep 19, 2003 4:13 pm    Post subject: Re: ICS UDP: Receive buffer limitations? Thanks I have been wondering about this for a while and never found out until now. BTW I'm not Martin, but perhaps he is reading. "tcpip" <donotspam (AT) me (DOT) com> wrote Quote: Hi Martin, There is another typical problem with UDP that is datagrams can arrive in a different order to the order they were sent in. ... sent after. I guess this happens over the internet. However on a LAN that is not subdivided, has no routers I think this could never happen. Can anyone confirm this? That happens even on the plane LAN. The network interface driver or intermediate switcher, hub, or router can change the UDP packets order to utilize the media bandwidth to fill the pipe more efficiently or simple by mistake in hash/queue algorithm. I had once some nasty problem with gigabit driver pulling from the kernel queue the smaller packets to fill the buffer hole before send to the wire over the one IRP call, - too smart. Rgds, Igor. Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         BorlandTalk.com Forum Index -> Delphi Internet Winsock All times are GMT Page 1 of 1   Jump to: Select a forum C++ Builder Newsgroups----------------C++ BuilderC++ Builder (ActiveX)C++ Builder (Announcements)C++ Builder (Command Line Tools)C++ Builder DatabasesC++ Builder Databases (ADO)C++ Builder Databases (Desktop)C++ Builder Databases (InterBase Express)C++ Builder Databases (Multi-Tier)C++ Builder Databases (SQL Servers)C++ Builder (Graphics)C++ Builder (IDE)C++ Builder (Internet)C++ Builder (Internet Socket)C++ Builder (Internet Web)C++ Builder (Jobs)C++ Builder (Language)C++ Builder (Language C++)C++ Builder (Compatibility with MFC)C++ Builder (Native API)C++ Builder (Non-Technical)C++ Builder (Students)C++ Builder (Third Party Tools)C++ Builder (Upgrade)C++ Builder (VCL)C++ Builder (VCL Components Usage)C++ Builder (VCL Components Development)C++ Builder (VCL Usage)C++ Builder (WinAPI)C++ Builder Installation Delphi Newsgroups----------------Delphi (General)Delphi ActiveX Controls UsageDelphi ActiveX Controls DevelopmentDelphi As400Delphi DatabasesDelphi Databases (ADO)Delphi Databases (dbExpress)Delphi Databases (Desktop)Delphi Databases (InterBase Express)Delphi Databases (Multi-Tier)Delphi Databases (SQL Servers)Delphi DeploymentDelphi GraphicsDelphi IDEDelphi InternationalizationDelphi InternetDelphi Internet Isapi-WebBrokerDelphi Internet WebSnapDelphi Internet WinsockDelphi JobsDelphi Language BASMDelphi Language ObjectPascalDelphi Model Driven Architecture (General)Delphi Model Driven Architecture (Third Party)Delphi MultimediaDelphi Native APIDelphi Non-TechnicalDelphi ObjectPascalDelphi OLE AutomationDelphi OO designDelphi OpenToolsAPIDelphi Reporting-ChartingDelphi RTLDelphi StudentsDelphi TeamSourceDelphi Thirdparty ToolsDelphi Thirdparty Tools (General)Delphi Thirdparty Tools (IntraWeb)Delphi Thirdparty Tools (RaveReports)Delphi UpgradeDelphi VCLDelphi VCL ComponentsDelphi VCL Components UsingDelphi VCL Components WritingDelphi WebServices SOAPDelphi WinAPIDelphi XMLDelphi Installationcomp.lang.pascal.delphi.components.misccomp.lang.pascal.delphi.components.usagecomp.lang.pascal.delphi.components.writingcomp.lang.pascal.delphi.databasescomp.lang.pascal.delphi.misccomp.sources.delphi JBuilder Newsgroups----------------JBuilderJBuilder AnnouncementsJBuilder Applet IssuesJBuilder As400JBuilder CompilerJBuilder CORBA/RMIJBuilder DatabasesJBuilder DB SwingJBuilder DebuggerJBuilder DeploymentJBuilder DocumentationJBuilder EnterpriseJBuilder HandheldJBuilder IDEJBuilder JavaJBuilder JavaAPIJBuilder Java LanguageJBuilder Java SwingJBuilder Java BeansJBuilder Java Beans UsageJBuilder Java Beans DevelopmentJBuilder JBCLJBuilder JDataStoreJBuilder JobsJBuilder LinuxJBuilder MacintoshJBuilder & MobileSetJBuilder MultiLingual AppsJBuilder Non-TechnicalJBuilder OpenToolsJBuilder Servlets (JSP)JBuilder StudentsJBuilder Team DevelopmentJBuilder Thirdparty ToolsJBuilder UpgradeJBuilder Installation InterBase Newsgroups----------------InterBaseInterBase Bug DiscussionInterBase GeneralInterBase IBConsoleInterBase OpenSourceInterBase ReplicationInterBase SQLInterBase ThirdpartyInterBase Installation Miscellaneous Borland Newsgroups----------------AppCenterAppServerEnteraVisiBroker  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.