 |
BorlandTalk.com
Borland discussion newsgroups
|
| View previous topic :: View next topic |
| Author |
Message |
Charles Wright
Guest
|
 Posted: Wed Mar 03, 2004 12:08 am Post subject: Implementing a Server and Client for a custom data protocol |
 |
|
I am working on a client and server for a custom protocol that involves
small packets of mostly binary data.
The protocol actually defines that there should be two independent channels,
(each uni-directional).
The reason for the two uni-directional channels is that for each application
one channel concerned with receiving data,
and the other is for sending. The theory is that this makes it a more event
driven system. Since Indy is not event driven,
(at least using blocking), I am not sure this gains an advantage.
Does this make sense in terms of Indy components?
The packets of data represent small binary pieces of information
(telemetry), and depending on
the situation, one or more of these packets would need to be sent on a
fairly frequent basis from either side.
Should I just send the packets individually (which I can define as a
constant size), or should try to send them
as a collection? It seems this complicates the decoding of the collection,
since I will have to know how many packets
per collection, but perhaps it is more efficient in terms of use of TCP?
Are there any optimizations that I can do as a result of this?
Thanks in advance for any suggestions.
Charlie.
|
|
| Back to top |
|
 |
Zinvob
Guest
|
| Posted: Wed Mar 03, 2004 1:08 am Post subject: Re: Implementing a Server and Client for a custom data proto |
|
|
It's possible to have event communication both ways in Indy by using a
thread on the client connection like you do on the server. Then you can call
your event from this thread when receiving any data. Just remember that the
event is called from a thread, so you must be careful with threadsafe
related stuff.
Or you could use some other components based on nonblocking methods. The
best types are notification through WSA Events, or by using IO with
Completion Port.
The last one gives you most if your server is going to have a very heavy
load, if you don't have the extreme load you might want to use WSA Events as
it is less complex. You could use message events, but that's in my opinion a
crazy thing to do. But under some circumstances that may also be an
alternative.
Combining your "packets" is mostly necessary if they are connected in some
way that all of them needs to be transfered before you do something. If you
are adding compression to your transfer, combining packets will give you
more compression most of the time. You could fix your protocol to be able to
both combine protocol packets and to send them separatly by adding one
protocol command for your "combined" packet. Then you could handle this
protocol command before the others, splitting up the packets. This will make
it appear the same on the other side no matter how it's tranferred, and no
matter what extra commands and packet types you must add.
-Atle
|
|
| Back to top |
|
|
Remy Lebeau (TeamB)
Guest
|
| Posted: Wed Mar 03, 2004 1:50 am Post subject: Re: Implementing a Server and Client for a custom data proto |
|
|
"Charles Wright" <cwright (AT) dca (DOT) net> wrote
| Quote: | The protocol actually defines that there should be two
independent channels, (each uni-directional).
The reason for the two uni-directional channels is that
for each application one channel concerned with receiving
data, and the other is for sending.
|
Why can't you send and receive on the same channel? Sockets are
bi-directional, and can be read from and written to at the same time from
separate threads since each direction has its own separate buffering.
| Quote: | The theory is that this makes it a more event driven system.
Since Indy is not event driven, (at least using blocking), I
am not sure this gains an advantage.
|
All you have to do is perform your reading in a separate thread, and then
you can trigger your own events to your main code when data arrives. That
is exactly how Indy already implements such events as you have described
(yes, there are some asynchronous events in Indy's otherwise synchronous
design - mainly in servers).
| Quote: | The packets of data represent small binary pieces of
information (telemetry), and depending on the situation,
one or more of these packets would need to be sent on
a fairly frequent basis from either side.
|
The above mentioned approach will handle that just fine.
| Quote: | Should I just send the packets individually (which I
can define as a constant size), or should try to send
them as a collection?
|
Makes no difference at all. TCP is a byte stream, it has no concept of what
the data actually is. Only the sender and receiver know how to interpret
the data. Individual packet size does not matter at all, as long as the
receiver knows how to retreive the appropriate data size for each packet.
This is best accomplished by placing a fixed-size header at the beginning of
each packet, and then the header contains the size of the remaining data of
the packet. Thus, each packet can be dynamic in size based on the data that
it actually contains. The receiver reads the fixed-sized header first, and
then reads the amount of data specified by the header.
Gambit
|
|
| Back to top |
|
|
Charles Wright
Guest
|
| Posted: Wed Mar 03, 2004 2:59 am Post subject: Re: Implementing a Server and Client for a custom data proto |
|
|
"Remy Lebeau (TeamB)" <gambit47.no.spam (AT) no (DOT) spam.yahoo.com> wrote
| Quote: |
"Charles Wright" <cwright (AT) dca (DOT) net> wrote in message
news:4045221d (AT) newsgroups (DOT) borland.com...
The protocol actually defines that there should be two
independent channels, (each uni-directional).
The reason for the two uni-directional channels is that
for each application one channel concerned with receiving
data, and the other is for sending.
Why can't you send and receive on the same channel? Sockets are
bi-directional, and can be read from and written to at the same time from
separate threads since each direction has its own separate buffering.
|
From an Indy point of view that makes sense. I guess my question comes
from the following scenario.
It is expected that there will be one main server and a number of clients
maintaining
a constant connection. The client will send data to the server to update
information
about its state, which the server will then use to update the server's model
of that
client's state.
(That part I understand pretty well).
Depending on events from other clients, or the server internal modelling,
updates to the
client model will need to be sent back to the client.
I am guessing that the server will need to keep track of the relationship
between
the client model and its connection (thread?) to be able to direct
appropriate info
back to the client? My expectation is that the modeling will be done in
either the main
thread or other threads.
The client then (using a listener thread) would pick up the data and update
its model
based on the data.
| Quote: |
The theory is that this makes it a more event driven system.
Since Indy is not event driven, (at least using blocking), I
am not sure this gains an advantage.
All you have to do is perform your reading in a separate thread, and then
you can trigger your own events to your main code when data arrives. That
is exactly how Indy already implements such events as you have described
(yes, there are some asynchronous events in Indy's otherwise synchronous
design - mainly in servers).
This makes sense. |
| Quote: | This is best accomplished by placing a fixed-size header at the beginning
of
each packet, and then the header contains the size of the remaining data
of
the packet. Thus, each packet can be dynamic in size based on the data
that
it actually contains. The receiver reads the fixed-sized header first,
and
then reads the amount of data specified by the header.
Is there any disadvantage (other than a slight waste of bandwidth) to use a |
record with
a variant section? (The largest component element is 100 bytes). This
makes it easy to
decode the type of data that is coming across, and I can just use a
sizeof().
Thanks for the help,
Charlie
|
|
| Back to top |
|
|
Martin James
Guest
|
| Posted: Wed Mar 03, 2004 3:32 am Post subject: Re: Implementing a Server and Client for a custom data proto |
|
|
| Quote: | It is expected that there will be one main server and a number of clients
maintaining
a constant connection. The client will send data to the server to update
information
about its state, which the server will then use to update the server's
model
of that
client's state.
(That part I understand pretty well).
Depending on events from other clients, or the server internal modelling,
updates to the
client model will need to be sent back to the client.
I am guessing that the server will need to keep track of the relationship
between
the client model and its connection (thread?) to be able to direct
appropriate info
back to the client? My expectation is that the modeling will be done in
either the main
thread or other threads.
The client then (using a listener thread) would pick up the data and
update
its model
based on the data.
|
This all sounds perfectly normal. At the server, you can post off the
received data in an object, together with all other context stuff for the
client, eg. the peerThread instance, for processing in some other thread.
Indy has some Tnotify stuff for inter-thread comms, or you could use
postThreadMessage.
| Quote: | Is there any disadvantage (other than a slight waste of bandwidth) to use
a
record with
a variant section? (The largest component element is 100 bytes). This
makes it easy to
decode the type of data that is coming across, and I can just use a
sizeof().
|
Well, there's the disadvantage that you are forever stuck with fixed-length
records unless you rebuild both ends. Possibly, this just does not matter
in your app/protocol. You might want to add a simple checksum in your
record somewhere - just as a check that the record is framed correctly. The
frames should not get out of step with TCP, but it's nice to know that just
one extra byte that's added in by a dodgy router somewhere cannot corrupt
your database by grossly stuffing your variant parts/fields.
I'm sure you know this, but, just in case you hadn't realised - no long
strings or dynamic arrays in your record!
Rgds,
Martin
|
|
| Back to top |
|
|
Remy Lebeau (TeamB)
Guest
|
| Posted: Wed Mar 03, 2004 6:34 am Post subject: Re: Implementing a Server and Client for a custom data proto |
|
|
"Charles Wright" <cwright (AT) dca (DOT) net> wrote
| Quote: | Depending on events from other clients, or the server internal
modelling, updates to the client model will need to be sent back
to the client.
|
That is simple enough to handle. All you have to do is store the client's
socket connection with its associated model, so that when the model changes,
the exact socket associated with it can be written to as needed. As I
mentioned, you can read from and write to a connection at the same time from
different threads.
| Quote: | I am guessing that the server will need to keep track of
the relationship between the client model and its connection
(thread?) to be able to direct appropriate info back to the
client?
|
The client thread should maintain a pointer to its associated model (you can
use the TIdPeerThread.Data property for that) so that the client thread
known which model to update when needed. The model should maintain a
pointer back to the client thread's TIdTCPConnection so that it known which
connection to write to when needed. You can use the server's OnConnect even
to link the client thread to the model and vice versa, and then use the
OnDisconnect event to unlink them.
| Quote: | My expectation is that the modeling will be done in either
the main thread or other threads.
|
Makes no difference, as long as your code is thread-safe in general.
| Quote: | The client then (using a listener thread) would pick up
the data and update its model based on the data.
|
Such a thread already exists, via the TIdTCPServer and its TIdPeerThread
class. Simply do all of the reading in the server's OnExecute event, using
the thread's Data property to gain access to the model when needed.
| Quote: | Is there any disadvantage (other than a slight waste
of bandwidth) to use a record with a variant section?
|
Not really. It makes the communication very organized.
| Quote: | This makes it easy to decode the type of data that is
coming across, and I can just use a sizeof().
|
You could have the header, or the data itself, contain another value that
indicates the type of data it is. Otherwise, if your data packets are
always different sizes, you could just compare the header's specified data
size to your sizeof() value. I find the first approach is generally cleaner
and more flexible, though.
Gambit
|
|
| Back to top |
|
|
Remy Lebeau (TeamB)
Guest
|
| Posted: Wed Mar 03, 2004 6:35 am Post subject: Re: Implementing a Server and Client for a custom data proto |
|
|
"Martin James" <mjames_falcon (AT) dial (DOT) pipex.com> wrote
| Quote: | I'm sure you know this, but, just in case you hadn't
realised - no long strings or dynamic arrays in your record!
|
If you use a header with dynamically sized data, those are not an issue.
They can be handled quite easily.
Gambit
|
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
