3 Aranea Orthogonal Communications Protocol
9 <Origin>,<TimeSeq>,<Hop>,<FrmUser>,<To>,<ToUser>|<Tag>,<Data>...
13 For many years DX Clusters have used a protocol which was designed
14 for a non-looped tree of nodes. This environment has probably never, reliably,
15 been achieved in practice; certainly not recently.
17 There have always been loops, sometimes bringing the network to its
18 knees. In modern usage, both in order to get some resilience and also
19 to expedite information flow, we use internet based, deliberately
20 looped networks with filtering. Whilst this works, after a fashion, there
21 are all sorts of problems that the current PC protocol can never
25 describes a complete replacement for the PC protocol. It allows a
26 fully looped network, is inherently extensible and should be simple
27 to implement (especially in perl).
29 All implementations of this protocol shall B<only> use this protocol
30 for inter-node communications.
35 designed to be an extensible basis for any type of one too many
36 "instant" line-based communications tasks.
38 This protocol is designed to be flood routed in a meshed network in
39 as efficient a manner as possible. The reason we have chosen this
40 mechanism is that most L</Messages> need to be broadcast to all nodes.
42 Experience has shown that nodes will appear and (more infrequently)
43 disappear without much (or any) notice.
44 Therefore, the constantly changing and uncoordinated
45 nature of the network doesn't lend itself to fixed routing policies.
47 Having said that: directed routing is available where routes have
48 been learned through past traffic.
49 Those L</Messages> that could be routed (mainly single line one to
50 one "talk" L</Messages>)
51 happen sufficiently infrequently that, should they need to be flood routed
52 (because no route has been learned yet) it is a small cost overall.
56 A message is a single line of UTF8 encoded and HTTP escaped text
57 terminated in the standard internet manner with a <CR><LF>.
59 Each message consists of a L</Routing Section> and a L</Command Section>.
60 The two sections are separated with the '|' character.
62 characters (as well as non-printable characters, <CR>, <LF> and
63 a small number of other reserved characters)
64 can only be sent escaped. This is described further in the
65 L</Command Section> and L</Fields>.
67 Most of this document is concerned with the L</Routing Section>, however
68 some L</Standard Commands> which all implementation should issue and
69 must accept are described.
73 In the past messaging applications such as DX Cluster software have maintained
74 a fairly strict division between "nodes" and "users". This protocol attempts
75 to get away from that distinction by allowing any entity to connect to any
78 Applications that use this protocol are essentially all peers and therefore
79 nodes the only real difference between a "node" and a "user" (using this
80 protocol) is that a "node" has one or more listeners running that will,
81 potentially, allow incoming connections. A "user" simply becomes an end
82 point that never uses the L</FrmUser> or L</ToUser> slots in the
85 The reason for this is that modern clients are more intelligent than simple
86 character based connections such as telnet or ax25. They wish to be able to
87 distinguish between the various classes of message, such as: DX spots,
88 announces, talk, logging info etc. It is a pain to have to do it, as now,
89 by trying to make sense of the (slightly different for each piece of node
90 software) human readable "user" version of the output. Far better to pass on
91 regular, specified, easily computer decodable versions of the message,
92 i.e. in this protocol, and leave
93 the human presentation to the client software.
95 Having said that, the protocol allows for traditional, character based,
96 connections, as in the past. But it is up to applications
97 to service and control that type of connection and to provide human readable
100 One of the legacy, character based connections that will probably have to be
101 serviced is that of existing PC protocol based nodes. They should be treated
102 as local clients, B<not> as peers in this protocol. It is likely that, in order
103 to do this, some extra L</Tag>s will need to be defined at application level.
105 =head1 Routing Section
107 The application that implements this protocol is essentially a line
108 oriented message router. One line equals one message. Each line is
109 effectively a datagram.
111 It is assumed that nodes are connected to
112 each other using a "reliable" streaming protocol such as TCP/IP or
113 AX25. Having said that: in context, L</Messages> in this protocol could be
114 multi/broadcast, either "as is" or wrapped in some other framing
117 Because this is an unreliable, best effort, "please route my packets
118 through your node" protocol, there is no guarantee that a message
119 will get to the other side of a mesh of nodes. There may be a
120 discontinuity either caused by outage or deliberate filtering.
122 However, as it is envisaged that most L</Messages> will be flood routed or,
123 in the case of directed L</Messages> (those that have L</To> and/or
124 L</ToUser> fields) down some/most/all interfaces showing a route for that
125 direction, it is unlikely that L</Messages> will be lost in practice.
127 =head2 Field Description
129 Only the first three fields in the L</Routing Section> are compulsory
130 and indicate that this is a broadcast to be sent to all nodes coming
131 from the L</Origin>. If the message needs to be identified as coming
132 from a user on a node, then the L</FrmUser> field is added.
134 Adding a L</To> and/or L</ToUser> field will restrict the destinations
135 or recipients that receive this message.
137 The L</Hop> field is incremented on receipt of a message on a node.
139 Fields are separated by the comma ',' character with the last field
140 required followed by the vertical bar '|' character.
142 If trailing fields are missed out then superfluous commas can also
143 be left out. If intervening fields are missing then no space needs
144 to be left for the separating comma.
146 The characters allowed in the routing section are restricted. Any
147 invalid characters in any field will cause the whole message to be
150 More detailed descriptions of the fields follow:
156 This is a compulsory field. It is the name of the originating node.
157 The field can contain up to 12 characters in the set [-A-Z0-9_] in
158 any order. Higher layers may restrict this further.
160 The field must not be changed by any other node.
164 This is a compulsory field. It is a 10 hexadecimal digit string which
165 consists of a day no (1-31),
166 a flag to indicate NTP syncronisation in use,
167 seconds within that day (0-86399) [total of 6 hex digits]
168 that are concatenated with a sequence number (0-65535)
169 [4 hex digits] making the total of 10 hexadecimal digits.
171 The date portion is constructed as:
173 my $date = ((((gmtime)[3] < 1) | $ntpflag) < 18) | (time % 86400);
175 The sequence number is simply an unsigned short (or 16 bit) number
178 Each message originated at this node will increment the sequence
183 This is a compulsory field. It is the number of hops from the
184 originating node. It is incremented immediately on receipt and
185 before determining its value.
187 So the originating node sends a message with a L</Hop> of 0, the
188 neighbouring nodes must increment this field before passing
189 it on to higher layers for onward processing.
191 Implementations may have an upper limit to this field and may
192 silently drop incoming L</Messages> with a L</Hop> count greater than the
197 This field is optional. It is the identifier of the originating
198 user. If it is missing then the message is
199 assumed to come from the originating node itself.
201 It can consist of up to 12 characters in the set [-A-Z0-9_]
202 in any order. Higher layers may restrict this further.
206 This field is optional. It is a string of up to 12 characters
207 in the set [-A-Z0-9_] in any order.
209 This field is used either to indicate particular node destination
210 or to differentiate this broadcast in some way by making this
211 message as a member of a L</Channel>. Any message can be sent
212 down any L</Channel>. The names of L</Channel>s and their usage
213 is entirely up to the implementor.
215 It is assumed that node names can be differentiated from user
216 names and L</Channel> names.
218 If the field is set to a particular node destination, it will
219 be routed (rather than broadcast) to that node. However, any
220 intervening nodes are free to duplicate the message and send
221 it down more than one, likely looking, interface - depending on any
222 network policies that may pertain.
226 This field is optional. It is a string of up to 12 characters
227 in the set [-A-Z0-9_] in any order. Higher layers may restrict
230 Conventionally this field is used to indicate the user to whom
231 this message is directed. In an ideal world the L</To> field
232 will be set, by the originating node, to the identifier of the node
233 on which this user resides.
235 If the L</To> field is not set then this message will be
236 broadcast. However, should a node become apparent (on route)
237 then nodes are free to fill in the L</To> field and proceed
238 with a more directed approach.
240 If it becomes apparent (on route) that there may be more than
241 one possible L</To> destination for a L</ToUser> then a node
242 may duplicate the message (keeping the same L</TimeSeq>) and
243 route it onwards. Because of the L</DeDuplication> inherent in
244 the system, it is indeterminate as to which destination will
245 receive the message. It is possible for all or just some
246 destinations to receive the message. The tuple (L</Origin>,
247 L</TimeSeq>) will determine uniqueness.
249 This field can, in the case where L</To>
250 is set to the name of a node, be set to a L</Channel>. If this
251 is the case then this will cause this message to be sent to
252 a L</Channel> on the L</To> node only.
258 Channels are a concept very similar to that on IRC. It is a
259 way of segregating data flows in a network. In principle, subject
260 to local policy or application requirements, any data (or
261 L</Command Section>) can be sent down any channel.
263 It is up to the implementation whether to use this feature or not.
267 It is assumed that nodes will be connected in a looped network with
268 more than one route available (in many cases) to another node.
270 In anycase, most traffic is not directed, but broadcast to all users
273 Each message is uniquely identified by the (L</Origin>,L</TimeSeq>)
274 tuple. The basic system will learn which interfaces can see what nodes
275 by looking at the tuple and merging that with the L</Hop> count.
276 Each interface remembers the latest L</TimeSeq> with the lowest L</Hop>
277 for each L</Origin> that arrives on that interface. It also remembers
278 the number of L</Messages> for that L</Origin> that has been received on
281 Any message for onward broadcast is duplicated and sent out on all
282 interfaces that it did not come in on.
284 Any message that is directed to a particular node will be sent out on
285 the "best" interface based on routing information gathered so far. If there
286 is more than one possible route then, depending on network or local
287 policy, the message may be duplicated and sent on other interfaces
292 On receipt of a message, its unique tuple (L</Origin>,L</TimeSeq>) is
293 checked against a hash table. If it exists: the message is silently
294 dropped. If it does not exist in the hash table then the tuple is
297 The hash table is periodically cleaned, removing tuples that
298 have expired. The length of time a tuple remains in the hash table
299 is implementation dependant but could easily be several days, if
302 This mechanism only ensures that a message broadcast around the network
303 travels the least distance and through the fewest nodes possible. It
304 is up to higher layers to make sure that data carried is not, itself,
309 # on link startup from GB7BAA (both sides hello)
310 GB7TLH,3D02350001,0|HELLO,Aranea,1.2,24.123
311 GB7BAA,3D02355421,1|HELLO,Aranea,1.1,23.245
313 # on user startup to GB7TLH
314 GB7TLH,3D042506F2,0,G1TLH|HELLO,PClient,1.3
316 # on user disconnection
317 GB7TLH,3D9534F32D,0,G1TLH|BYE
319 # a talk (actually 'text') message to a user (some distance away
320 # from the origin node)
321 GB7TLH,3D03450019,3,G1TLH,GB7BAA,G8TIC|T,Hiya Mike what's happening?
323 # a talk/chat/text message to a channel or group
324 GB7TLH,0413525F23,2,G1TLH,VHF|T,2m is opening on MS
326 # a ping to find the whereabouts and distance of a user from a node
327 # the hex number on the end is the ping ID
328 GB7TLH,1512346543,0,,,G7BRN|PING,9F4D
330 # the same from a user on GB7TLH
331 GB7TLH,1512346543,0,G1TLH,,G7BRN|PING,23
333 # this effectively asks whether the user is on-line on a particular node
334 GB7TLH,1512346543,0,G1TLH,GB7DJK,G7BRN|PING,35DE
336 # A possible reply, same ID as ping followed by the no of hops on the
337 # ping that was received
338 GB7DJK,1512450534,3,G7BRN,GB7TLH,G1TLH|PONG,35DE,3
341 =head1 Command Section
343 The L</Command Section> of the message contains the actual data being
344 passed. It is called the Command Section because all commands
345 are identified with a L</Tag> each of which is implemented by
346 the software using this protocol. Each </Tag> (usually) is followed by one
351 The L</Tag> consists of string of uppercase letters and digits, starting
352 with a leading, uppercase, letter. Tags should be as short as is meaningful.
360 Invalid tags include:
366 The L</Tag> is separated from its data L</Fields> by a comma ','.
371 in any subsequent data shall be separated by a comma ','.
373 be HTTP encoded such that reserved characters (comma ',',
377 and non printable characters less than 127 (or %7F in hex)
378 [including newline and carraige return] are translated to
379 their two hex digit equivalent preceeded by the percent '%' character.
383 "%0D%0A" is "<carriage return><linefeed>".
384 "hello%2C there" is "hello, there"
386 This is not standard CSV, fields are not quoted (delimited with either
389 All national characters above 127 are UTF8 encoded in the
390 standard perl 5.8.x way. It follows that all (perl) programs that
391 are written according to this specification must say:
395 A message (or line) is terminated with <carriage return><linefeed>
396 0x0d 0x0a. Incoming L</Messages> must be accepted even when terminated
397 with just <linefeed>.
399 Care must be taken to make sure that fields have any reserved characters
400 encoded. In particular: it is perfectly permissible to have <linefeed>
401 characters in a field - so long as they are escaped.
403 Fields come in two styles: either simple fields (just containing
404 data) or B<key>=B<value> pairs. Each pair must be separated from
405 the next by a comma ','. The B<key> must consist of the set of
406 characters [a-z0-9_] (ie lowercase letters, digits and underscore),
407 with a leading letter. The B<value> must be HTTP encoded as
408 specified above and can otherwise contain any character.
410 There is no maximum size specified for a message. It is up to each
411 implimentation to enforce one (if only for their own protection).
413 =head2 Standard Commands
415 There are a number of L</Standard Commands> which must be accepted by
422 HELLO,<software name>,<version>,<build>,<comments>
424 Command sent on connection to another node. Both sides send their information
425 to the other. All the possible arguments are optional, although some of the
426 arguments should be sent in order to help diagnose problems. This command is
433 Command sent to all connections when the software is shutting down. This is sent
434 by the node just before shutdown occurs. This is really only used to help the
435 network prune its routing tables. It isn't a requirement. The <comment> field
440 DISC,<node name>,<comments>
442 Command sent when a node has disconnected from this node. This message is sent when
443 an interface shuts down. It need not be sent if a L<BYE> from an interface for
444 that node has just been received. This command should be broadcast.
446 The <node name> is mandatory and is the name of the interface that has just
453 Command to send a ping to a node or user. This command is used both by the software
454 and users to determine a) whether a node or user exists and b) how good the path is
457 The <ping id> is a unique string which is usually the hexadecimal equivalent of an
458 integer that is incremented every time it is used. But it can be anything that
459 will identify this ping using the tuple (L<Origin>,<ping id>) as unique.
463 PONG,<ping id>,<no of hops on ping>
465 Command to reply to a ping. This is sent as a reply to an incoming ping command.
466 The <ping id> is the one supplied and the <no of hops on ping> is the number of
467 hops it took for the ping to arrive.
473 All implementations must be able to send "text" (encoded as specified in
474 L</Fields>). There would be little point in doing all this otherwise!
480 Dirk Koopman, G1TLH, E<lt>djk@tobit.co.ukE<gt>
482 =head1 COPYRIGHT AND LICENSE
484 Copyright 2004 by Dirk Koopman, G1TLH
486 This library is free software; you can redistribute it and/or modify
487 it under the same terms as Perl itself.