[NNTP] 32 bit article counters - some history

[Clive D.W. Feather]

> Note that this is a long-standing point of discussion for this working
> group.  We first reached consensus on the current text about eight years
> ago; Clive was the one who drafted it originally.

True.

I've had a look back. My first wording was written on 1996-12-27 and
proposed a maximum of 999,999,999.

> However, this is the first time that I
> recall someone bringing up a configuration that files all articles into a
> single group.

Actually, to my slight amazement, *I* wrote on 1996-12-15:

| as a thought experiment,
| would the following conform to 977 and, if not, why not ?
[...]
|   - Articles are numbered uniquely across the entire newsbase (with an
|     article having the same number in every group it appears in).
|   - Articles are numbered sequentially as they arrive from 1.
|
| I see nothing stating that article numbers must be less than (say) 2^32.

64-bit systems were mentioned in the ensuing threads. For example, Brian
Kantor wrote:

| Since the article numbers are not transmitted in a binary form, I see
| no reason to artificially restrict the range of article numbers by
| specifying the size or length of the representation.  A specification
| that the numbers are DECIMAL and represent actual article counts should
| be sufficient.  (I.e., we should not force epochal resets of a newsgroup
| article count just because it won't fit in N digits.)

and I replied:

| We need to specify the representation used in the protocol (decimal unsigned,
| leading zeroes allowed).
|
| If there is no upper limit on article numbers, all client software needs to
| implement arbitrary-precision arithmetic "just in case". This is, to my
| mind, unreasonable, and so we should put an upper bound.
|
| All reasonable systems can handle unsigned integers of at least 32 bits
| (the ISO C Standard requires them to). Therefore 4 294 967 295 (2^32-1)
| seems a reasonable limit. [I know I said 999999999 before; I won't embarrass
| myself by repeating my reasoning.]

The ISO C Standard point is an interesting one. The current version of the
Standard requires 64 bit support, but is not yet widely rolled out.

I replied to Chris Caputo:

| > If Usenet (as an example ;-) continues to grow as it has, we may eat this
| > space pretty quickly in which case we either need to expand the range to
| > 64 bits and/or precisely define how to handle rollovers.
| 
| There's no existing practice for handling such rollovers. Here's some
| possibilities:
| 
| (1) Ignore it. If a server reaches the limit for a group, the server
| operator declares a flag day when article numbers in several groups will
| be reset.
| 
| (2) Assume that, by the time the problem is significant, we can write
| 977-ter (replacement for 977-bis) to say that 64 bits are allowed (I think
| it's too early for that).
| 
| (3) Define some mechanism for giving groups epoch numbers. I don't like
| this offhand - too much has to change.
| 
| (4) Make article numbers be modulo 2^32. Actually, this idea can be
| improved on. Divide the number space into 4 quadrants:
|     Q0 = 1         to 2^30-1
|     Q1 = 2^30      to 2^31-1
|     Q2 = 2^31      to 2^31+2^30-1
|     Q3 = 2^31+2^30 to 2^32-1
| Some time after the low water mark for a group enters Q3 and before the
| high water mark is reaching danger, the server should reset the group by
| subtracting 2^31 from all article numbers. If the client has a recorded HWM
| (or LWM) in Q3, and sees a GROUP response in Q1, it should assume this has
| happened and adjust its records in the same way. Obviously, the reset has
| to be atomic, but this could be done on Unix by:
|     - link each article to its new number
|     - change the active file
|     - wait for all clients using the old numbers to disconnect (or
|       forcibly disconnect them);
|     - remove the links with the old numbers.

Later on, Chris Caputo wrote:

| The preceeding two comments have really got me thinking we shouldn't place
| any limitations on decimal length or range of article numbers.  Why don't
| we just let the market decide how many decimal digits to use?  Since this
| is a text based protocol, it seems we shouldn't even worry about what
| range a 32-bit number will allow.  I certainly am thankful Brian and
| others didn't limit article numbers to 16-bits in RFC977, a bit length
| that might have been appropriate for the market when the rfc was drafted.
| We'd all be having flag days several times a year, if they did.  In this
| same light, let's not limit RFC977bis.  The market will decide when it is
| time to grow the range. 
| 
| I propose we change the draft to not place limitations on decimal length
| or range.

and the topic seems then to have died.

-- 
Clive D.W. Feather  | Work:  <clive at demon.net>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <clive at davros.org>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
Thus plc            |                            |