[NNTP] LIST EXTENSIONS (again)

Russ Allbery rra at stanford.edu
Thu Nov 4 22:53:23 PST 2004


Okay, I've read and I think digested all of the long threads about the
NNTPv2 label and related issues.  I'm pleasantly surprised that everyone
agreed that we should have the protocol version advertised; I thought that
was going to be controversial!

Here is my take on this:

I think we need to be able to say what version of the base protocol
specification the server follows (everyone agrees on this).  I would
prefer to copy IMAP as closely as possible here because IMAP has done this
and we know that it works in practice, which means that unless we can see
a really significant problem with IMAP's approach, I don't think we should
think too hard.  It's possible that IMAP just hasn't encountered the
problems that could come up, but I think it's also equally possible that
IMAP's track record shows that the problems don't actually come up.

I don't think we need to know what version of an extension the server
supports because I don't expect extensions to have versions.  Note that no
documented standard command has ever changed its behavior in a significant
and non-backwards-compatible way over the life of the NNTP protocol; the
only things we've had to change were for commands like XOVER where
implementations diverged without a specification.  (Well, I guess that the
dropping of the optional distribution argument to NEWNEWS could be called
a non-backward-compatible change, but I think that's a stretch.)

If a command changes in a non-backward-compatible way, I think a new
command should be introduced instead and the old one left alone and
deprecated.  This is analogous to header fields.  New names are cheap;
backward compatibility problems aren't.  And even if we had a versioning
mechanism, I know there would be problems with older clients that didn't
support it quite right.  That's code that's likely to break since it's
never exercised in practice and won't be for years.

If a command changes but in a backward-compatible way, this is why we have
additional arguments to the extension advertisement.  Like with OVER
MSGID, we can define a way to advertise the new capability with more
specificity than is offered by a version number.

I'm comfortable setting the basic parameters of the extension in stone
once it's published, at least barring another major NNTP revision.  And I
agree with Ken that if it turns out we desperately need this, we can add
it later, with a bit of trouble.

I see what Clive is getting at by making the extension label "NNTP 2" or
the like instead of "NNTPv2", but I think it's better to go the route that
lets us advertise compatibility with multiple NNTP versions rather than
going the route that lets clients compare numbers.  Again, I'd rather
follow IMAP here.  I don't see a need for minor protocol revisions
indicating backward-compatible changes; I think that is subsumed by the
ability to advertise compatibility with multiple revisions (listing both
NNTPv2 and NNTPv3 extension labels, for instance).

I don't think using an RFC number instead of an arbitrary version number
is a good idea for the reasons that Ken stated.  There's too much chance
that RFCs may have significant errata, there may be multiple RFCs that
update each other, etc.  We're safer with an arbitrary version number like
v2.  (And again, let's follow IMAP.)

I do think that we want to adopt something out of the discussion that
Andrew and Mark had about avoiding round trips.  NNTP is too heavy on
round trips right now, and that's been bothering me when we list all the
places clients need to issue LIST EXTENSIONS again.  At the least, we need
to advertise whether MODE READER is required.  I think there are four
possible states the server can be in:  all commands supported; reader
commands not supported until after MODE READER; transit commands not
supported at all; and reader commands not supported at all.  I think the
best way to indicate those states would be with two extensions:

    NOREADER [MODE-READER]
    NOTRANSIT

(those may not be the best names).  NOTRANSIT is a negative extension
saying that IHAVE is not supported, which saves a client the need to try
it and get 502.  NOREADER is a negative extension saying that reader
commands (ARTICLE, BODY, POST, GROUP, LAST, NEXT, NEWGROUPS, and NEWNEWS)
are not available.  If the optional MODE-READER parameter is given, it
indicates that they will be available (possibly still requiring STARTTLS
and authentication) after issuing MODE READER.  Note that this label
doesn't need to imply anything about extensions, since that can be done
with the presence or absence of the extension labels themselves.

After Mark's messages, I've been convinced that we should mandate the
order of the state change commands.  The correct order as near as I can
tell would be:

    MODE READER
    STARTTLS
    AUTHINFO

(where you can skip the initial MODE READER if your initial LIST
EXTENSIONS doesn't have the NOREADER extension listed).  That still means
that a connection with TLS and authentication requires seven commands to
initiate in the worst case, though, which is rather obscene:

    LIST EXTENSIONS
    MODE READER
    LIST EXTENSIONS
    STARTTLS
    LIST EXTENSIONS
    AUTHINFO SASL
    LIST EXTENSIONS

Of course, nearly all servers won't require MODE READER, which cuts out
the second and third commands.

One could cut this down by one more command by adding some sort of initial
extension advertisement to the greeting, but this worries me for the
reasons stated by Clive and Andrew, just in a cleanliness sense.  It's
worth considering, though.

One can also gain a round trip with a different idea, namely having
successful successful AUTHINFO SASL (which is a new commands, so we can do
whatever we want with it) include LIST EXTENSIONS output in its response.
I'm not sure what I think of that; I'm guessing it's a bad idea since no
one else does it.

Assuming most servers don't have MODE READER, the normal case with TLS and
auth will be five round trips (plus however many SASL needs).  Adding
[STARTTLS] or some such thing to the welcome banner would make that four.
Assuming that sending extension information after AUTHINFO is a bad idea,
I don't see any way to get it lower than four -- two of the commands have
to be issued, the LIST EXTENSIONS after STARTTLS is required for security
reasons since you can't trust the results of it beforehand, and the LIST
EXTENSIONS after AUTHINFO is required if authenticating changes anything
about what you can do.  (Although I'm wondering how often authenticating
is really going to change anything other than noting that you can no
longer run STARTTLS or AUTHINFO again, and if clients would really need to
check there.)

Clive D W Feather <clive at demon.net> writes:
> Ken Murchison said:

>>> (0) Adopt the "restrict characters in command names and extension
>>> labels proposal".

>> If this is the proposal to allow upper/lower and numeric chars (with a
>> few extras), I agree.

> It is.

I agree with this as well.

> Approach B: the server lists all the RFCs to which it conforms. The client
> needs only know a single RFC (unless it has multiple code paths).
> - Following a case 1 change, the number does *not* change; the server
>   continues to reply "3977".
> - Following a case 2 change, the number does change; the server replies
>   "3977 4321". The client recognises 3977 and works, or it recognises
>   4321 and uses the additional features.
> - Following a case 3 change, the number does change; the server replies
>   "4321". The client needs to know that it can talk to 3977 but not 4321
>   servers, or it needs separate code paths contingent on the two.

This is equivalent to having NNTPv2, NNTPv3, etc. extension labels that
can all be advertised together if the revisions are compatible somehow.  I
think this is the best approach to take.

>> Has RFC3977 actually been reserved for the new NNTP spec (ala
>> RFC282[12]), or did you just pick this as an example?  I think
>> reserving 3977 it is a good idea, but we'd need to move fast if it
>> hasn't already been done, since the RFCs are up to at least 3955
>> already.

> I haven't done it. I'd very much like it to be done, but the RFC editors
> don't seem to be talking to me - they still haven't answered my comment
> about them mangling the last few drafts (as Ken, I think, first
> realized).

> Russ? Can you have a go at this?

I'll ask our AD.

>>> (3) A conforming server MUST NOT advertise any extension label, other
>>> than one beginning with X, not in the IANA registry.

>> Agreed.

> Note, I think that's stronger than what we had before.

Yes, but I think it's the right thing to do.

Optional LIST commands discussed in another message.  (Mark, to give you
the summary, I'm proposing making one of them mandatory and the other
three regular extensions with regular extension labels; the argument I
gave you before about them is sounding less persuasive to me.)

>> If we nail down capabilities for these two items, then a client can
>> learn everything it needs to from LIST EXTENSIONS and no longer has to
>> guess or do a trial-and-error.  This is a "good thing" IMHO.

> Agreed.

> Um, I'm not going to swear there's no other significant options that are
> worth noting here. Can anyone think of any?

Do we want to get into advertising whether commands are permitted for a
particular client?  (POST and NEWNEWS are the ones most frequently not
permitted.)

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>



More information about the ietf-nntp mailing list