[NNTP] NNTP working group status

[Russ Allbery]

We've been doing a lot of brainstorming here for a while, and I've been
letting it run because I think we've been getting a lot of productive work
done, but I'm overdue for sending out some sort of status message saying
where we're at in terms of the overall goals of the working group.

I'm also overdue on a status update to Scott on where we are, so this
message will double as that.

First of all, where we are formally:

 * The base protocol document (version -23, I believe) passed IETF last
   call and has now passed IESG approval.  If we believed it was the right
   thing to do, it's actually at the stage where we would send it to the
   RFC editor for publication.  I don't think that's the right thing to
   do (see below), but I wanted everyone to know what the actual status of
   it is.

 * We have three other documents in progress.  The AUTHINFO and STARTTLS
   documents are, I believe, done changing, except for the outcome of the
   current discussions (more on that in a moment).  Except for the whole
   LIST EXTENSIONS issue that we're debating now, they would be ready for
   working group and then IETF Last Call.

 * I believe that the STREAMING document is essentially done, although
   since it hasn't baked as much as the other two, it's more likely to
   turn up some additional comments in a working group last call.

 * I think it is to our advantage to publish all four of these documents
   around the same time, since together they comprise pretty much all of
   the significant and useful extensions that are currently in use by
   deployed NNTP clients and servers.

 * We are significantly overdue on all of our outstanding milestones,
   many of which have already been moved multiple times.

Now, here's why we're not currently just pushing forward with publishing
the work we've already done, which was fully my intention as of a month
ago:

Due to a wide host of reasons, including massively extended working group
life, the tendency for NNTP to work in a vacuum anyway, and lack of broad
working group participation, we've been working in a bit of a vacuum for a
while.  I was pretty much resigned to this, and resigned to publishing
something that not everyone would like, on the grounds that it was better
to have *some* published standard than basically nothing (which is what we
have right now).  In the course of reviewing a detail of the AUTHINFO
draft, however, Mark Crispin made several excellent comments about the
NNTP draft that seemed worth considering.

The issues that we reopened discussion on in the mailing list were:

 * There are some completely broken implementations of LIST EXTENSIONS and
   some implementations that don't list extensions that they actually
   provide.  From a client perspective, it would be nice to have a clear
   indication of whether LIST EXTENSIONS can be trusted.  IMAP solved a
   similar problem by introducing a capability that stated the protocol
   version number, and it was proposed that NNTP do the same (adding an
   NNTPv2 extension, or something similar).

 * The NNTP specification still allowed for optional commands, where one
   wouldn't know if a command was available without trying to execute it.
   This is not the model that other Internet protocols with extension
   mechanisms use, so it was proposed that we go back and add clearly
   advertised extensions for all of the optional commands, indicating
   whether they were available.

 * There was no indication whether a client had to issue MODE READER at
   the beginning of a session, and it's normally not needed.  It was
   proposed that we add some extension indicating whether MODE READER is
   required, since normally it won't be.

In the process of that discussion, we've also talked about other things (a
more expressive alternative to LIST EXTENSIONS, whether we can eliminate
MODE READER entirely, whether we can provide a short list of extensions as
part of the opening banner in order to reduce round trips for clients).

Now, I view the role of a working group chair to be somewhat similar to
the role of a release manager, in that I have to try to balance the
competing priorities of making the standard as good as we can and getting
something out the door.  While we have a lot of energy and activity right
now, I mistrust (based on past experience) our ability to maintain that
energy level for very long, and therefore think we need to get documents
out the door before we lose energy again.  We have to strike a balance
between getting the documents just right and getting them published so
that people can actually start benefiting from them.

I do think that if we can get an extension mechanism that would make Mark
happy, that is a significant improvement to the document and is worth
delaying it for another three months (which is probably going to be the
minimum delay if we go back, change it in a fundamental way, and put it
forward again for Last Call, which I assume would be the procedure).
Mark's points about nailing down exactly what the server will let one do
in the LIST EXTENSIONS response are very well-taken, as is the point about
adding some sort of protocol token to avoid known-bad implementations.

Accordingly, I think it's right to ask Scott if he also feels like we
should reopen discussion on LIST EXTENSIONS (more formally than we have so
far) and make modifications to the standard in order to meet the above
three goals.  This would likely also result in some changes to the LIST
command to try to resolve the issues with the current draft where some of
the commands are optional and there's no advertised way of knowing whether
they will work.  It will also probably mean dropping LIST DISTRIBUTIONS
from the standard entirely and leaving it as a separate extension that we
don't document in this initial set of documents, and making LIST
NEWSGROUPS mandatory.

However, I think that if we're going to go back and revisit things we've
already decided, even for very good reasons, we need to sharply limit the
scope of that discussion so that we can complete it and move forward.
This means that we're not going to fix everything that could be fixed, and
it means that the next version of the standard is likely to require more
work, but it does mean that we'll get the standard out the door faster so
that people can start using it.

Accordingly, I would strongly prefer to reopen discussion *only* on LIST
EXTENSIONS and what should be contained in it and on the LIST command and
what LIST subcommands we choose to document (so that we can fix up the
issues related to optional commands there).  This means leaving MODE
READER alone for now, which includes the following admonition:

   Servers are encouraged to not require this command even though clients
   SHOULD send it when appropriate.  It is present to support some news
   architectures that switch between modes based on whether a given
   connection is a peer-to-peer connection with another server or a news
   reading client.

but not removing the command from the standard entirely.  I do think we
can document 433 as an alternate port for NNTP servers that can be used
only by peers, which is one way in which some servers have avoided MODE
READER, but I don't really want to have an extended discussion about that
either.

Now, in the interests of full disclosure, please note that I maintain a
free news server (INN) which is essentially the only software package that
requires MODE READER at present.  I didn't add it, don't particularly like
it, and configure my own servers to not require it, but there is certainly
the possibility of bias in my own feelings on the subject.  I'm
uncomfortable making a working group chair decision on this because of
that, and I want to make sure Scott knows about that and also say that I'm
quite happy for some third party to make the decision about whether to
reopen discussion of or remove MODE READER from the NNTP standard, and
I'll happily abide by the decision of that person.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>