ietf-nntp Standardization of LIST OVERVIEW.FMT

[Russ Allbery]

Hello everyone,

I'm writing this message as the co-chair of the NNTP working group.  We're
finishing up work on the next version of the NNTP standard, and one of the
few remaining issues involves LIST OVERVIEW.FMT.

If you are an implementor of either news reading or news serving software,
or knowledgeable about how some is implemented, it would be very helpful
if you could read this message and answer a few questions.  Feel free to
mail me directly with your responses and I'll post a summary.  Also,
anyone with an opinion is certainly invited to answer the final question
about what we should do.

For background, many news servers currently support a non-standard command
LIST OVERVIEW.FMT, which returns something like:

    Subject:
    From:
    Date:
    Message-ID:
    References:
    Bytes:
    Lines:
    Xref:full

where the first seven fields are fixed and then following are any
additional fields stored in the overview database by that server (and
returned by the XOVER command for articles).  The :full bit means that the
header name is included in the overview data, required for all headers
after the basic seven.

In practice, the result of this command generally reflects what will be
stored in the overview database for all newly arriving messages.  This is
generally configurable by the administrator of the server, and older
messages may have different headers stored in the overview database.

Part of the charter of the NNTP working group is to standardize overview
(in the process, standardizing an OVER command).  The open question is
whether LIST OVERVIEW.FMT should be standardized at the same time, and if
so, with what semantics.  The available options are:

 1. Do not standardize LIST OVERVIEW.FMT.  Servers can continue to
    implement it if they choose, but other standard commands won't use or
    rely on it, and implementation will likely become less common.  The
    only way to see what headers are included in overview information
    would be to send an OVER command and look at the output.

 2. Standardize LIST OVERVIEW.FMT, but specify that it reflects only the
    current configuration of the server (namely, what headers will be put
    into overview for newly arrived messages), and that any given article
    may have a completely different mix of headers in its overview data.
    This is the existing meaning of the command.  It's not clear whether
    this is actually useful to anyone for anything.

 3. Standardize LIST OVERVIEW.FMT but make it optional, and declare the
    meaning to be the list of headers included in the overview database
    for *all* articles stored on the server.  If the server cannot
    guarantee consistency in that fashion, it should not implement the
    command.  Note that while this makes it more likely that one can draw
    conclusions like "Content-Type is listed in LIST OVERVIEW.FMT, it's
    not included in the OVER information for this article, and therefore
    this article doesn't have a Content-Type header," they're still not
    certain.  It's possible that the overview database for the entire
    server will have changed between the issuing of LIST OVERVIEW.FMT and
    the issuing of the OVER command.

Now, for the questions:

[Q1] If you know how a reading client is implemented, does that reading
     client ever send LIST OVERVIEW.FMT?

[Q2] If the client does send LIST OVERVIEW.FMT, what does it use the
     returned data for?

[Q3] If you know how a server is implemented, does it implement LIST
     OVERVIEW.FMT?

[Q4] If the server does implement LIST OVERVIEW.FMT, what are its
     semantics?  Does it follow (2) above, (3) above, or some other
     meaning entirely?  And if the last, what meaning?

And finally:

[Q5] What approach do you think should be taken by the next NNTP standard?

Any and all feedback will be gratefully accepted.  Thank you!

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>