ietf-nntp Summary/analysis of LIST OVERVIEW.FMT responses

[Russ Allbery]

Below are the detailed results from my previous questions about what to do
about LIST OVERVIEW.FMT.  First, here's the summary and my personal
analysis.

The first thing to notice is that while support for LIST OVERVIEW.FMT is
near universal among servers (only one of the eight servers about which I
got information does not support it), most clients don't use it.  Only one
third of the twelve news readers about which I got information send the
command at all, and one of the ones that does just stores it for debugging
purposes.

Another interesting thing to note is that INN was the only server about
which I got information that didn't just hard-code a particular overview.
(Well, Supernews is a slightly different case, but their configuration is
that of the standard 8-element overview and they're the only people
running that software, so that's functionally equivalent.)  Note the large
omission of details for Typhoon or Diablo here, however; I didn't get any
responses from anyone who knew what they did with LIST OVERVIEW.FMT.

This seems to indicate that in practice, the results of LIST OVERVIEW.FMT
are generally consistent with the entire overview database because what
goes into the overview database isn't changed.

Of the client usages, I think the most interesting cases are tin and
XanaNews.  XanaNews uses the return code of LIST OVERVIEW.FMT to determine
whether XOVER is supported, essentially as a LIST EXTENSIONS replacement.
This means there would be no need to standardize it for that purpose, but
that servers supporting XOVER should probably continue to support LIST
OVERVIEW.FMT.

tin, on the other hand, actually uses LIST OVERVIEW.FMT for the sort of
operation that it was intended to be used for, namely to determine whether
Xref:full is included in overview.  This is also the sort of application
that would continue to be needed unless we were going to require that Xref
always be included in overview, which I'm mildly inclined against.

When it comes to the opinions about what to do about the command, about
50% favor not standardizing it one way or another (several of whom feel
that overview should just be standardized as the usual eight headers).
About 25% think that it should be standardized, and the majority of those
think that the server overview database should be required to be
consistent with what it returns.

Now, one of the things that occurred to me after posting the survey is
that saying that a server cannot implement LIST OVERVIEW.FMT unless the
overview database is consistent with it has the problem that if the server
doesn't want to guarantee that consistency, it may break some clients that
think that XOVER isn't supported if LIST OVERVIEW.FMT isn't.  One of the
responses brought that out specifically, saying that if we're going to
leave XOVER alone, we should leave LIST OVERVIEW.FMT alone as well and
define a new command if necessary.  On the other hand, in practice,
getting that consistency seems to not normally be a problem, as large
volume news servers seem to mostly stick to the basic 8 headers, and the
sites that do change what's in overview tend to be smaller servers where
an overview rebuild isn't out of the realm of possibility.

I'm not sure how much clarity came out of this survey, although I think
the sentiment seemed to run towards dropping the command.  However, the
example of tin is a fairly good one.

Thoughts?


Here are the results of my recent questions about LIST OVERVIEW.FMT
support, summarized.  If clients are listed by a person's name, that's the
person speaking for a client (I wasn't sure what client it was).

[Q1] If you know how a reading client is implemented, does that reading
     client ever send LIST OVERVIEW.FMT?

Yes (33%):  XanaNews, tin, Hamster, NewsCache
No  (67%):  Gravity, Curt Welch (x2), MyNews, Dialog, Dave Sparks,
            fetchnews, Gnus

[Q2] If the client does send LIST OVERVIEW.FMT, what does it use the
     returned data for?

XanaNews: Whether the server supports XOVER.
tin: To see if Xref is included in overview.
Hamster: Just stores it for debugging.
NewsCache: Uses it to interpret the XOVER return to convert it between the
  server overview database and the internal overview database.

[Q3] If you know how a server is implemented, does it implement LIST
     OVERVIEW.FMT?

Yes (88%):  newsreader.com, Hamster, Supernews, leafnode, NewsCache,
            Cyrus, INN
No  (12%):  MyNews

[Q4] If the server does implement LIST OVERVIEW.FMT, what are its
     semantics?  Does it follow (2) above, (3) above, or some other
     meaning entirely?  And if the last, what meaning?

newsreader.com: Hard-coded response (including Xref).
Hamster: Hard-coded response (including Xref).
Supernews: Always just the required fields plus Xref.
leafnode: Hard-coded response (including Xref).
NewsCache: Hard-coded response (including Xref).
Cyrus: Hard-coded response (not including Xref).
INN: Reflects whatever is currently going into overview.

[Q5] What approach do you think should be taken by the next NNTP
     standard?

2 -- Standardize and require the database be consistent.
1 -- Standardize and require that the first eight fields always be the same.
4 -- Do not standardize.
2 -- Do not standardize and fix what's in overview.
3 -- Undecided.

25% standardize, 50% do not standardize, 25% undecided

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>