ietf-nntp Summary/analysis of LIST OVERVIEW.FMT responses

[Russ Allbery]

Ken Murchison <ken at oceana.com> writes:
> Russ Allbery wrote:

>> I thought a lot about this approach, but the difficulty with it comes
>> in with how a server would implement it in the face of an existing
>> XOVER implementation.  People actively use additional overview fields
>> (Newsgroups and Keywords being the most common) for some applications;
>> they just don't use LIST OVERVIEW.FMT to figure out that they're there.

> Gee, more non-standard stuff.  It's clear that the horse has been out in
> front of the cart for far too long ;) Where does a client get off
> _assuming_ that certain headers will be present and in a specific order?
> IMO, that's a poorly written client.

Why would the client have to assume anything at all?  That's why all the
fields after the first seven are labelled.  It's quite standard; it's part
of the original overview specification.

> Why would Newsgroups be needed in the overview data?  The client already
> knows what newsgroup they are in because it has to select it before
> using [X]OVER.

In order to killfile on crossposts.

> In any event, then lets just standardize whichever set of
> headers/metadata are deemed important by the majority of clients.

As a maintainer of INN, I can tell you what I see in practice.  People
have conflicting desires and conflicting goals.  Small sites may want to
put large numbers of headers into overview to make them more convenient;
large sites can't afford the bandwidth consumption of shipping useless
headers.

HDR is definitely a better solution; on that, I agree with you.  I'm just
worried about the transition issues.

> I guess an important question that was missing in the poll, was which
> fields do clients expect to be in [X]OVER.

A lot of people did mention that in responses; the vast majority of
clients just expect the standard eight.  Other headers are generally for
more specialized applications (other than Newsgroups, which was a bit of a
fad for a while).  Some news readers can use the information if it's there
and cope if it's not.

>> While telling them to switch to HDR isn't a bad idea, servers that want
>> to support those clients have to keep supporting XOVER until they all
>> switch.  But OVER wouldn't allow inclusion of those extra fields, which
>> now makes implementing OVER quite a bit more difficult.

> How so?  If you already implement XOVER, OVER is trivial.  Its either
> the same output, or a subset of it.  Are we back to implementation
> driving the protocol again?

It's all nice and good to say that implementation shouldn't drive the
protocol, but if we require stuff that implementors can't find a fast way
of implementing, they just won't implement the protocol, and then what
good is having a protocol?

Dumping records down a network connection is clearly faster than parsing
and rewriting those records before doing so.  How much faster?  I don't
know.  It's certainly enough that as an implementor, I'd be worried about
the speed hit and start thinking why I should bother to implement OVER at
all when everyone already uses XOVER.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>