ietf-nntp HDR

[Andrew Gierth]

>>>>> "Charles" == Charles Lindsey <chl at clw.cs.man.ac.uk> writes:

 >> However, now I think some more, I recall something. Did we say
 >> that this command could work for any header ? Or is it intended to
 >> work from the overview database ? In the latter case, it does make
 >> sense to have a 412 response to the message-id form, and it does
 >> make sense for the article number to be included in the
 >> response. This would not be consistent with ARTICLE, of course.

 Charles> Yes, I had meant to raise that question, but forgot. I see
 Charles> three possibilities:

 Charles> (1) The header in the response MUST be exactly as in the
 Charles> article (including all folding, comments, TABs, whatever).

this would be a serious implementation problem, since common practice
is to store the overview in pre-digested form and run XHDR commands
using the overview database (at least for overview headers).

 Charles> (2) The header MAY/SHOULD/MUST be bowdlerized by contracting
 Charles> whitespace (including folding) to a single SP, as in the
 Charles> OVER response.

No, that's _not_ what the OVER response does! As I've explained on
several occasions, collapsing whitespace in overview headers _does_
confuse clients (because existing servers don't do it for XOVER, and
the content of the mandatory overview fields are expected (by clients)
to be the same for any given article regardless of the server).
Currently, overview responses _do_ differ when folding or strange
control characters are present, which is why a) we need to define
exactly how this should be done and b) we should ensure that for the
no-control-characters, no-folding case, where current servers all
behave the same anyway, we should stick with the existing behaviour.

At some point when we discussed it before, we defined the "overview
transformation" for any given header as follows:

  - unfold the header by removing all CRLF pairs
  - replace any remaining occurrences of CR,LF or TAB,NUL with spaces
  - leave all other octets intact

(my server, in addition, removes leading whitespace, and treats bare
LF as a linebreak in addition to CRLF)

Existing practice with XHDR etc. is to do this, and in any case all
servers must return the same data, so (MAY/SHOULD) are not applicable,
this should be a MUST.

 Charles> (3) In addition to (2), the server MAY decline to respond in
 Charles> the case of headers not included in the overview. But in
 Charles> that case what response code should be sent?

Existing practice with XHDR/XPAT is (as I've repeatedly pointed out)
broken in this respect: some servers (Typhoon and its ilk) behave as
though the requested header were empty, other servers (e.g. INN, and
mine) return a 501 error.

I'd like to be able to allow HDR on arbitrary headers, but at the
moment it's out of the question for performance reasons.

-- 
Andrew.