[NNTP] NNTP Compression

[Ken Murchison]

Julien ÉLIE wrote:

> Does MODE COMPRESS do the same thing as COMPRESS?  I see an explanation 
> here:
>    http://sedore.com/nntprelay/compressed-nntp-not.htm
> but as I do not know zlib APIs, I am unsure it does the same thing.

I'd have to look at the zlib docs and the actual Diablo code (which I 
have no desire to do), but tt looks like MODE COMPRESS is actually 
putting a gzip header on each chunk of compressed data.  The COMPRESS 
command as standardized by IMAP/NNTP COMPRESS doesn't bother putting an 
unnecessary header on the data.  That's why IMAP/NNTP COMPRESS uses 
[de|in]flatInit2() calls rather than the simpler [de|in]flateInit(), so 
it can specify more params rather than using defaults.

>> * The text referring to attachments needs work. Should we discuss yEnc
>> and/or uuencode? I'm not sure the text regarding binary form even applies
>> to NNTP.
> 
> When you say:
> 
>    > A very simple strategy is to change the level to 0 at the start of
>    > a multi-line data block provided the first two bytes are either
>    > 0x1F 0x8B (as in deflate-compressed files) or 0xFF 0xD8 (JPEG),
>    > and to keep it at 1-5 the rest of the time.
> 
> It would apply to BODY only (at the start of a multi-line data block).

It would also apply to ARTICLE, POST, IHAVE, TAKETHIS.

> But we usually have a MIME message or yEnc.  Not a binary file.

Correct, so this text can probably be removed.  IMAP has the ability to 
fetch individual MIME parts, and an extension to do so with any CTE 
stripped.

> Maybe yEnc and uuencode should be mentioned, yes.

yEnc and uuencode most likely would be similar to base64 in terms of how 
to handle them.


> Besides, in Section 3 about compression efficiency, can another type
> of data be mentioned?  I think about overview data (OVER), HDR and
> LIST answers.  Maybe also NEWNEWS answers.

All NNTP data other than large article bodies fall into the same 
category, so I don't think we need to mention all of them specifically.


>> In order to increase interoperability, it is desirable to have as few
>> different compression algorithms as possible, so this document specifies
>> only one.
> 
> Why couldn't we define several algorithms (some are more efficient than
> DEFLATE, for instance LZMA or LZMA2) and say that DEFLATE *MUST* be
> implemented when COMPRESS is implemented?


Well, IMAP/NNTP COMPRESS are really just stop-gaps until existing TLS 
implementations support compression.  As mentioned in the text TLS 
really should be the preferred way to negotiate compression.  So I don't 
see the need to add more algorithms to NNTP COMPRESS.



>> Second [...] Third
>> independent of the order
> 
> Isn't it better to use "Secondly", "Thirdly"/"Finally", "independently of"?

This text was already approved by the LEMONADE WG and the RFC editor, so 
I don't see any need to change it.


>> Example of a server failing to activate compression:
>>
>> [C] CAPABILITIES
>> [S] 101 Capability list:
>> [S] VERSION 2
>> [S] COMPRESS DEFLATE
>> [S] .
> 
> That example is weird:  a news server that requires compression to show
> other commands.  Maybe IHAVE and STREAMING should also be advertised?
> (case of a transit-only server)

Yes, you're right.  I was trying to keep extra clutter out of the 
examples, but I do need to add reader and/or feeder commands.


> Are there SASL mechanisms that negotiate a compression layer?  I think not,
> but better ask...

No currently.  Again, TLS is the preferred method for negotiating 
compression.


-- 
Kenneth Murchison
Systems Programmer
Carnegie Mellon University