[NNTP] NNTP Compression

[Julien ÉLIE]

Hi Ken,

> Below is the spec that the Lemonade WG did for IMAP and I think something similar would work for NNTP.  This could work for 
> Client->Server traffic and/or Peer->Peer traffic.  I'd be willing to write up an I-D if folks think it would be useful.  Cyrus 
> IMAP and Mozilla T-Bird v3 both support it and it works great.
>
> http://tools.ietf.org/html/rfc4978

I think it would be useful to have a similar capability for NNTP.


As it was noted in the previous discussion, clients may want to
use compression only for a few commands.  Wouldn't it be wise
to have an NNTP command that can both activate and deactivate
the compression?

For instance:

[C] CAPABILITIES
[S] 101 Capability list:
[S] VERSION 2
[S] AUTHINFO USER SASL
[S] COMPRESS DEFLATE LZMA LZMA2
[S] HDR
[S] LIST ACTIVE COUNTS HEADERS NEWSGROUPS OVERVIEW.FMT
[S] OVER
[S] READER
[S] SASL GSSAPI OTP PLAIN NTLM LOGIN DIGEST-MD5 CRAM-MD5
[S] STARTTLS
[S] .
[C] COMPRESS LZMA2 ON
[S] 202 Let's talk compressed
-- now compressed --
[C] LIST COUNTS
[S] 215 List of newsgroups follows
[S] misc.test 3002322 3000234 1234 y
[S] .
[C] LIST NEWSGROUPS
[S] 215 List of descriptions follows
[S] misc.test General Usenet testing
[S] .
[C] GROUP misc.test
[S] 211 1234 3000234 3002322 misc.test
[C] OVER 3000234-3002322
[S] 224 Overview information follows
...
[S] .
[C] COMPRESS LZMA2 OFF
[S] 202 Let's talk uncompressed
-- now uncompressed --
[C] ARTICLE 3000234
[S] 220 3000234 <45223423 at example.com>
...
[S] .
[C] ARTICLE 3000235
...


Note that the syntax is just a suggestion and can be criticized.
I do not know what is the right second digit in the answer.
8 is for authentication and privacy.  So it may be 0?
Or create a meaning for 7?

I see that RFC 4978 (COMPRESS for IMAP) defines strategies
to find out incompressible attachments but it may not always
be possible to be sure we do not have to compress (yEnc articles,
base64 articles, etc.).  So maybe clients may want to retrieve
articles always uncompressed?
Perhaps the response code should be different for ON/OFF.

If we do not support OFF, then STARTCOMPRESSED or something similar
would be a better name because we already have STARTTLS.


> FYI, since all of our components in Cyrus use the same low-level protocol I/O layer (plaintext, TLS, SASL, COMPRESS=DEFLATE), I 
> can have both a client and server NNTP implementation available in about 5min.

That's great!

-- 
Julien ÉLIE

« -- Essayons d'interroger ce garde habilement sans éveiller ses soupçons...
  -- Hep ! Où est enfermé Assurancetourix ? » (Astérix)