[NNTP] Compressed LIST (and other commands) answers

Urs Janßen urs at tin.org
Mon Nov 30 07:06:28 PST 2009


On Sat, Nov 28, 2009 at 01:38:39AM -0800, Russ Allbery wrote:
> > Is it best to have new commands?
> 
> >  ZHDR LZMA BASE64 Injection-Info 1-
> >  ZLIST LZMA BASE64 ACTIVE
> >  ZOVER LZMA BASE64 1-
> >  ZTAKETHIS LZMA NONE 123 <valid at mid>
> 
> > or a generic one (name to be defined) like:
> 
> >  ZCOMMAND LZMA BASE64 0 HDR Injection-Info 1-
> >  ZCOMMAND LZMA BASE64 0 LIST ACTIVE
> >  ZCOMMAND LZMA BASE64 0 OVER 1-
> >  ZCOMMAND LZMA NONE 123 TAKETHIS <valid at mid>
> 
> I think a generic command is an interesting idea and definitely better
> than creating a bunch of new separate commands.

but what about response code then? (i.e. HDR returns 221, OVER 224 and LIST
ACTIVE 215 on success) - with seperate commands we could keep this.

> Although I wonder if, at that point, whether we want to just provide some
> facility negotiate compression of all subsequent traffic on the NNTP
> connection.  In other words, rather than treating this on a
> command-by-command basis, what if we model it after STARTTLS?  It has the
> advantage of simplicity of description, but it has the serious drawback
> that it's another data layer, and I know from SASL and TLS that managing
> layers can be a huge pain and it's very easy to do it poorly.

I dislike another layer as it's usualy muich more complicated to implemnt
(correct) and not adding a compession layer has the advantage that clients
could use compression for only the few things which usualy produce a lot of
data (LIST ACTIVE/NEWSGROUPS, OVER). and it avoids recompression of already
compressed data (i.e. binarie postings).

just adding compression for multiline server -> client responses would be a
big win (and if compression/decompession speed is an issue one could go with
LZO (<http://www.oberhumer.com/opensource/lzo/>) as this is where users
spend time to wait for responses from the servers; client -> server
communication usualy is usualy less critical (exception might be batched
postings or the like).

ZLIST, ZHDR, ZOVER might suffice (ZNEWGROUPS, ZNEWNEWS, ZPAT, ZLISTGOUP
might be handy but usualy do not generate that much output).

I'd like to see BASE64 encoding instead of YENC as it's 7bit clean, has
short and fixed line length (except for the last), is well documented and
a decoder is usualy already implemented in readers. if there will be more
than one encoding to chose from BASE64 should IMHO be the mandatory one.

For compression I don't care, XZ (LZMA2) (<http://tukaani.org/xz/>) gives
good compression but GZIP and BZIP2 are much more common, LZO is fast but
less efficient. if there will be more than one compression to chose from
GZIP should probabely the mandatory one.

urs
-- 
"Only whimps use tape backup: _real_ men just upload their important stuff
 on ftp, and let the rest of the world mirror it ;)" - Linus



More information about the ietf-nntp mailing list