roxen.lists.roxen.general

Subject Author Date
RE: [PATCH 14/17] New module: gzip-on-the-fly Arjan van Staalduijnen <Arjan[dot]van[dot]Staalduijnen[at]rtl[dot]nl> 21-01-2009
A vary_callback is a little function (lambda) which adds a minor "hook"
to the protocol cache mechanism, extending the key used to locate the
proper entry (just the URL is not always sufficient). My accept_callback
was just implemented like this:

string accept_encoding_callback( string u, RequestID id )
{
        if (has_value((id->request_headers["accept-encoding"] || ""),
"x-gzip"))
                return "x-gzip";
        else if (has_value((id->request_headers["accept-encoding"] ||
""), "gzip"))
                return "gzip";
        else
                return "";      // no gzip support
}

....making sure the proper entry is used for whichever possibility of
accept-encoding header it accepts (thus differenting between gzip,
x-gzip and none supported).

By adding a vary_callback "early" in the code I meant that you have to
make sure the proper callbacks are registered -before- the code
depending on the returns a result which depended on whichever value it
was depending upon. If your module is a filter module only, it suffices
to add the vary_callback somewhere in that filter part - but before it
returns a result (so, say, after you have checked the content-type
matched text/*, but before your discover the client does or does not
support gzip-encoding).

Vary_callbacks exist since Roxen 4.0 or 4.5, but a reason I never
completely finished this module yet was because I was bumping into some
issues (possible bugs) - but I ran out of time to figure out if it was a
bug in roxen's implementation or in my customizations to it.

Vary_callbacks can only process data which they have available at the
early stages of a request; basically that is a limited set of client
data (such as header values, or remote IP) only - not result mappings.


Regards,

Arjan

-----Oorspronkelijk bericht-----
Van: Stephen R. van den Berg [mailto:<srb[at]cuci.nl>] 
Verzonden: Tuesday, January 20, 2009 11:18 PM
Aan: Martin Stjernholm
CC: <roxen[at]roxen.com>; Martin Jonsson
Onderwerp: Re: [PATCH 14/17] New module: gzip-on-the-fly

Martin Stjernholm wrote:
>"Stephen R. van den Berg" <<srb[at]cuci.nl>> wrote:

>>  server/modules/filters/gziponthefly.pike |  102
++++++++++++++++++++++++++++++

>Just recently, Marty (cc'd) made an experimental patch to get
>compression directly in the http protocol. One reason to do that was
>to make it happen after the protocol module has fixed the charset
>encoding. Another is that it can be tied in with the protocol cache
>(it might not be wise to spend cpu on compressing dynamic answers).

>Maybe you can compare solutions?

Sure.  I'd be happy to review Marty's.
With respect to dynamic answers, there certainly is a point to
compressing them.  CPU is (again) getting cheaper than bandwidth these
days.
Using a light compression factor you normally outrun the network and
don't use a lot of CPU.
My compressor singles out text/* content-types, it doesn't even try
for others (all others should be compressed already).

P.S. Any guidance on the usage of register_vary_callback ?
I looked at all the sample code in Roxen and Sitebuilder, all the docs,
and even at the source supposedly invoking the callbacks, but it's not
at all clear; especially with Arjan telling me I can use the callbacks
both on request and on result headers.  It seems to work for request
headers only.
-- 
Sincerely,
           Stephen R. van den Berg.

"Please, somebody give him a blowjob so we can impeach him."


__________________________________________________________
Deze e-mail en de inhoud is vertrouwelijk en uitsluitend bestemd voor de
geadresseerde(n). Indien u niet de geadresseerde bent van deze e-mail verzoeken
wij u dit direct door te geven aan de verzender door middel van een reply e-mail
en de ontvangen e-mail uit uw systemen te verwijderen. Als u geen geadresseerde
bent, is het niet toegestaan om kennis te nemen van de inhoud, deze te kopieren,
te verspreiden, bekend te maken aan derden noch anderszins te gebruiken.

The information contained in this e-mail is confidential and may be legally
privileged. It is intended solely for the addressee. If you are not the intended
recipient, any disclosure, copying, distribution or any action taken or omitted
to be taken in reliance on it, is prohibited and may be unlawful. Please notify
us immediately if you have received it in error by reply e-mail and then delete
this message from your system.
__________________________________________________________