Tuesday, June 26, 2012

[Fixed] multisite caching on django

When you work with multisite on django you need to do some chaching tricks for optimize perfomance of you sites. Typically you use cache middleware and have no problem for one site

    MIDDLEWARE_CLASSES = [
        'django.middleware.cache.UpdateCacheMiddleware',
        ...
        'django.middleware.cache.FetchFromCacheMiddleware'
    ]
But for many sites you can't use it from the box (otherwise you get equal page for east.mycorp.com/contact.html and west.mycorp.com/contact.html). Trick is simple, you need to do some changes in cache middleware, placed in path to you django installation>/django/middleware/cache.py You need overwrite two function:
  • get_cache_key
  • learn_cache_key
Becasuse basicly django use request.get_full_path() for build cache_key, and when you deal with multisite you get the same result for east.mycorp.com/contact.html and west.mycorp.com/contact.html. We change get_full_path to build_absolute_uri So we put function:
  • _generate_cache_header_key
  • _generate_cache_key
  • get_cache_key - unchanged
  • learn_cache_key - unchanged
from /django/utils/cache.py to our new code in cache.py:
from django.conf import settings
from django.core.cache import get_cache, DEFAULT_CACHE_ALIAS
from django.utils.cache import patch_response_headers, get_max_age, _i18n_cache_key_suffix
from django.utils.hashcompat import md5_constructor
from django.utils.encoding import iri_to_uri

import re
cc_delim_re = re.compile(r'\s*,\s*')

# custom function for Multisiting
def _generate_cache_header_key(key_prefix, request):
    """Returns a cache key for the header cache."""
    #path = md5_constructor(iri_to_uri(request.get_full_path()))
    path = md5_constructor(iri_to_uri(request.build_absolute_uri())) # patch using full path
    cache_key = 'views.decorators.cache.cache_header.%s.%s' % (
        key_prefix, path.hexdigest())
    return _i18n_cache_key_suffix(request, cache_key)

def _generate_cache_key(request, method, headerlist, key_prefix):
    """Returns a cache key from the headers given in the header list."""
    ctx = md5_constructor()
    for header in headerlist:
        value = request.META.get(header, None)
        if value is not None:
            ctx.update(value)
    #path = md5_constructor(iri_to_uri(request.get_full_path()))
    path = md5_constructor(iri_to_uri(request.build_absolute_uri()))
    cache_key = 'views.decorators.cache.cache_page.%s.%s.%s.%s' % (
        key_prefix, request.method, path.hexdigest(), ctx.hexdigest())
    return _i18n_cache_key_suffix(request, cache_key)

def get_cache_key(request, key_prefix=None, method='GET', cache=None):
    """
    Returns a cache key based on the request path and query. It can be used
    in the request phase because it pulls the list of headers to take into
    account from the global path registry and uses those to build a cache key
    to check against.

    If there is no headerlist stored, the page needs to be rebuilt, so this
    function returns None.
    """
    if key_prefix is None:
        key_prefix = settings.CACHE_MIDDLEWARE_KEY_PREFIX
    cache_key = _generate_cache_header_key(key_prefix, request)
    if cache is None:
        cache = get_cache(settings.CACHE_MIDDLEWARE_ALIAS)
    headerlist = cache.get(cache_key, None)
    if headerlist is not None:
        return _generate_cache_key(request, method, headerlist, key_prefix)
    else:
        return None

def learn_cache_key(request, response, cache_timeout=None, key_prefix=None, cache=None):
    """
    Learns what headers to take into account for some request path from the
    response object. It stores those headers in a global path registry so that
    later access to that path will know what headers to take into account
    without building the response object itself. The headers are named in the
    Vary header of the response, but we want to prevent response generation.

    The list of headers to use for cache key generation is stored in the same
    cache as the pages themselves. If the cache ages some data out of the
    cache, this just means that we have to build the response once to get at
    the Vary header and so at the list of headers to use for the cache key.
    """
    if key_prefix is None:
        key_prefix = settings.CACHE_MIDDLEWARE_KEY_PREFIX
    if cache_timeout is None:
        cache_timeout = settings.CACHE_MIDDLEWARE_SECONDS
    cache_key = _generate_cache_header_key(key_prefix, request)
    if cache is None:
        cache = get_cache(settings.CACHE_MIDDLEWARE_ALIAS)
    if response.has_header('Vary'):
        headerlist = ['HTTP_'+header.upper().replace('-', '_')
                      for header in cc_delim_re.split(response['Vary'])]
        cache.set(cache_key, headerlist, cache_timeout)
        return _generate_cache_key(request, request.method, headerlist, key_prefix)
    else:
        # if there is no Vary header, we still need a cache key
        # for the request.get_full_path()
        cache.set(cache_key, [], cache_timeout)
        return _generate_cache_key(request, request.method, [], key_prefix)

And save it to the new folder SiteCache in your django project directory. Now add to settings.py:
MIDDLEWARE_CLASSES = (
    'SiteCache.cache.UpdateCacheMiddleware',
    ...
    'SiteCache.cache.FetchFromCacheMiddleware',
)
Congratulations, now request to cached page east.mycorp.com/contact.html and west.mycorp.com/contact.html return proper results!

3 comments:

  1. Hello,
    I'm missing somithing out there.

    I got a SiteCache folder in my django project with cache.py file (the patched version of /django/utils/cache.py).

    But I really don't understand how it can works if i add this to settings:

    MIDDLEWARE_CLASSES = (
    'SiteCache.cache.UpdateCacheMiddleware',
    ...
    'SiteCache.cache.FetchFromCacheMiddleware',
    )

    Obviously i'll get this error: "Middleware module "SiteCache.cache" does not define a "UpdateCacheMiddleware" class."

    Of course SiteCache.cache is overriding django.utils.chace and NOT django.middleware.cache, so why put it in middleware settings??

    ReplyDelete
    Replies
    1. Hi!
      You need to put SiteCache folder near you settings.py file, not inside your app folder.

      Delete
  2. still not working..
    can u send me detail of this ...
    i am trying to best but not working .... ?
    Help me @ cyberkishor@gmail.com

    ReplyDelete