Caching on Test, Stage, and Production Sites - pulibrary/pul_library_drupal GitHub Wiki

This caching strategy is in place for both library-prod and library-dev. http://library.princeton.edu, http://library-stage.princeton.edu, and http://library-test.princeton.edu are running behind varnish.

Cold Stop full Cache Flush

> sudo /etc/init.d/varnish stop
> sudo su - drupal
> cd /var/www/library
> drush cc # option 1   # this will be the slowest
> exit
> redis-cli flushall
> sudo /etc/init.d/httpd restart
> sudo /etc/init.d/varnish start

Hot Stop full Cache Flush

> sudo su - drupal
> cd /var/www/library
> drush cc # option 1   # this will be the slowest
> exit
> sudo /etc/init.d/varnish restart

Prime Cache

  • Top 500 will run for about 10 minutes
  • Resource pages 20-25 minutes
> sudo su - drupal
> cd /var/www/cacheprimer
## For the top "500" URLs
> cat library_core_urls.txt | sh binge.sh
## For all DB Resource Pages
> cat resource_list.txt | sh binge.sh

# To first Purge the cache then binge run the following
> cat library_core_urls.txt | sh binge_and_purge.sh
## For all DB Resource Pages
> cat resource_list.txt | sh binge_and_purge.sh

Redis

Redis is being used as the caching daemon for the production drupal site using the "predis" client stored is sites/all/libraries/predis https://github.com/nrk/predis. Redis runs on port 6379. That is the default port for redis.

To check the current status of the redis cache try:

redis-cli INFO | grep keyspace_

You should see a lot more hits than misses

[PRINCETON\kr2@Library-Prod library]$ redis-cli INFO | grep keyspace_
keyspace_hits:126106
keyspace_misses:36365

Varnish

Varnish is configured as a reverse proxy cache. Currently is running on port 6081. The varnish admin client runs on port 6082.

A snapshot of current performance can be found with any of the following commands:

  • varnishstat
  • varnishtop
  • varnishhist

Varnish Admin Client

This allows to interact and reload cache settings by issuing commands directly to varnish. Enter it with the following command run as sudo:

sudo varnishadm -T localhost:6082 -S /etc/varnish/secret

You should see:

[PRINCETON\kr2@Library-Prod library]$ sudo varnishadm -T localhost:6082 -S /etc/varnish/secret
[sudo] password for PRINCETON\kr2: 
200        
-----------------------------
Varnish Cache CLI 1.0
-----------------------------
Linux,2.6.32-279.2.1.el6.x86_64,x86_64,-sfile,-smalloc,-hcritbit

Type 'help' for command list.
Type 'quit' to close CLI session.

Common Varnish Commands

Swapping VCL files without clearing cache:

vcl.list
vcl.load newvcl /etc/varnish/default.vcl
vcl.use newvcl

Purging cached pages:

ban req.url == "/" ## clear site root 
ban req.http.host ~ drupal.org && req.url == "/"
ban req.httpd.X-Cache-Tags ~ "node:10,?"

View the current ban list (still fuzzy on what this exactly is):

ban.list

Purging Specific URLs from Varnish

Varnish supports the http PURGE method as means to remove specific URLs from the cache. You can only run this on the server where Varnish is running at the moment. Example:

curl -XPURGE http://library.princeton.edu/
curl -XPURGE http://library.princeton.edu/find/databases

If you need to flush patterns you will need to use the varnishadm client. This works only for specific URLs.