Problems and solutions - strix-technica/reddit GitHub Wiki

Problems and solutions

This is a list of problems and solutions I've encountered while experimenting with reddit.

I highly recommend that you create rsyslogd rules to separate logs so you don't have to go digging through piles of unrelated stuff in /var/log/syslog.

Tips:

  • Never modify run.ini (aka development.ini) directly; always modify development.update and run make ini or else updates to example.ini will have to be manually imported (and you do not want to have to do that).
  • If you restart a dependency service (rabbitmq, postgres, cassandra etc), you need to restart reddit itself to re-establish connections with those services. Execute (as root) reddit-restart
  • The default configuration is set up for development. For half decent performance, you should turn off debugging and also disable client-side compilation of CSS etc (but see below for a bug concerning uncompressedJS).
  • When messing with the stock CSS and if you have static assets enabled (as above), you need to get reddit to reload names.json. The easiest way to rebuild your CSS and trigger a reload is the following command (though I'm unsure whether zookeeper will also need to be restarted): touch development.update; make clean_css css names ini

Zookeeper is no longer optional.

Ensure you have zookeeper installed:

apt-get install zookeeperd

Updates to mcrouter

As reddit develops, new routes for memcached sometimes get added. If you're getting memcached errors (such as set failed or Operation now in progress):

  • Carefully cross-check the contents of install/setup_mcrouter.sh with /etc/mcrouter/global.conf.
  • If not up-to-date, copy the contents across
  • Ensure that /etc/mcrouter/global.conf is still world-readable!

Operation now in progress in particular probably means that mcrouter can't read its config file. Check /var/log/upstart/reddit-mcrouter-global.log for error messages.

Cassandra not running

If you're getting complaints from reddit to the effect Could not connect to 127.0.0.1:9160: error(111, 'Connection refused'), it probably means that Cassandra is not running which, on the recommended platform (Ubuntu 14.04), may be because you're running a too-recent version. If (as root)

/usr/sbin/cassandra -f

says Cassandra 3.0 and later require Java 8u40 or later. then your Cassandra is too recent (perhaps the package hold got removed somehow), in which case you need to downgrade:

apt-get install cassandra=1.2.19
apt-mark hold cassandra

Missing graphics

There is (as of this writing) a bug in the way that static content (served when ini setting compressedJS set True, which is the default) is served because of the relative URLs used for certain image assets. Pull request 1603 will fix it, if and when the reddit devs merge it.

For now, you can either apply the patch yourself or use [this hackish work-around)(https://www.reddit.com/r/redditdev/comments/4i6tig/problem_with_static_css_builds_uncompressedjs/).