GraphiteQuickNotes - henk52/knowledgesharing GitHub Wiki
- Graphite Web sites
- Installation
- Feeding graphite
- http://serverfault.com/questions/593157/graphite-shows-none-for-all-data-points-even-though-i-send-it-data
- http://graphite.readthedocs.org/en/1.0/tools.html
- http://graphite.readthedocs.org/en/latest/feeding-carbon.html#getting-your-data-into-graphite
- http://graphite.readthedocs.org/en/latest/feeding-carbon.html#existing-tools-and-apis
- Graphite intro
- https://kevinmccarthy.org/2013/07/18/10-things-i-learned-deploying-graphite/
- http://graphite.readthedocs.org/en/latest/overview.html
- https://ianunruh.com/2014/05/monitor-everything-part-4.html
- http://slidedeck.io/gord1anknot/graphite-vs-elasticsearch
- https://www.safaribooksonline.com/library/view/monitoring-with-graphite/9781491916421/ch04.html
- https://graphite.readthedocs.org/en/latest/
- Other noteworthy pages
- http://dieter.plaetinck.be/post/a_few_common_graphite_problems_and_how_they_are_already_solved/
- http://dieter.plaetinck.be/post/25-graphite-grafana-statsd-gotchas/
- https://grafana.com/blog/2016/03/03/25-graphite-grafana-and-statsd-gotchas/
- https://sysadmin.compxtreme.ro/clean-up-whisper-database/
- https://github.com/graphite-project/carbon/issues/389
- https://www.hostedgraphite.com/docs/
- Graphana
- How do I install the app?
- How do I provide data to it?
- Why aren't there any data points?
- How do I show data for the past 24 hours
- How do I login?
- How do I customize the view?
- How do I extract data and analyze that data?
- How do I provide large amounts of data efficient?
- Installing user: https://github.com/graphite-project/graphite-web/issues/667
- remove all references to EPEL repo(2018-02-27 the epel version fails, there should be a fix on the way)
- Add 'opstools' to the CentOS repo file
- [CentOS-opstools]
- name=CentOS-opstools
- baseurl=http://dm/mirrors/CentOS/$releasever/opstools/$basearch/
- gpgcheck=0
- gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-$releasever
- yum install -y graphite-web python-carbon
- vi /etc/carbon/storage-schemas.conf
- Add 'default'
- systemctl enable carbon-cache
- systemctl start carbon-cache
- systemctl status carbon-cache
- vi /etc/graphite-web/local_settings.py
- TIME_ZONE = 'CET'
- SECRET_KEY = 'UNSAFE_DEFAULT'
- PYTHONPATH=/usr/share/graphite/webapp django-admin syncdb --settings=graphite.settings
- echo > /etc/httpd/conf.d/welcome.conf
- vi /etc/httpd/conf.d/graphite-web.conf
- sudo chown apache:apache /var/lib/graphite-web/graphite.db
- touch /var/lib/graphite-web/index
- sudo systemctl enable httpd
- sudo systemctl start httpd
- sudo systemctl status httpd
- sudo firewall-cmd --permanent --add-service=http
- sudo firewall-cmd --permanent --add-port=2003-2004/tcp
- sudo firewall-cmd --reload
[default]
pattern = .*
retentions = 10s:4h, 2m:3d, 5m:8d, 13m:32d, 1h:1y
- https://stackoverflow.com/questions/35454154/importerror-cannot-import-name-generic
- https://github.com/graphite-project/graphite-web/issues/2227
- https://django-tagging.readthedocs.io/en/develop/
- https://github.com/graphite-project/graphite-web/issues/1727
- https://www.tecmint.com/yum-lock-disable-blacklist-certain-package-update-version/
PYTHONPATH=/usr/share/graphite/webapp django-admin syncdb --settings=graphite.settings
Traceback (most recent call last):
File "/usr/bin/django-admin", line 5, in <module>
management.execute_from_command_line()
File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
utility.execute()
File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute
django.setup()
File "/usr/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models()
File "/usr/lib/python2.7/site-packages/django/apps/config.py", line 202, in import_models
self.models_module = import_module(models_module_name)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/usr/lib/python2.7/site-packages/graphite/events/models.py", line 6, in <module>
from tagging.managers import ModelTaggedItemManager
File "/usr/lib/python2.7/site-packages/tagging/managers.py", line 8, in <module>
from tagging.models import Tag, TaggedItem
File "/usr/lib/python2.7/site-packages/tagging/models.py", line 10, in <module>
from django.contrib.contenttypes import generic
ImportError: cannot import name generic
- MAX_CREATES_PER_MINUTE = 70
* /etc/carbon/carbon.conf
- https://www.digitalocean.com/community/tutorials/how-to-configure-collectd-to-gather-system-metrics-for-graphite-on-ubuntu-14-04
- https://www.tecmint.com/install-collectd-and-collectd-web-to-monitor-server-resources-in-linux/
- sudo yum install -y collectd collectd-utils collectd-disk collectd-sensors
- sudo vi /etc/collectd.conf
- sudo setsebool collectd_tcp_network_connect on
- sudo systemctl enable collectd
- sudo systemctl start collectd
- sudo systemctl status collectd
<Plugin write_graphite>
<Node "graphing">
Host "GRAPHITE_HOST_IP"
Port "2003"
Protocol "tcp"
ReconnectInterval 0
LogSendErrors true
Prefix "collectd."
# Postfix "collectd"
StoreRates true
# AlwaysAppendDS false
EscapeCharacter "_"
# SeparateInstances false
# PreserveSeparator false
# DropDuplicateFields false
</Node>
</Plugin>
<Plugin disk>
Disk "/^[hs]d[a-f][0-9]?$/"
IgnoreSelected false
# UseBSDName false
# UdevNameAttr "DEVNAME"
</Plugin>
write_graphite plugin: Connecting to localhost:2003 via tcp failed. The last error was: failed to connect to remote host: Permission denied
See: https://stackoverflow.com/questions/34787016/collectd-permission-denied-to-connect-to-remote-host
Fix: setsebool collectd_tcp_network_connect on
write_graphite plugin: Connecting to localhost:2003 via tcp failed. The last error was: failed to connect to remote host: Permission denied
- wget https://dl.grafana.com/oss/release/grafana_5.4.3_amd64.deb
- sudo dpkg -i grafana_5.4.3_amd64.deb
- sudo apt install ./grafana_5.4.3_amd64.deb
- sudo vi /etc/grafana/grafana.ini
- sudo systemctl start grafana-server
- Create the machine instance
- connect to the machine
- puppet module install dwerder-graphite
- update the machine
- apt-get update
- apt-get upgrade
- puppet apply tst.pp
- Content of tst.pp:
- class { 'graphite': }
- Content of tst.pp:
- http://www.michael-noll.com/blog/2013/06/06/installing-and-running-graphite-via-rpm-and-supervisord/
- sudo python /usr/lib/python2.7/site-packages/graphite/manage.py syncdb
- python /usr/share/graphite/graphite-web.wsgi
# https://doauto.wordpress.com/2013/06/22/how-to-change-a-file-using-puppet/
file_line { 'whitelist':
path => '/opt/graphite/webapp/graphite/whitelist/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'graphite':
path => '/opt/graphite/webapp/graphite/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'render':
path => '/opt/graphite/webapp/graphite/render/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'account':
path => '/opt/graphite/webapp/graphite/account/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'browser':
path => '/opt/graphite/webapp/graphite/browser/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'version':
path => '/opt/graphite/webapp/graphite/version/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'dashboard':
path => '/opt/graphite/webapp/graphite/dashboard/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'composer':
path => '/opt/graphite/webapp/graphite/composer/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'events':
path => '/opt/graphite/webapp/graphite/events/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'metrics':
path => '/opt/graphite/webapp/graphite/metrics/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'graphlot':
path => '/opt/graphite/webapp/graphite/graphlot/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
file_line { 'cli':
path => '/opt/graphite/webapp/graphite/cli/urls.py',
line => 'from django.conf.urls import *',
match => '^from django\.conf\.urls\.defaults import \*',
}
sudo systemctl restart httpd
Delete the '.defaults' from 'from django.conf.urls.defaults import *'
vi /opt/graphite/webapp/graphite/whitelist/urls.py vi /opt/graphite/webapp/graphite/urls.py vi /opt/graphite/webapp/graphite/render/urls.py vi /opt/graphite/webapp/graphite/account/urls.py vi /opt/graphite/webapp/graphite/browser/urls.py vi /opt/graphite/webapp/graphite/version/urls.py vi /opt/graphite/webapp/graphite/dashboard/urls.py vi /opt/graphite/webapp/graphite/composer/urls.py vi /opt/graphite/webapp/graphite/events/urls.py vi /opt/graphite/webapp/graphite/metrics/urls.py vi /opt/graphite/webapp/graphite/graphlot/urls.py vi /opt/graphite/webapp/graphite/cli/urls.py service apache2 restart
tail -f /opt/graphite/storage/error.log
File "/opt/graphite/webapp/graphite/urls.py", line 15, in <module> from django.conf.urls.defaults import * ImportError: No module named defaults
Seems to not be a problem, running the httpd it works without issue.
- /var/log/httpd/graphite-web-error.log
python /usr/share/graphite/graphite-web.wsgi
Traceback (most recent call last):
File "/usr/share/graphite/graphite-web.wsgi", line 22, in <module>
import graphite.metrics.search # noqa
File "/usr/lib/python2.7/site-packages/graphite/metrics/search.py", line 111, in <module>
searcher = IndexSearcher(settings.INDEX_FILE)
File "/usr/lib/python2.7/site-packages/graphite/metrics/search.py", line 17, in __init__
retcode = subprocess.call(build_index_path)
File "/usr/lib/python2.7/subprocess.py", line 522, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1335, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
The quick fix is to restart the httpd service, that takes care of it.
https://theholyjava.wordpress.com/2014/05/05/graphite-shows-metrics-but-no-data-troubleshooting/
or it could be a problem of e.g. using a db with 5 second resolution and your data only being sent every 16 seconds. Fix: Send your data 4 times with a 0,5,10,15 second timestamp delta, then it will show up.
Must be owned by the HTTPD e.g.
chown apache:apache /var/lib/graphite-web/index(This is not about missing datapoints, but about the metric entry missing)
The metrics are sent but not created.