When monitoring the OPENSTACK STORAGE SERVICE, Swift, we are looking at gathering key metrics from within the storage cluster in order to make decisions on its health. For this, we can use a small piece of middleware named swift-informant, together with StatsD and Graphite, to produce near real-time stats of our cluster.
We will be configuring StatsD and Graphite on a server that has access to the OpenStack Storage proxy server. Ensure that, this server has enough RAM (at least 1 GB), disk (at least 10 GB), and CPU (1 CPU for small test environments) capacity for the environment you are running.
If you would like to Enrich your career as an Openstack certified professional, then visit Mindmajix – A Global online training platform: “Openstack Course” |
To install StatsD and Graphite, carry out the following steps:
For this, we will be configuring a new Ubuntu 12.04 server. Once Ubuntu has been installed, we need to install some prerequisite packages.
apt-get -y install git python-pip gcc python2.7-dev apache2 libapache2-mod-python python-cairo python-django libapache2-mod-wsgi python-django-tagging
sudo pip install carbon sudo pip install whisper
sudo pip install graphite-web
cd /opt/graphite/conf
sudo mv carbon.conf.example carbon.conf
sudo mv storage-schemas.conf.example storage-schemas.conf
ServerName 172.16.0.253
DocumentRoot "/opt/graphite/webapp"
ErrorLog /opt/graphite/storage/log/webapp/error.log CustomLog /opt/graphite/storage/log/webapp/access.log common
name='%{GROUP}' inactivity-timeout=120 WSGIProcessGroup graphite WSGIApplicationGroup %{GLOBAL}
WSGIImportScript /opt/graphite/conf/graphite.wsgi process-group=graphite application-group=%{GLOBAL}
WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi
Alias /content/ /opt/graphite/webapp/content/ SetHandler None
Order deny,allow Allow from all
sudo a2ensite graphite
sudo mv graphite.wsgi.example graphite.wsgi
sudo chown -R www-data:www-data /opt/graphite/storage/log/ sudo touch /opt/graphite/storage/index
sudo chown www-data:www-data /opt/graphite/storage/index
sudo service apache2 restart
Related Article: Setting up a Chef Environment For OpenStack |
cd /opt/graphite/webapp/graphite sudo python manage.py syncdb
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes Username (Leave blank to use 'root'):
E-mail address: user@somedomain.com Password:
Password (again):
Superuser created successfully. Installing custom SQL … Installing indexes …
No fixtures found.
sudo chown -R www-data:www-data /opt/graphite/storage
cd /opt/graphite
sudo bin/carbon-cache.py start
sudo apt-get update
sudo apt-get -y install nodejs
git clone https://github.com/etsy/statsd.git
cd statsd
cp exampleConfig.js Config.js
{
graphitePort: 2003
graphiteHost: "localhost"
port: 8125
}
nohup node stats.js Config.js &
Related Article: Collecting Usage Statistics - Open Stack Cloud |
We are now ready to configure the OpenStack Swift proxy server to include the swift-informant middleware in the pipeline. This is done with configuring the /etc/swift/proxy-server.conf file.
git clone https://github.com/pandemicsyn/swift-informant.git cd swift-informant
sudo python setup.py install
[pipeline:main]
pipeline = informant healthcheck cache swift3 s3token tokenauth keystone proxy-server
[filter:informant]
use = egg:informant#informant statsd_host = 172.16.0.9
statsd_port = 8125
standard statsd sample rate 0.0 <= 1
statsd_sample_rate = 0.5
list of allowed methods, all others will generate a "BAD_METHOD" event
valid_http_methods = GET,HEAD,POST,PUT,DELETE,COPY
send multiple statsd events per packet as supported by statsdpy
combined_events = no
prepends name to metric collection output for easier recognition, e.g. company.swift.
metric_name_prepend =
sudo swift-init proxy-server restart
Gaining insight into what our OpenStack Storage cluster is doing can be achieved by including a piece of middleware in the pipeline of our OpenStack Storage proxy server named swift- informant, along with StatsD and Graphite. StatsD is a node.js service that listens for statistics sent to it in UDP packets. Graphite takes this data and gives us a real-time graph view of our running services.
Installation and configuration is done in stages. We first install and configure a server that will be used for StatsD and Graphite. Graphite can be installed using Python’s Package Index (using the pip tool), and for this, we install three pieces of software: carbon (the collector), whisper (fixed-size RRD service), and the Django Web Interface, graphite-web. Using the pip tool installs these services to the /opt directory of our server.
/usr/lib/python2.7/dist-packages/django. Alias /media/ "/usr/lib/python2.7/dist-packages/django/contrib/admin/media/"
We then ensure that the Graphite WSGI (Web Service Gateway Interface) file is in place at the appropriate path, as specified by the WSGIScriptAlias directive at /opt/graphite/conf/graphite.wsgi.
Name | Dates | |
---|---|---|
OpenStack Training | Jun 28 to Jul 13 | |
OpenStack Training | Jul 02 to Jul 17 | |
OpenStack Training | Jul 05 to Jul 20 | |
OpenStack Training | Jul 09 to Jul 24 |
Ravindra Savaram is a Content Lead at Mindmajix.com. His passion lies in writing articles on the most popular IT platforms including Machine learning, DevOps, Data Science, Artificial Intelligence, RPA, Deep Learning, and so on. You can stay up to date on all these technologies by following him on LinkedIn and Twitter.
1 /15