Domoticz, InfluxDB, Grafana

Domoticz part 5: Grafana, InfluxDB, Telegraf – beautiful and easy graphs

Grafana is the answer to the nagging question we’be been asking ourselves over the years – how to quickly and nicely present our data gathered from devices. InfluxDB on the other hand is the database that is as easy and simple to use, thus making it an ideal candidate for this job.
Let me show you how to quickly and easily put our data from domoticz and other devices to InfluxDB and then using Grafana – display in much more useful way than default domoticz graphs do.

What will we need?

Software

  • InfluxDB
  • Grafana
  • bash scripts

Hardware

  • Raspberry Pi – v2 or v3 (armv7)
  • OPTION: Raspberry Pi A/B/B+ lub Zero (armv6)
  • OPTION: A different ARM based micro-computer (i.e.: OPi PC” rel=”noopener” target=”_blank”>OrangePi using CPU H3/H2/H5) – they could be cheaper. It’s also entirely possible to use x86 computer with Debianem, but it’s going to eat way more than 4-6W when using ARM based. Here I used finally Orange Pi PC with latest armbian distribution, tested first on latest Raspberry Pi 2

Intallation

InfluxDB

Let’s download https://www.influxdata.com/downloads/ latest InfluxDB version (at the time of writting it was 1.0.2) – “Standalone Linux Binaries (ARM)”:

Then we have to create user influxdb:

Accept the questions with yes, add some data if you like, password.
Check if the user exists:

We should get similar, positive response:

Now, let’s install from influxdb binary:

All went well? Let’s start our InfluxDB:

It works! Let’s make sure it starts with the system, this time using a different tool:

Run it, push space to get “*” at influxdb.

Grafana

This time it’s a bit easier let’s use the ready package from https://github.com/fg2it/grafana-on-raspberry/. Choose a package either for Raspberry Pi 2/3 (armv7) or for Raspberry Pi A/B/B+ lub Zero (armv6), and download it using wget. Here I’m using Raspberry Pi 2/3 package:

This package tells us to add Grafana to start with the system in this way:

Let’s start grafana:

Configuration

Aktualne wersje InfluxDB zarządzamy poprzez linię komend (CLI):

Aby stworzyć bazę danych wystarczy:

Current InfluxDB ara managed by (CLI):

Creata database:

The older version of InfluxDB by default use the web interface – by using IP of our Raspberry at port 8083. Basic configuration allows unauthorized access via http. This is the interface:

InfluxDB: Interface
InfluxDB: Interface

so, let’s just choose “Create Database”, and name our first database “domoticz”:

InfluxDB: Creating domoticz database
InfluxDB: Creating domoticz database

That is it! Either with http interface or CLI – no need to create data, tables – we can simply add data to it.

Sending from domoticz to InfluxDB

Using Domoticz – use the menu and select “HTTP”:

HTTP push in Domoticz

We need to find data that is interesting and put it into Influxdb – so let’s choose atmospheric pressure:

http-data-pl-1

So how does this work? First, we choose a variable – our data, that is already in Domoticz system – from menu “Device name”. Then – we need a number to identify – I use the same as the IDX number of the device in domoticz – so here it’s “1450”. Tick the “Activate connection”, at the right side and provide URL that will be our connection via http to write toInfluxdb: IP, port – this time it’s 8086, database name, username/password if used and precision.
Method is HTTP POST, authorization – let’s leave that for later.
The most important part is the “Data:” here we enter our InfluxDB variable, that we going to use later to create graphs, it’s value and time. Keeping things small and simple, we shall call it “device_1450”. Use tsharkie, if you’re curious how the URL’s work, what is the response (HTTP code 204 if it’s fine). That is it – data will be stored in “domoticz”, database, no need to create types, indexes, tables – quite new approach, eh?

Sending data from any device (not domoticz)

Since InfluxDB is so easy going, everybody is getting ideas. We talk via HTTP, meaning that we can put the data using simple tools i.e. using curl, a popular program present in each modern distribution.:

Here’s and example on how to use it:

This very simple example uses SNMP (the old 32-bit counters) that queries the wireless interface to ISP, to show the “Internet speed”. curl is used here to send the data to database named “snmp”, the valiable is inwisp for Input and outwisp for Output, but to spice up things a bit – we use host (it’s optional), to be able to later group the devices. Then the value – which is the speed. We don’t put extra time value – we use the current system time.
The whole script – below (it’s much easier to use telegraf, but I still love the MRTG’s approach to collect data) is an 17 yeras old 17 bash script!. I used it via screen while starting the system same way as with the – Flightradar and SDR blog entry.

The stored data in snmp database should give you the following reply:

The other way to find out if we’re in the database is to use the Influxdb HTTP interface and issue from the menu – SHOW MEASUREMENTS:
InfluxDB:

Grafana – finally we’re getting to the beautiful graphs!

Grafana runs on port 3000, login and password admin/admin – now choose in left top corner – Data Sources. Add Source and type in – Type, URL, Access at proxy, user and password for the database:

Grafana: Add Data Source - InfluxDB
Grafana: Add Data Source – InfluxDB

If the data source is working – we can go to “Dashboard” and create our first :

Grafana: Add new Dashboard
Grafana: Add new Dashboard

Next – let’s add our first graph:

Grafana: Panel
Grafana: Panel

Now we see the default panel with single data series to plot/graph. It looks nice, but this is not our data yet:

Grafana: First Graph
Grafana: First Graph

Next step – select our database from the “Panel data source” – change from -Grafana- to SNMP. “A” – our data series should look like this:

Grafana: Add db, toggle edit mode
Grafana: Add db, toggle edit mode

Green line is gone, because we need to provide correct data – we do it in the same way – changing to the edit mode in the query “A” and fill in just like in exemple below. Next choose again SNMP “+ Data query” and fill in the “B” query – all changes are updated on the fly:

Grafana: example Internet speed series data entry
Grafana: example Internet speed series data entry

Go to “Axes” menu – here, select proper data type – “Unit”, “data rate” -> “kilobits/sec”. In the “General” name the Panel “Internet” and this is it. The floppy disc icon saves our new dashboard:

Grafana: first graph
Grafana: first graph

It’s time now to install aditional grafana puligns to get more of the graphs, gauges etc: https://grafana.net/plugins

Telegraf

The last interesting piece of software is Telegraf. This a very feature-rich piece of monitoring software, that is able to gather data from aplication statues, interfaces, CPU, and runs also on Linux> Simple installation follows simple configuration.
First – create new databes in InfluxDB – “telegrafPI” but this time using command line:

Now let’s install telegraf:
We can either add repository:

to

or get appropriate pacakge from:

or even use

Configuration is dead simple telegraf pre-configures the file based on what we tell it to, then we should save it in:

So to get you started lets monitor CPU:

Based that this is the same computer we monitor where influxdB is installed – of not – choose proper IP of the DB host.
In Grafana – after adding new database source, we’re using the following data series for plotting:

To get the result below, tick the “Stacked” in Display tab:

Grafana + Telegraf: CPU Example
Grafana + Telegraf: CPU Example

And this is it, go and create beautiful graphs!