This document is not ready for publication. Remove this notification and remove viewer restrictions when ready.
The Service stores metrics data to an InfluxDB service with a default retention policy of 24 hours. This may not be long enough time for your use cases. This document will show with examples how to store metrics data for a longer period of time, and how to configure Grafana to display it.
The theory
To store measurements for a longer time, it must be stored with a Retention Policy of desired length. It doesn’t make sense to store ALL data for the specified time: When Grafana shows values from a month in a graph, it physically cannot display data at 50 millisecond accuracy. Having a lot of data at very short intervals takes a lot of processing, bandwidth and disk space. So you must think about downsampling the data as well.
Downsampling and retaining data is done with two tools:
Continuous Query (CQ) - an automatically, periodically run query which collects and downsamples data
Retention Policy (RP) - Definition of how long the data is kept
InfluxDB 1.8 documentation
Using already existing data
The ID service may have already added Continuous Queries and Retention Policies that meet your needs. Check the existing queries with the influx prompt:
> use oneportal Using database oneportal > show continuous queries name: _internal name query ---- ----- name: oneportal name query ---- ----- openid_api_requests_hourly CREATE CONTINUOUS QUERY openid_api_requests_hourly ON oneportal BEGIN SELECT count(auth_header_exists) INTO oneportal.one_year.openid_api_requests_hourly FROM oneportal.one_day.openid_api_request GROUP BY time(1h), path, method END openid_api_requests_daily CREATE CONTINUOUS QUERY openid_api_requests_daily ON oneportal BEGIN SELECT sum(count) INTO oneportal.five_years.openid_api_requests_daily FROM oneportal.one_year.openid_api_requests_hourly GROUP BY time(1d), path, method END mgmt_api_requests_hourly CREATE CONTINUOUS QUERY mgmt_api_requests_hourly ON oneportal BEGIN SELECT count(auth_header_exists) INTO oneportal.one_year.mgmt_api_requests_hourly FROM oneportal.one_day.mgmt_api_request GROUP BY time(1h), path, method, api_client_id, oauth2_client_id END mgmt_api_requests_daily CREATE CONTINUOUS QUERY mgmt_api_requests_daily ON oneportal BEGIN SELECT sum(count) INTO oneportal.five_years.mgmt_api_requests_daily FROM oneportal.one_year.mgmt_api_requests_hourly GROUP BY time(1d), path, method, api_client_id, oauth2_client_id END log_entry_hourly CREATE CONTINUOUS QUERY log_entry_hourly ON oneportal BEGIN SELECT count(event_id) INTO oneportal.one_year.log_entry_hourly FROM oneportal.one_day.log_entry GROUP BY time(1h), target_namespace_id, target_type, source, log_level, log_life END log_entry_daily CREATE CONTINUOUS QUERY log_entry_daily ON oneportal BEGIN SELECT sum(count) INTO oneportal.five_years.log_entry_daily FROM oneportal.one_year.log_entry_hourly GROUP BY time(1d), target_namespace_id, target_type, source, log_level, log_life END filterable_service_find_max_results_hourly CREATE CONTINUOUS QUERY filterable_service_find_max_results_hourly ON oneportal BEGIN SELECT max(results) INTO oneportal.one_year.filterable_service_find_max_results_hourly FROM oneportal.one_day.filterable_service_find GROUP BY time(1h), service_class END filterable_service_find_count_queries_hourly CREATE CONTINUOUS QUERY filterable_service_find_count_queries_hourly ON oneportal BEGIN SELECT count(results) INTO oneportal.one_year.filterable_service_find_count_queries_hourly FROM oneportal.one_day.filterable_service_find GROUP BY time(1h), service_class END name: ruuvitag name query ---- -----
Above we see from the first result that there exists a query called openid_api_requests_hourly which collects a field called count which is a sum of auth_header_exists values (e.g. a count of requests), grouped with 1 hour accuracy for each path and method combination. It is available in the one_year retention policy with name openid_api_requests_hourly.
This existing continuous query data can be used in Grafana with the following configuration:
Creating a custom RP and CQ
Let’s suppose there isn’t a CQ and RP that meet our requirements.
Requirements
We need to have a Grafana display which shows Management API request counts by path, and we need to show data for the last 30 days.
What we already have
We can show a graph for the last 24 hours with the following Grafana Panel configuration:
The essential info we see from this configuration is:
Data is stored with the default retention policy (which we know is
one_dayby using theshow retention policiescommand) with measurement namemgmg_api_request.There’s a field or tag called
auth_header_exists, and in this graph we display the number of them by using thecount()function.We group by time and also the tag
path.The
pathtag is also used as an alias, the value of which is shown in the legend.
Create a Retention Policy
We need a retention policy that is at least 30 days long. Let’s see what is available:
> show retention policies name duration shardGroupDuration replicaN default ---- -------- ------------------ -------- ------- autogen 0s 168h0m0s 1 false five_years 43680h0m0s 168h0m0s 1 false one_day 24h0m0s 1h0m0s 1 true one_year 8736h0m0s 168h0m0s 1 false
We see that one_day is indeed the default, and the next longest is one_year. Since that is too long, we should create a new RP called one_month that has a duration of 30 days, or 720 hours.
> CREATE RETENTION POLICY "one_month" ON "oneportal" DURATION 720h REPLICATION 1
Now we can confirm that it was added.
> show retention policies name duration shardGroupDuration replicaN default ---- -------- ------------------ -------- ------- autogen 0s 168h0m0s 1 false five_years 43680h0m0s 168h0m0s 1 false one_day 24h0m0s 1h0m0s 1 true one_year 8736h0m0s 168h0m0s 1 false one_month 720h0m0s 24h0m0s 1 false