This document is not ready for publication. Remove this notification and remove viewer restrictions when ready.
The Service stores metrics data to an InfluxDB service with a default retention policy of 24 hours. This may not be long enough time for your use cases. This document will show with examples how to store metrics data for a longer period of time, and how to configure Grafana to display it.
The theory
To store measurements for a longer time, it must be stored with a Retention Policy of desired length. It doesn’t make sense to store ALL data for the specified time: When Grafana shows values from a month in a graph, it physically cannot display data at 50 millisecond accuracy. Having a lot of data at very short intervals takes a lot of processing, bandwidth and disk space. So you must think about downsampling the data as well.
Downsampling and retaining data is done with two tools:
Continuous Query (CQ) - an automatically, periodically run query which collects and downsamples data
Retention Policy (RP) - Definition of how long the data is kept
InfluxDB 1.8 documentation
Using already existing data
The ID service may have already added Continuous Queries and Retention Policies that meet your needs. Check the existing queries with the influx prompt:
> use oneportal Using database oneportal > show continuous queries name: _internal name query ---- ----- name: oneportal name query ---- ----- openid_api_requests_hourly CREATE CONTINUOUS QUERY openid_api_requests_hourly ON oneportal BEGIN SELECT count(auth_header_exists) INTO oneportal.one_year.openid_api_requests_hourly FROM oneportal.one_day.openid_api_request GROUP BY time(1h), path, method END openid_api_requests_daily CREATE CONTINUOUS QUERY openid_api_requests_daily ON oneportal BEGIN SELECT sum(count) INTO oneportal.five_years.openid_api_requests_daily FROM oneportal.one_year.openid_api_requests_hourly GROUP BY time(1d), path, method END mgmt_api_requests_hourly CREATE CONTINUOUS QUERY mgmt_api_requests_hourly ON oneportal BEGIN SELECT count(auth_header_exists) INTO oneportal.one_year.mgmt_api_requests_hourly FROM oneportal.one_day.mgmt_api_request GROUP BY time(1h), path, method, api_client_id, oauth2_client_id END mgmt_api_requests_daily CREATE CONTINUOUS QUERY mgmt_api_requests_daily ON oneportal BEGIN SELECT sum(count) INTO oneportal.five_years.mgmt_api_requests_daily FROM oneportal.one_year.mgmt_api_requests_hourly GROUP BY time(1d), path, method, api_client_id, oauth2_client_id END log_entry_hourly CREATE CONTINUOUS QUERY log_entry_hourly ON oneportal BEGIN SELECT count(event_id) INTO oneportal.one_year.log_entry_hourly FROM oneportal.one_day.log_entry GROUP BY time(1h), target_namespace_id, target_type, source, log_level, log_life END log_entry_daily CREATE CONTINUOUS QUERY log_entry_daily ON oneportal BEGIN SELECT sum(count) INTO oneportal.five_years.log_entry_daily FROM oneportal.one_year.log_entry_hourly GROUP BY time(1d), target_namespace_id, target_type, source, log_level, log_life END filterable_service_find_max_results_hourly CREATE CONTINUOUS QUERY filterable_service_find_max_results_hourly ON oneportal BEGIN SELECT max(results) INTO oneportal.one_year.filterable_service_find_max_results_hourly FROM oneportal.one_day.filterable_service_find GROUP BY time(1h), service_class END filterable_service_find_count_queries_hourly CREATE CONTINUOUS QUERY filterable_service_find_count_queries_hourly ON oneportal BEGIN SELECT count(results) INTO oneportal.one_year.filterable_service_find_count_queries_hourly FROM oneportal.one_day.filterable_service_find GROUP BY time(1h), service_class END name: ruuvitag name query ---- -----
Above we see from the first result that there exists a query called openid_api_requests_hourly which collects a field called count which is a sum of auth_header_exists values (e.g. a count of requests), grouped with 1 hour accuracy for each path and method combination. It is available in the one_year retention policy with name openid_api_requests_hourly.
This existing continuous query data can be used in Grafana with the following configuration:
