icalendar-timeseries-server/README.md

iCalendar Timeseries Server

This project is a small service that scrapes iCalendar files served over HTTP, parses their contents and returns the data in a timeseries format compatible to the /api/v1/query API endpoint of a Prometheus server. This allows e.g. a Grafana administrator to add a Prometheus data source pointing at this server, returning the events in the calendars in the event metric.

Example

Consider the following iCalendar file:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//ACME//NONSGML Rocket Powered Anvil//EN
BEGIN:VEVENT
UID:20190603T032500CEST-foo
SUMMARY:Foo
DESCRIPTION:An example event
DTSTART;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T032500
DTEND;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T040000
END:VEVENT
BEGIN:VEVENT
UID:20190603T032500CEST-bar
SUMMARY:Bar
DESCRIPTION:Another example event
DTSTART;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T032500
DTEND;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T040000
END:VEVENT
END:VCALENDAR

The server would transform this into the following API response:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "event",
          "calendar": "0",
          "uid": "20190603T032500CEST-foo",
          "summary": "Foo",
          "description": "An example event"
        },
        "value": [
          1560043497,
          1
        ]
      },
      {
        "metric": {
          "__name__": "event",
          "calendar": "1",
          "uid": "20190603T032500CEST-bar",
          "summary": "Bar",
          "description": "Another example event"
        },
        "value": [
          1560043497,
          1
        ]
      }
    ]
  }
}

Dependencies

• bottle: Serve the results
• dateutil: Work with recurring events
• icalendar: Parse iCalendar
• isodate: Parse ISO-8601 time periods
• jinja2: Template value replacements
• pytz: Work with timezones

Configuration

Configuration is done through a JSON config file:

Example

{
    "addr": "127.0.0.1",
    "port": 8090,
    "start_delta": "-PT3H",
    "end_delta": "P30D",
    "tz": "Europe/Zurich",
    "calendars": {
        "private": {
            "url": "https://example.cloud/dav/me/private.ics",
            "auth": {
                "type": "basic",
                "username": "me",
                "password": "mysupersecurepassword"
            }
        },
        "public": {
            "interval": "P1D",
            "url": "https://example.cloud/dav/me/public.ics"
        },
        "confidential": {
            "url": "https://example.cloud/dav/me/confidential.ics",
            "ca": "/etc/ssl/ca.pem",
            "auth": {
                "type": "tls",
                "keyfile": "/etc/ssl/client.pem",
                "passphrase": "mysupersecurepassword"
            }
        }
    },
    "key_replace": {
        "summary": "01_summary",
        "description": "02_description"
    },
    "value_replace": {
        "summary": "{{ summary|truncate(100) }}",
        "description": "{{ description|truncate(100) }}"
    }
}

Explanation

JMESPath	Type	Description
addr	string	The address to listen on.
port	int	The port to listen on.
start_delta	string	A signed ISO 8601 duration string, describing the event range start offset relative to the current time.
end_delta	string	An unsigned ISO 8601 duration string, describing the event range end offset relative to the current time.
tz	string	The local timezone.
calendars	dict	The calendars to scrape.
keys(calendars)	string	Name of the calendar.
calendars.*.url	string	The HTTP or HTTPS URL to scrape.
calendars.*.interval	string	An unsigned ISO 8601 duration string, describing the scrape interval for this calendar.
calendars.*.ca	string	Path to the CA certificate file to validate the server's TLS certificate against, in PEM format (optional).
calendars.*.auth	dict	Authorization config for the calendar.
calendars.*.auth[].type	string	Authorization type, one of none (no authorization), basic (HTTP Basic Authentication), tls (TLS client certificate).
calendars.*.auth[?type=='basic'].username	string	The Basic Auth username to authenticate with.
calendars.*.auth[?type=='basic'].password	string	The Basic Auth password to authenticate with.
calendars.*.auth[?type=='tls'].keyfile	string	Path to the key file containing the TLS private key, client certificate and certificate chain, in PEM format.
calendars.*.auth[?type=='tls'].passphrase	string	Passphrase for the private key (optional).
key_replace	dict	Labels to rename, might be necessary e.g. for column ordering in Grafana.
keys(key_replace)	string	The labels to rename.
key_replace.*	string	The names to rename the labels to.
value_replace	dict	Label values to postprocess.
keys(value_replace)	string	Original label name to postprocess, also may introduce new labels.
value_replace.*	string	The new value for the label. Supports Jinja2 templates. All original label values can be accessed by using their original name as a Jinja2 variable name.

Queries

The most basic query is simply the metric name:

event

In addition, PromQL label filters can be used.

event{calendar="public",foo=~".*"}

Why Prometheus API

• It's JSON. A JSON generator is builtin in Python, so no further dependency.
• The Prometheus Data Source is builtin into Grafana.
• The API is simple.