icalendar-timeseries-server/README.md

5.5 KiB

iCalendar Timeseries Server

This project is a small service that scrapes iCalendar files served over HTTP, parses their contents and returns the data in a timeseries format compatible to the /api/v1/query API endpoint of a Prometheus server. This allows e.g. a Grafana administrator to add a Prometheus data source pointing at this server, returning the events in the calendars in the event metric.

Example

Consider the following iCalendar file:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//ACME//NONSGML Rocket Powered Anvil//EN
BEGIN:VEVENT
UID:20190603T032500CEST-foo
SUMMARY:Foo
DESCRIPTION:An example event
DTSTART;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T032500
DTEND;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T040000
END:VEVENT
BEGIN:VEVENT
UID:20190603T032500CEST-bar
SUMMARY:Bar
DESCRIPTION:Another example event
DTSTART;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T032500
DTEND;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T040000
END:VEVENT
END:VCALENDAR

The server would transform this into the following API response:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "event",
          "calendar": "0",
          "uid": "20190603T032500CEST-foo",
          "summary": "Foo",
          "description": "An example event"
        },
        "value": [
          1560043497,
          1
        ]
      },
      {
        "metric": {
          "__name__": "event",
          "calendar": "1",
          "uid": "20190603T032500CEST-bar",
          "summary": "Bar",
          "description": "Another example event"
        },
        "value": [
          1560043497,
          1
        ]
      }
    ]
  }
}

Dependencies

  • bottle: Serve the results
  • dateutil: Work with recurring events
  • icalendar: Parse iCalendar
  • isodate: Parse ISO-8601 time periods
  • jinja2: Template value replacements
  • pytz: Work with timezones

Configuration

Configuration is done through a JSON config file:

Example

{
    "addr": "127.0.0.1",
    "port": 8090,
    "start_delta": "-PT3H",
    "end_delta": "P30D",
    "tz": "Europe/Zurich",
    "calendars": {
        "private": {
            "url": "https://example.cloud/dav/me/private.ics",
            "auth": {
                "type": "basic",
                "username": "me",
                "password": "mysupersecurepassword"
            }
        },
        "public": {
            "interval": "P1D",
            "url": "https://example.cloud/dav/me/public.ics"
        },
        "confidential": {
            "url": "https://example.cloud/dav/me/confidential.ics",
            "ca": "/etc/ssl/ca.pem",
            "auth": {
                "type": "tls",
                "keyfile": "/etc/ssl/client.pem",
                "passphrase": "mysupersecurepassword"
            }
        }
    },
    "key_replace": {
        "summary": "01_summary",
        "description": "02_description"
    },
    "value_replace": {
        "summary": "{{ summary|truncate(100) }}",
        "description": "{{ description|truncate(100) }}"
    }
}

Explanation

JMESPath Type Description
addr string The address to listen on.
port int The port to listen on.
start_delta string A signed ISO 8601 duration string, describing the event range start offset relative to the current time.
end_delta string An unsigned ISO 8601 duration string, describing the event range end offset relative to the current time.
tz string The local timezone.
calendars dict The calendars to scrape.
keys(calendars) string Name of the calendar.
calendars.*.url string The HTTP or HTTPS URL to scrape.
calendars.*.interval string An unsigned ISO 8601 duration string, describing the scrape interval for this calendar.
calendars.*.ca string Path to the CA certificate file to validate the server's TLS certificate against, in PEM format (optional).
calendars.*.auth dict Authorization config for the calendar.
calendars.*.auth[].type string Authorization type, one of none (no authorization), basic (HTTP Basic Authentication), tls (TLS client certificate).
calendars.*.auth[?type=='basic'].username string The Basic Auth username to authenticate with.
calendars.*.auth[?type=='basic'].password string The Basic Auth password to authenticate with.
calendars.*.auth[?type=='tls'].keyfile string Path to the key file containing the TLS private key, client certificate and certificate chain, in PEM format.
calendars.*.auth[?type=='tls'].passphrase string Passphrase for the private key (optional).
key_replace dict Labels to rename, might be necessary e.g. for column ordering in Grafana.
keys(key_replace) string The labels to rename.
key_replace.* string The names to rename the labels to.
value_replace dict Label values to postprocess.
keys(value_replace) string Original label name to postprocess, also may introduce new labels.
value_replace.* string The new value for the label. Supports Jinja2 templates. All original label values can be accessed by using their original name as a Jinja2 variable name.

Queries

The most basic query is simply the metric name:

event

In addition, PromQL label filters can be used.

event{calendar="public",foo=~".*"}

Why Prometheus API

  • It's JSON. A JSON generator is builtin in Python, so no further dependency.
  • The Prometheus Data Source is builtin into Grafana.
  • The API is simple.