icalendar-timeseries-server/README.md

# iCalendar Timeseries Server

This project is a small service that scrapes iCalendar files served
over HTTP, parses their contents and returns the data in a timeseries
format compatible to the `/api/v1/query` API endpoint of a Prometheus
server.  This allows e.g. a Grafana administrator to add a Prometheus
data source pointing at this server, returning the events in the
calendars in the `event` metric.

## Example

Consider the following iCalendar file:

```
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//ACME//NONSGML Rocket Powered Anvil//EN
BEGIN:VEVENT
UID:20190603T032500CEST-foo
SUMMARY:Foo
DESCRIPTION:An example event
DTSTART;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T032500
DTEND;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T040000
END:VEVENT
BEGIN:VEVENT
UID:20190603T032500CEST-bar
SUMMARY:Bar
DESCRIPTION:Another example event
DTSTART;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T032500
DTEND;TZID=Europe/Zurich;VALUE=DATE-TIME:20190603T040000
END:VEVENT
END:VCALENDAR
```

The server would transform this into the following API response:

```json
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "event",
          "calendar": "0",
          "uid": "20190603T032500CEST-foo",
          "summary": "Foo",
          "description": "An example event"
        },
        "value": [
          1560043497,
          1
        ]
      },
      {
        "metric": {
          "__name__": "event",
          "calendar": "1",
          "uid": "20190603T032500CEST-bar",
          "summary": "Bar",
          "description": "Another example event"
        },
        "value": [
          1560043497,
          1
        ]
      }
    ]
  }
}
```

## Dependencies

- `bottle`: Serve the results
- `dateutil`: Work with recurring events
- `icalendar`: Parse iCalendar
- `isodate`: Parse ISO-8601 time periods
- `jinja2`: Template value replacements
- `pytz`: Work with timezones

## Configuration

Configuration is done through a JSON config file:

### Example

```json
{
    "addr": "127.0.0.1",
    "port": 8090,
    "start_delta": "-PT3H",
    "end_delta": "P30D",
    "tz": "Europe/Zurich",
    "calendars": {
        "private": {
            "url": "https://example.cloud/dav/me/private.ics",
            "auth": {
                "type": "basic",
                "username": "me",
                "password": "mysupersecurepassword"
            }
        },
        "public": {
            "interval": "P1D",
            "url": "https://example.cloud/dav/me/public.ics"
        },
        "confidential": {
            "url": "https://example.cloud/dav/me/confidential.ics",
            "ca": "/etc/ssl/ca.pem",
            "auth": {
                "type": "tls",
                "keyfile": "/etc/ssl/client.pem",
                "passphrase": "mysupersecurepassword"
            }
        }
    },
    "key_replace": {
        "summary": "01_summary",
        "description": "02_description"
    },
    "value_replace": {
        "summary": "{{ summary|truncate(100) }}",
        "description": "{{ description|truncate(100) }}"
    }
}
```

### Explanation


| JMESPath | Type | Description |
|----------|------|-------------|
| `addr` | string | The address to listen on. |
| `port` | int | The port to listen on. |
| `start_delta` | string | A signed ISO 8601 duration string, describing the event range start offset relative to the current time. |
| `end_delta` | string | An unsigned ISO 8601 duration string, describing the event range end offset relative to the current time. |
| `tz` | string | The local timezone. |
| `calendars` | dict | The calendars to scrape. |
| `keys(calendars)` | string | Name of the calendar. |
| `calendars.*.url` | string | The HTTP or HTTPS URL to scrape. |
| `calendars.*.interval` | string | An unsigned ISO 8601 duration string, describing the scrape interval for this calendar. |
| `calendars.*.ca` | string | Path to the CA certificate file to validate the server's TLS certificate against, in PEM format (optional). |
| `calendars.*.auth` | dict | Authorization config for the calendar. |
| `calendars.*.auth[].type` | string | Authorization type, one of `none` (no authorization), `basic` (HTTP Basic Authentication), `tls` (TLS client certificate). |
| `calendars.*.auth[?type=='basic'].username` | string | The Basic Auth username to authenticate with. |
| `calendars.*.auth[?type=='basic'].password` | string | The Basic Auth password to authenticate with. |
| `calendars.*.auth[?type=='tls'].keyfile` | string | Path to the key file containing the TLS private key, client certificate and certificate chain, in PEM format. |
| `calendars.*.auth[?type=='tls'].passphrase` | string | Passphrase for the private key (optional). |
| `key_replace` | dict | Labels to rename, might be necessary e.g. for column ordering in Grafana. |
| `keys(key_replace)` | string | The labels to rename. |
| `key_replace.*` | string | The names to rename the labels to. |
| `value_replace` | dict | Label values to postprocess. |
| `keys(value_replace)` | string | Original label name to postprocess, also may introduce new labels. |
| `value_replace.*` | string | The new value for the label. Supports Jinja2 templates. All original label values can be accessed by using their original name as a Jinja2 variable name. |

## Queries

The most basic query is simply the metric name:

```
event
```

In addition, PromQL label filters can be used.

```
event{calendar="public",foo=~".*"}
```

## Why Prometheus API

- It's JSON. A JSON generator is builtin in Python, so no further dependency.
- The Prometheus Data Source is builtin into Grafana.
- The API is simple.