easywks/README.md

# EasyWKS

### OpenPGP WKS for Human Beings

---

## What is WKD/WKS?

Due to all the issues involved with the PGP key servers we're using today,  GnuPG introduced a feature named [**Web Key
Directory**][wkd] (WKD):  Instead of searching for keys on the usual key servers, WKD is taking a **fully**
decentralized and federated approach, where each mail domain is responsible for hosting its users public keys on an
HTTPS web directory.  For example, in order to retrieve the key for `john.doe@example.org`, they key can be located at

    https://openpgpkey.example.org/.well-known/openpgpkey/example.org/hu/iy9q119eutrkn8s1mk4r39qejnbu3n5q?l=john.doe

In order to get the keys there, GnuPG developed an email-based protocol named [**Web Key Service**][wks] (WKS), which
lets users publish their public keys once they have proven ownership of the key using an email-based challenge-responce
mechanism.

At the time, WKS and WKD aren't yet part of the OpenPGP standard, but there's an [IETF Draft RFC][ietf] that's being
updated from time to time.

## Why EasyWKS?

I've experienced the WKS standard to be extremely cumbersome to conform to.  The tools shipped with GnuPG,
`gpg-wks-server` and `gpg-wks-client` are usable well enough as long as you have shell access on a mailserver and pipe
the output of `gpg-wks-client` into `sendmail`.  However, I'm usually doing my email via SMTP using a variety of
clients.

No matter which client I tried, I did not manage to get them to send mails in the format required by `gpg-wks-server`.
Even pasting the output of `gpg-wks-client` into `openssl s_client` after manually performing SMTP auth proved to be
difficult in case any line of the PGP-encrypted message happened to start with `Q` or `R`.

So I decided to write a WKS server that's much more lenient regarding the exact format of the mails it receives.
Instead of enforcing the strict format mandated by the standard, EasyWKS only requires:

- For the initial submission request: An unsigned & encrypted PGP/MIME message anywhere in the MIME tree, and the
    ASCII-armored PGP-Key anywhere inside the encrypted message.
- For the confirmation response: A signed & encrypted PGP/MIME message anywhere in the MIME tree, and the
    confirmation response anywhere inside the encrypted message.

This makes EasyWKS usable with every mail client, no matter whether WKS support is built-in or not.

EasyWKS aims to be a drop-in replacement for gpg-wks-server; see "Bootstrapping" below to learn how to migrate from
gpg-wks-server to EasyWKS.

## Requirements

- Python 3.6 or newer
- PyYAML
- bottle.py
- PGPy
- dnspython (for DANE support)
- Twisted (for DANE support)

## License

MIT

## Setup

### Installation

TODO

Create a cronjob, e.g. in `/etc/cron.d/easywks`

```crontab
0 3 * * *    webkey    /path/to/easywks clean
```

### Configuration

Configuration is done in `/etc/easywks.yml` (or any other place as specified by `--config`)

```yaml
---
# EasyWKS works inside this directory.  Its PGP keys as well as all
# the submitted and published keys are stored here.
directory: /var/lib/easywks

# Number of seconds after which a pending submission request is
# considered stale and should be removed by easywks clean.
pending_lifetime: 604800

# Some clients (including recent versions of gpg-wks-client follow an
# older version of the WKS standard where signing the confirmation
# response is only recommended, but not required.  Set this option to
# true if you want to accept such unsigned responses.
permit_unsigned_response: false

# Port configuration for the webserver.  Put this behind an
# HTTPS-terminating reverse proxy!
httpd:
  host: 127.0.0.1
  port: 8080
  # Some older HTTP clients omit the ?l=<userid> query suffix.  Set
  # this to false in order to permit such clients to retrieve keys.
  #require_user_urlparam: true

# Defaults to stdout, supported: stdout, smtp
mailing_method: smtp

# Configure smtp client options
smtp:
  # Connect to this SMTP server to send a mail.
  host: localhost
  port: 25
  # if tls=True, starttls is ignored
  tls: false
  starttls: false
  # Omit username/password if authentication is not needed.
  username: webkey
  password: SuperS3curePassword123

# Configure the LMTP server
lmtpd:
  host: "::1"
  port: 8024

# Configure the authoritative DNS server for DANE zones
dnsd:
  host: "::1"
  port: 8053

# You can override the mail response templates with your own text.
# The following templates can be overridden:
#   - "header": Placed in front of every message.
#   - "footer": Appended to every message.
#   - "confirm": Sent with the confirmation request.
#   - "done": Sent after a key was published.
#   - "error": Sent when an error occurs.
# The following placeholders can be used (enclosed in curly braces):
#   - {domain}: The email domain for with the request is processed.
#   - {sender}: The submitter's mail address.
#   - {submission}: The submission address.
#  When overriding the "error" template, there's an additional
#  placeholder you can use:
#   - {error}:  The error message.
#responses:
#  error: |
#    An error has occurred while processing your request:
#
#    {error}
#
#    If this error persists, please contact admin@example.org for help.

# Every domain served by EasyWKS must be listed here
domains:
  example.org:
    # Users send their requests to this address.  It's up to you to
    # make sure that the mails sent their get handed to EasyWKS.
    submission_address: webkey@example.org
    # If you want the PGP key for this domain to be password-
    # protected, or if you're supplying your own password-protected
    # key, set the passphrase here:
    passphrase: "Correct Horse Battery Staple"
  # Defaults are gpgwks@<domain> and no password protection.
  example.com: {}
```

### Bootstrapping

Run `easywks init` (as the correct user) to initialize all files and directories.  This will also generate a PGP key for
each domain, stored in `<workdir>/<domain>/key.pgp`.  If you want to, you can replace this key by a key you generated
yourself.  If your key is password-protected, you have to supply the passphrase in the config file or remove the
password protection.

If you are migrating from gpg-wks-server to EasyWKS, you can point EasyWKS to gpg-wks-server's working directory
(usually `/var/lib/gnupg/wks`).  EasyWKS uses the same directory layout and file formats as gpg-wks-server, so it should
be able to take over where gpg-wks-server stopped.  The only thing you need to do is to export the private keys from the
GnuPG keyring and write them to their domain's `key.pgp`.

### Webserver Setup

There are generally two ways to get WKD working:

- The gpg-wks-server approach, i.e. regularly copying the `hu` directories from EasyWKS' working directory to the 
    webroot.
- Running the EasyWKS web server behind a Reverse Proxy, e.g. using the follwing systemd unit:

```unit file (systemd)
[Unit]
Description=OpenPGP WKS for Human Beings - HTTP Server

[Service]
Type=simple
ExecStart=/path/to/easywks webserver
Restart=on-failure
User=webkey
Group=webkey
WorkingDirectory=/var/lib/easywks

[Install]
WantedBy=multi-user.target
```

Once this service is up and running, you can configure your webserver to proxy requests to EasyWKS.  E.g. with Apache2:

```
ProxyPass /.well-known/openpgpkey/ http://127.0.0.1:8080/.well-known/openpgpkey/
ProxyPassReverse /.well-known/openpgpkey/ http://127.0.0.1:8080/.well-known/openpgpkey/
```

Note that the webserver built into EasyWKS validates that the `?l=<uid>` matches the uid hash (the last URL component).

### MTA Setup

#### Postfix

You can either connect EasyWKS using a pipe transport or as a LMTP server.

##### Pipe Transport

Add an entry in `/etc/postfix/master.cf`:

```
webkey  unix  -       n       n       -       -       pipe
  flags=DRhu user=webkey argv=/path/to/easywks process
```

Then tell postfix to forward mails to the submission addresses to said transport, e.g. in `/etc/postfix/transport`:

```
# The colon at the end is important, it lets postfix know
# that "webkey" is a transport, not a name
gpgwks@example.org   webkey:
webkey@example.com   webkey:
```

Configure EasyWKS to send outgoing mails via SMTP.

##### LMTP Server

Configure EasyWKS to run the LMTP server, e.g. using the following systemd unit:

```unit file (systemd)
[Unit]
Description=OpenPGP WKS for Human Beings - LMTP Server

[Service]
Type=simple
ExecStart=/path/to/easywks lmtpd
Restart=on-failure
User=webkey
Group=webkey
WorkingDirectory=/var/lib/easywks

[Install]
WantedBy=multi-user.target
```

Also configure EasyWKS to send outgoing mails via SMTP.

Then tell postfix to forward mails to the submission addresses to said transport, e.g. in `/etc/postfix/transport`:

```
gpgwks@example.org   lmtp:localhost:10024
webkey@example.com   lmtp:localhost:10024
```

### DANE DNS Setup

Apart from WKD, EasyWKS can also serve PGP keys using RFC7929 DNS records ("OPENPGPKEY" or "TYPE61" records).  However,
since EasyWKS does not implement DNSSEC signing, it cannot do this alone.  The authoritative DNS server in EasyWKS only
responds to AXFR zone transfer requests.  In order for DANE lookups to work, the zones must be replicated (AXFR'd) by an
authoritative secondary nameserver that signs the zones itself.

#### EasyWKS DNS Server

Configure EasyWKS to run the DNS server, e.g. using the following systemd unit:

```unit file (systemd)
[Unit]
Description=OpenPGP WKS for Human Beings - DANE DNS Server

[Service]
Type=simple
ExecStart=/path/to/easywks dnsd
Restart=on-failure
User=webkey
Group=webkey
WorkingDirectory=/var/lib/easywks

[Install]
WantedBy=multi-user.target
```

If you're using EasyWKS' DANE feature, it is highly recommended to configure the SOA and NS records for each domain
you're serving.  Generally you want to add NS records for all nameservers that will be serving your zone, and at least
set the MNAME and RNAME components of the SOA record:

```yaml
domains:
  example.org:
    ns:
      - ns1.example.org.
      - ns2.example.org.
      - ns1.example.com.
      - ns2.example.com.
    soa:
      mname: ns1.example.org.
      rname: dnsadmin.example.org.
      refresh: 300
      retry: 60
      expire: 1209600
      minimal: 300
```

#### Knot

Knot is an authoritative nameserver that supports signing a replicated zone by the secondary (replicating) nameserver.

To configure Knot to transfer the zone from EasyWKS, set up an EasyWKS remote and use it as the replication master for
DANE zones.  DNSSEC signing must be enabled as well:

```yaml
remote:
  - id: remote-easywks.example.org
    address: [::1]@10053

zone:
  - domain: _openpgpkey.example.org
    master: remote-easywks.example.org
    dnssec-signing: on
    dnssec-policy: ...
```

## EasyWKS Client

The file `client.py` contains a self-contained WKS client, which
prompts you for your email address and IMAP/SMTP/POP3 password, and
then attempts to figure out the mail servers via common
autoconfiguration methods.  Afterwards it will attempt a WKS key submission:

```console?prompt=$, 
$ ./client.py
Enter email: john.doe@example.org
Chose A58D3221F8079F35FF084890505A563492A56583
Enter IMAP/POP3/SMTP password (will not echo): ********
Autoconfigured incoming server: imaps://john.doe@example.org@imap.example.org:993
Autoconfigured outgoing server: smtp+starttls://john.doe@example.org@smtp.example.org:587
Please confirm: [Y/n] y
Retrieved submission key
Retrieved key to publish
Created encrypted message
Sending submission request
Awaiting response
Received confirmation request
Nonce: 95184efbc5d2f75ed4b56162
Creating confirmation response. GnuPG may prompt you for your passphrase.
Sending confirmation response
Awaiting publish response
Decrypting WKS response. GnuPG may prompt you for your passphrase.

Hi there!

This is the EasyWKS system at example.org

Your key has been published to the Web Key Directory.
You can test WKD key retrieval e.g. with:

gpg --auto-key-locate=wkd,nodefault --locate-key john.doe@example.org

For more information on WKD and WKS see:

  https://gnupg.org/faq/wkd.html
  https://gnupg.org/faq/wks.html


Regards
EasyWKS

--
Dance like nobody is watching.
Encrypt live everybody is.

```

## Manual Key Import

In addition to WKS, EasyWKS also provides a command line interface for
importing keys from standard input.  This feature is mainly intended
to be used for technical email accounts where using WKS might prove to
be difficult:

```console?prompt=$, 
$ cat pubkey.asc | easywks import
Skipping foreign email john.doe@notmydepartment.org
Imported key A58D3221F8079F35FF084890505A563492A56583 for email john.doe@example.org
```

[wkd]: https://wiki.gnupg.org/WKD
[wks]: https://wiki.gnupg.org/WKS
[ietf]: https://datatracker.ietf.org/doc/html/draft-koch-openpgp-webkey-service-12