Welcome to my web log. See the first post for an introduction. See the archive page for all posts. (There is an english language feed if you don't want to see Finnish.)

Archives Tags Moderation policy Main site

All content outside of comments is copyrighted by Lars Wirzenius, and licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License. Comments are copyrighted by their authors. (No new comments are allowed.)


GDPR is a new EU regulation for privacy. The name is short for "General Data Protection Regulation" and it covers all organisations that handle personal data of EU citizens and EU residents. It will become enforceable May 25, 2018 (Towel Day). This will affect Debian. I think it's time for Debian to start working on compliance, mainly because the GDPR requires sensible things.

I'm not an expert on GDPR legislation, but here's my understanding of what we in Debian should do:

  • do a privacy impact assessment, to review and document what data we have, and collect, and what risks that has for the people whose personal data it is if the data leaks

  • only collect personal information for specific purposes, and only use the data for those purposes

  • get explicit consent from each person for all collection and use of their personal information; archive this consent (e.g., list subscription confirmations)

  • allow each person to get a copy of all the personal information we have about them, in a portable manner, and let them correct it if it's wrong

  • allow people to have their personal information erased

  • maybe appoint one or more data protection officers (not sure this is required for Debian)

There's more, but let's start with those.

I think Debian has at least the following systems that will need to be reviewed with regards to the GDPR:

  • db.debian.org - Debian project members, "Debian developers"
  • nm.debian.org
  • contributors.debian.org
  • lists.debian.org - at least membership lists, maybe archives
  • possibly irc servers and log files
  • mail server log files
  • web server log files
  • version control services and repositories

There may be more; these are just off the top of my head.

I expect that mostly Debian will be OK, but we can't just assume that.

Posted Tue Oct 10 18:08:00 2017 Tags:

How do you attract contributors to a new free software project?

I'm in the very early stages of a new personal project. It is irrelevant for this blog post what the new project actually is. Instead, I am thinking about the following question:

Do I want the project to be mainly for myself, and maybe a handful of others, or do I want to try to make it a more generally useful, possibly even a well-known, popular project? In other words, do I want to just solve a specific problem I have or try to solve it for a large group of people?

If it's a personal project, I'm all set. I can just start writing code. (In fact, I have.) If it's the latter, I'll need to attract contributions from others, and how do I do that?

I asked that question on Twitter and Mastodon and got several suggestions. This is a summary of those, with some editorialising from me.

  • The most important thing is probably that the project should aim for something that interests other people. The more people it interests, the easier it will be to attract contributors. This should be written up and displayed prominently: what does (or will) the software do and what can it e used for.

  • Having something that kind of works, and easy to improve, seems to also be key. An empty project is daunting to do anything with. Part of this is that the software the project is producing should be easy to install and get running. It doesn't have to be fully featured. It doesn't even have to be alpha level quality. It needs to do something.

    If the project is about producing a spell checker, say, and it doesn't even try to read an input file, it's probably too early for anyone else to contribute. A spell checker that lists every word in the input file as badly spelt is probably more attractive to contribute to.

  • It helps to document where a new contributor should start, and how they would submit their contribution. A list of easy things to work on may also help. Having a roadmap of near future developent steps and a long-term vision will make things easier. Having an architectural document to explain how the system hangs together will help.

  • A welcoming, constructive atmosphere helps. People should get quick feedback to questions, issues, patches, in order to build momentum. Make it fun for people to contibute, and they'll contribute more.

  • A public source code repository, and a public ticketing system, and public discussion forums (mailing lists, web forums, IRC channels, etc) will help.

  • Share the power in the project. Give others the power to make decisions, or merge things from other contributors. Having a clear, functioning governance structure from the start helps.

I don't know if these things are all correct, or that they're enough to grow a successful, popular project.

Karl Foger'l seminal book Producing Open Source Software should also be mentioned.

Posted Sun Oct 1 10:21:00 2017 Tags:

This is a difficult announcement to write. The summary is if you use Obnam you should switch to another backup program in the coming months.

The first commit to Obnam's current code base is this:

commit 7eaf5a44534ffa7f9c0b9a4e9ee98d312f2fcb14
Author: Lars Wirzenius <liw@iki.fi>
Date:   Wed Sep 6 18:35:52 2006 +0300

    Initial commit.

It's followed by over 5200 more commits until the latest one, which is from yesterday. The NEWS file contains 58 releases. There are 20761 lines of Python, 15384 words in the English language manual, with translations in German and French. The yarn test suite, which is a kind of a manual, is another 13382 words in English and pseudo-English. That's a fair bit of code and prose. Not all of it mine, I've had help from some wonderful people. But most of it mine.

I wrote all of that because backups were fun. It was pleasing to use my own program to guarantee the safety of my own data. The technical challenges of implmenting the kind of backup program I wanted were interesting, and solving interesting problems is a big part of why I am a programmer.

Obnam has a kind user base. It's not a large user base: the Debian "popularity contest" service estimates it at around 500. But it's a user base that is kind and has treated me well. I have tried to reciprocate.

Unfortunately, I have not had fun while developing Obnam for some time now. This has changed. A few years ago, I lived in Manchester, UK, and commuted by train to work. It was a short train ride, about 15 minutes. At times I would sit on the floor with my laptop on my knees, writing code or the manual. Back then Obnam was a lot of fun. I was excited, and enthusiastic.

In the past two years or so, I've not been able to feel that excitement again. My native language, Finnish, has an expression describing unpleasant tasks: something is as much fun as drinking tar. That describes Obnam in recent years for me.

Obnam has not turned out well, from a maintainability point of view. It seems that every time I try to fix something, I break something else. Usuaully what breaks is speed or memory use: Obnam gets slower or starts using even more memory.

For several years now I've been working on a new repository format for Obnam, code names GREEN ALBATROSS. It was meant to solve Obnam's problems as far as extensibility, performance, and resource use were concerned. It seems to have failed.

I'm afraid I've had enough. I'm going to retire Obnam as a project and as a program, and move on to doing something else, so I can feel excitement and pleasure again.

After some careful thought, I fear that the maintainability problems of Obnam can realistically only be solved by a complete rewrite from scratch, and I'm not up to doing that.

If you use Obnam, you should migrate to some other backup solution. Don't worry, you have until the end of the year. I will be around and I intend to fix any serious bugs in Obnam; in particular, security flaws. But you should start looking for a replacement sooner rather than later.

I will be asking Obnam to be removed from the Debian unstable and testing branches. The next Debian release (buster, Debian 10) won't include Obnam.

The Obnam mailing lists are kindly hosted by Daniel Silverstone, and they will remain, but later this year I will change them to be moderated. The Obnam git repository will remain. The web site will remain, but I will add a note that Obnam is no longer maintained. Other Obnam online resources may disappear.

If you would like to take over the Obnam project, and try to resolve the various issues, please contact me to discuss that.

Thank you, and may you never need to restore.

Posted Sun Aug 13 21:48:00 2017 Tags:

I realised recently that my laptop isn't set up to send TRIM or DISCARD commands to its SSD. That means the SSD firmware has a harder time doing garbage collection (see whe linked Wikipedia page for more details.)

After some searching, I found two articles by Christopher Smart: one, update. Those, plus some addition reading of documentation, and a little experimentation, allowed me to do this. Since the information is a bit scattered, here's the details, for Debian stretch, as much for my own memory as to make sure this is collected into one place.

  • Append ,discard to the fourth column on relevant lines in /etc/crypttab. For me, this means the fourth column should be luks,discard.
  • Change in /etc/lvm/lvm.conf that says issue_discards to enable it (assign 1 instead of 0).
  • Append rd.luks.options=discard to the GRUB_CMDLINE_LINUX_DEFAULT value in /etc/default/grub
  • Run sudo update-grub
  • Run sudo update-initramfs -u
  • Reboot.
  • Run sudo fstrim -av - if this works, you're good! If it gives you errors, then you get to debug. I have no idea what I'm talking about.
  • Copy /usr/share/doc/util-linux/examples/fstrim.* to /etc/systemd/system and run sudo systemctl enable fstrim.timer. This will tell systemd to run fstrim every week. (If you don't use systemd you'll have to adapt the systemd bits mentioned here. I've no idea how.)

Note that it seems to be a possible information leak to TRIM encryped devices. I don't know the details, but if that bothers you, don't do it.

I don't know of any harmful effects for enabling TRIM for everything, except the crypto bit above, so I wonder if it wouldn't make sense for the Debian installer to do this by default.

Posted Sat Aug 5 20:58:00 2017 Tags:

A couple of people objected to having Yakking on Planet Debian, so I've removed it.

Posted Wed Jul 19 08:53:00 2017 Tags:

In a case of blatant self-promotion, I am going to add the Yakking RSS feed to the Planet Debian aggregation. (But really because I think some of the readership of Planet Debian may be interested in the content.)

Yakking is a group blog by a few friends aimed at new free software contributors. From the front page description:

Welcome to Yakking.

This is a blog for topics relevant to someone new to free software development. We assume you are already familiar with computers, and are curious about participating in the production of free software. You don't need to be a programmer: software development requires a wide variety of skills, and you can be a valued core contributor to a project without being a programmer.

If anyone objects, please let me know.

Posted Thu Jul 13 13:01:00 2017

I've just released version 1.22 of Obnam, my backup application. It is the first release for this year. Packages are available on code.liw.fi/debian and in Debian unstable, and source is in git. A summary of the user-visible changes is below.

For those interested in living dangerously and accidentally on purpose deleting all their data, the link below shows that status and roadmap for FORMAT GREEN ALBATROSS. http://distix.obnam.org/obnam-dev/182bd772889544d5867e1a0ce4e76652.html

Version 1.22, released 2017-06-25

  • Lars Wirzenius made Obnam log the full text of an Obnam exception/error message with more than one line. In particular this applies to encryption error messages, which now log the gpg output.

  • Lars Wirzenius made obnam restore require absolute paths for files to be restored.

  • Lars Wirzenius made obnam forget use a little less memory. The amount depends on the number of genrations and the chunks they refer to.

  • Jan Niggemann updated the German translation of the Obnam manual to match recent changes in the English version.

  • SanskritFritz and Ian Cambell fixed the kdirstat plugin.

  • Lars Wirzenius changed Obnam to hide a Python stack trace when there's a problem with the SSH connection (e.g., failure to authenticate, or existing connection breaks).

  • Lars Wirzenius made the Green Albatross version of obnam forget actually free chunks that are no longer used.

Posted Sun Jun 25 15:40:00 2017 Tags:

tl;dr: Get vmdebootstrap replacement from http://git.liw.fi/vmdb2 and run it from the source tree. Tell me if something doesn't work. Send patches.

Many years ago I wrote vmdebootstrap, a tool for installing Debian on a disk image for virtual machines. I had a clear personal need: I was setting up a CI system and it needed six workers: one each for Debian oldstable, stable, and unstable, on two architectures (i386, amd64). Installing Debian six times in the same way is a lot of work, so I figured how difficult can it be to automate it. Turns out that not difficult at all, except to install a bootloader.

(Don't ask me why I didn't use any of the other tools for this. It was long ago, and while some of the tools that now exist probably did exist then, I like writing code and learning things while doing it.)

After a while I was happy with what the program did, but didn't want to upload it to Debian, and didn't want to add the kinds of things other people wanted, so I turned vmdebootstrap over to Neil Williams, who added a ton of new features. Unfortunately, it turned out that my initial architecture was not scaleable, and also the code I wrote wasn't very good, and there weren't any tests.

Neil did heroic work forcing my crappy software into doing things I never envisioned. Last year he needed a break and asked me to take vmdebootstrap back. I did, and have been hiding from the public eye ever since, since I was so ashamed of the code. (I created a new identity and pretended to be an international assassin and backup specialist, travelling the world forcing people to have at least one tested backup of their system. If you've noticed reports in the press about people reporting near-death experiences while holding a shiny new USB drive, that would've been my fault.)

Pop quiz: if you have a program with ten boolean options ("do this, except if that option is given, do the other thing"), how many black box tests do you need to test all the functionality? If one run of the program takes half an hour, how long will a full test suite run?

I did some hard thinking about vmdebootstrap, and came to the sad conclusion that it had reached the end of its useful life as a living software project. There was no reasonable way to add most of the additional functionality people were asking for, and even maintaining the current code was too tedious a task to consider seriously. It was time to make a clean break of the past and start over, without caring about backwards compatibility. After all, the old code wasn't going anywhere so anyone who needed it could still use it. There was no need to burden a new program with my past mistakes. All new mistakes were called for.

At the Cambridge mini-Debconf of November, 2016, I gave a short presentation of what I was going to do. I also posted about my plans to the debian-cloud list. In short, I would write a new, more flexible and cleaner replacement to be called vmdb2. For various personal reasons, I've not been able to spend as much time on vmdb2 as I'd like to, but I've now reached the point where I'd like to announce the first alpha version publically.

The source code is hosted here: http://git.liw.fi/vmdb2 . There are .deb packages at my personal public APT repo (http://liw.fi/code/), but vmdb2 is easy enough to run directly from a git checkout:

sudo ./vmdb2 foo.vmdb --output foo.img

There's no need to install it to try it.

What works:

  • vmdb2 can build a disk image with Debian installed, for amd64 only at this time
  • the boot loader is GRUB, either for UEFI or BIOS
  • the image boots under Qemu / KVM and also on actual hardware

What doesn't work:

  • other architecures, including building for a foreign archiecture
  • live CD building (no squashfs support)

I'm not opposed to adding support for those, but they're not directly interesting to me. For example, I only have amd64 machines. The best way to get support for additional features is to tell me how, preferably in the form of patches. (If I have to read tons of docs, or other people's code, and then write code and iterate while other people tell me it doesn't work, it's probably not happening.)

Why would you be interested in vmdb2? There's a lot of other tools to do what it does, so perhaps you shouldn't care. That's fine. I like writing tools for myself. But if this kind of tool is of interest to you, please do have a look.

A short tutorial: vmdb2 wants you to give it a "specification file" (conventionally suffixed .vmdb, because someone stole the .spec suffix, but vmdb2 doesn't care about the name). Below is an example. vmdb2 image specification files are in YAML, since I like YAML, and specify a sequence of steps to take to build the image. Each step is a tiny piece of self-contained functionality provided by a plugin.

steps:
  - mkimg: "{{ output }}"
    size: 4G

  - mklabel: msdos
    device: "{{ output }}"

  - mkpart: primary
    device: "{{ output }}"
    start: 0%
    end: 100%
    part-tag: root-part

The above create an image (name is specified with the --output option), which is four gigabytes in size, and create a partitition table and a single partition that fills the whole disk. The "tag" is given so that later steps can easily refer to the partition.

If you prefer another way to partition the disk, you can achieve that by adding more "mkpart" steps. For example, for UEFI you'll want to have an EFI partition.

  - mkfs: ext4
    partition: root-part

  - mount: root-part
    fs-tag: root-fs

The above formats the partition with the ext4 filesystem, and then mounts it. The mount point will be a temporary directory created by vmdb2, and a tag is again given to the mount point so it can be referred to.

  - unpack-rootfs: root-fs

The above unpacks a tar archive to put content into the filesystem, if the tar archive exists. The tar archive is specified with the --rootfs-tarball command line option.

  - debootstrap: stretch
    mirror: http://http.debian.net/debian
    target: root-fs
    unless: rootfs_unpacked

  - apt: linux-image-amd64
    fs-tag: root-fs
    unless: rootfs_unpacked

  - cache-rootfs: root-fs
    unless: rootfs_unpacked

The above will run debootstrap and install a kernel into the filesystem, but skip doing that if the rootfs tarball was used. Also, the tarball is created if it didn't exist. This way the tarball is used by all but the first run, which saves a bit of time. On my laptop and with a local mirror, debootstrap and kernel installation takes on the order of nine minutes (500 to 600 seconds), whereas unpacking the tar archive is a bit faster (takes around 30 seconds). When iterating over things other than debootstrap, this speeds things up something wonderful, and seems worth the complexity.

The "unless:" mechanism is generic. All the steps share some state, and the unpack-rootfs step sets the "rootfs_unpacked" flag in the shared state. The "unless:" field tells vmdb2 to check for the flag and if it is not set, or if it is set to false ("unless it is set to true"), vmdb2 will execute the step. vmdb2 may get more such flags in the future, if there's need.

  - chroot: root-fs
    shell: |
      sed -i '/^root:[^:]*:/s//root::/' /etc/passwd
      echo pc-vmdb2 > /etc/hostname

The above executes a couple of shell commands in a chroot of the root filesystem we've just created. In this case they remove a login password from root, and set the hostname. This is a replacement of the vmdebootstrap "customize" script, but it can be inserted anywhere into the sequence of steps. There's boot chroot and non-chroot variants of the step.

This is a good point to mention that writing customize scripts gets quite repetitive and tedious after a while, so vmdb2 has a plugin to run Ansible instead. You can customize your image with that instead, while the image is being built and not have to wait until you boot the image and running Ansible over ssh.

  - grub: bios
    root-fs: root-fs
    root-part: root-part
    device: "{{ output }}"
    console: serial

Finally, install a boot loader, grub. This shows the BIOS variant, UEFI is also supported. This also configures grub and the kernel to use a serial console. There's a "yarn" (test suite) to build and smoke test an image with vmdb2 to make sure at least the basic functionality works. The smoke test boots the image under Qemu, logs in as root, and tells the VM to power off. Very, very basic, but has already found actual bugs in vmdb2. The smoke test needs the serial console to work.

As with vmdebootstrap originally, I don't particularly want to maintain the package in Debian. I've added Debian packaging (so that I can install it on my own machines), but I already have enough packages to maintain, so I'm hoping someone else will volunteer to take on the Debian maintainership and bug handling duties.

If you would like vmdb2 to do more things to suit you better, I'm happy to explain how to write plugins to provide more types of steps.

If you are currently using vmdebootstrap, either directly or as part of another tool, I encourage you to have a look at vmdb2. In the long term, I would like to retire vmdebootstrap entirely, once vmdb2 can do everything vmdebootstrap can do, and few people use vmdebootstrap. This may take a while.

In any case, whether you want a new image building tool or not, happy hacking.

Posted Sun Jun 4 15:08:00 2017 Tags:

Introduction

I've written before about using a U2F key with PAM. This post continues the theme and explains how to use a smartcard with GnuPG for storing OpenPGP private keys. Specifically, a Yubikey 4 card, because that's what I have, but any good GnuPG compatible card should work. The Yubikey is both a GnuPG compatible smart card, and a U2F card. The Yubikey 4 can handle keys up to 4096 bits. Older Yubikeys can only handle keys up to 2095 bits.

The reason to do this is to make it harder for an attacker to steal your encryption keys.

I will assume you don't already have an OpenPGP key, or are willing to generate a new one. I will also assume you run Debian stretch; some of the desktop environment setup details may differ between Debian versions or between Linux distributions. You will need:

  • A Yubikey 4 (or other GnuPG compatible smartcard).
  • Two USB memory sticks to store master copies of the key you create.
  • Either a lot of patience, or a ChaosKey or something else to generate a lot of entropy for the kernel. Entropy is used by GnuPG to create encryption keys.

Terminology

Some terminology:

  • OpenPGP is a standard of encryption keys.

  • GnuPG (also known as GPG or gpg) is an encryption program that supports OpenPGP.

  • Encryption key or key pair is a secret and a public key for encryption. Key and key pair are often used as synonyms.

  • The secret keys is yours, and only you will have it or be able to you use. The public key is public, anyone can have a copy. The two are linked and you can't create a secret key linked to a public key. (If you can, you will become a famous cryptographer. Like Isaac Newton famous among apple growers.)

  • Master key is the important thing to keep track of. In the encryption world, the master key is what identifies you.

  • A subkey is a secondary key or key pair for encryption, derived from the master key. Subkeys can be created and revoked almost at will, and there can be any number of them. Subkeys are always associated with a single master key.

  • Subkeys can be dedicated for encryption, signing, and authentication. You can have one of each.

  • Encryption is the process of taking a file and making it illegible to everyone except the owner of a secret key. When the public key is used to encrypt, only the secret key can decrypt.

  • Signing is using a secret key to make a separate file that others can verify using the corresponding public key. This means the signer can "prove" they have the file. For example, you might sign emails to prove they came from you.

  • Authentication is the process of proving you're you. Typically, for example, a server will know your public key, and will give a large random number encrypted with your public key. Only you have the corresponding secret key, so only you can send the random number back to the server, and thereby the server knows you're you.

Outline

The process outline is:

  1. Create a new, signing-only master key with GnuPG.

  2. Create three "subkeys", one each for encryption, signing, and authentication. These subkeys are what everyone else uses.

  3. Export copies of the master key pair and the subkey pairs and put them in a safe place.

  4. Put the subkeys on the Yubikey.

  5. GnuPG will automatically use the keys from the card. You have to have the card plugged into a USB port for things to work. If someone steals your laptop, they won't get the private subkeys. Even if they steal your Yubikey, they won't get them (the smartcard is physically designed to prevent that), and can't even use them (because there's PIN codes or passphrases and getting them wrong several times locks up the smartcard).

  6. Use gpg-agent as your SSH agent, and the authentication-only subkey on the Yubikey is used as your ssh key.

Configure GnuPG

The process in more detail:

  • Configure GnuPG with regards to checksum and encryption algorithms. You can use the defaults, but depending on the version of GnuPG you have, they may be weaker than is recommended. These values are from Riseup.net OpenPGP guide, see link at the end.

    Add the following lines to ~/.gnupg/gpg.conf:

      personal-digest-preferences SHA512
      cert-digest-algo SHA512
      default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed
      personal-cipher-preferences TWOFISH CAMELLIA256 AES 3DES
      keyserver pool.sks-keyservers.net
    

Create new keys

  • Create new sign-only master key.
$ gpg --full-generate-key
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
 
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
       = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:43:54 PM EEST
Is this correct? (y/N) y

GnuPG needs to construct a user ID to identify your key.

Real name: Lars Wirzenius
Email address: liw@liw.fi
Comment: test key
You selected this USER-ID:
    "Lars Wirzenius (test key) <liw@liw.fi>>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: key 25FB738D6EE435F7 marked as ultimately trusted
gpg: directory '/home/liw/.gnupg/openpgp-revocs.d' created
gpg: revocation certificate stored as '/home/liw/.gnupg/openpgp-revocs.d/A734C10BF2DF39D19DC0F6C025FB738D6EE435F7.rev'
public and secret key created and signed.

Note that this key cannot be used for encryption. You may want to use
the command "--edit-key" to generate a subkey for this purpose.
pub rsa4096 2017-05-29 [SC] [expires: 2018-05-29]
      A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
      A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
uid Lars Wirzenius (test key) <liw@liw.fi>
  • Note that I set a 1-year expiration for they key. The expiration can be extended at any time (if you have the master secret key), but unless you do, the key won't accidentally live longer than the chosen time.

  • Review the key:

$ gpg --list-secret-keys
/home/liw/.gnupg/pubring.kbx
----------------------------
sec rsa4096 2017-05-29 [SC] [expires: 2018-05-29]
      A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
uid [ultimate] Lars Wirzenius (test key) <liw@liw.fi>
  • You now have the signing-only master key. You should now create three subkeys (keyid is the key identifier shown in the key listing, A734C10BF2DF39D19DC0F6C025FB738D6EE435F7 above). Use the --expert option to be able to add an authentication-only subkey.
$ gpg --edit-key --expert A734C10BF2DF39D19DC0F6C025FB738D6EE435F7z
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

sec rsa4096/25FB738D6EE435F7
     created: 2017-05-29 expires: 2018-05-29 usage: SC
     trust: ultimate validity: ultimate
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> addkey
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
  (10) ECC (sign only)
  (11) ECC (set your own capabilities)
  (12) ECC (encrypt only)
  (13) Existing key
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
       = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:44:52 PM EEST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

sec rsa4096/25FB738D6EE435F7
     created: 2017-05-29 expires: 2018-05-29 usage: SC
     trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
      created: 2017-05-29 expires: 2018-05-29 usage: S
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> addkey
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
  (10) ECC (sign only)
  (11) ECC (set your own capabilities)
  (12) ECC (encrypt only)
  (13) Existing key
Your selection? 6
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:45:22 PM EEST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

sec rsa4096/25FB738D6EE435F7
     created: 2017-05-29 expires: 2018-05-29 usage: SC
     trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
      created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
      created: 2017-05-29 expires: 2018-05-29 usage: E
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> addkey
Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
  (10) ECC (sign only)
  (11) ECC (set your own capabilities)
  (12) ECC (encrypt only)
  (13) Existing key
Your selection? 8

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Sign Encrypt

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? a

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Sign Encrypt Authenticate

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? s

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Encrypt Authenticate

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? e

Possible actions for a RSA key: Sign Encrypt Authenticate
Current allowed actions: Authenticate

   (S) Toggle the sign capability
   (E) Toggle the encrypt capability
   (A) Toggle the authenticate capability
   (Q) Finished

Your selection? q
RSA keys may be btween 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
       = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tue 29 May 2018 06:45:56 PM EEST
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

sec rsa4096/25FB738D6EE435F7
     created: 2017-05-29 expires: 2018-05-29 usage: SC
     trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
     created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
     created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
     created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> save

Export secret keys to files, make a backup

  • You now have a master key and three subkeys. They are hidden in the ~/.gnupg directory. It is time to "export" the secret keys out from there.
$ gpg --export-secret-key --armor keyid > master.key
$ gpg --export-secret-subkeys --armor keyid > subkeys.key
  • You should keep these files safe. You don't want to lose them, and you don't want anyone else to get access to them. I recommend you format two USB memory sticks, format them using full-disk encryption, and copy the exported files to both of them. Then keep them somewhere safe.

    There's ways of making this part more sophisticated, but that's for another time.

  • The next step involves some hoop-jumping. What we want is to have the master secret key NOT on you machine, so we tell GnuPG to remove it. We exported it above, so we won't lose it. However, deleting the master secret key also removes the secret subkeys. But we can import those without importing the master secret key.

$ gpg --delete-secret-key keyid
$ gpg --import subkeys.key
  • Now verify that you have the secret subkeys, but not the master key. There should be one line starting with sec# (note the hash mark, which indicates the key isn't available), and three lines starting with ssb (no hash mark).
$ gpg -K
/home/liw/.gnupg/pubring.kbx
----------------------------
sec# rsa4096 2017-05-29 [SC] [expires: 2018-05-29]
      A734C10BF2DF39D19DC0F6C025FB738D6EE435F7
uid [ultimate] Lars Wirzenius (test key) <liw@liw.fi>
ssb rsa4096 2017-05-29 [S] [expires: 2018-05-29]
ssb rsa4096 2017-05-29 [E] [expires: 2018-05-29]
ssb rsa4096 2017-05-29 [A] [expires: 2018-05-29]

Install subkeys on a Yubikey

  • Now insert the Yubikey in a USB slot. We can start transferring the secret subkeys to the Yubikey. If you want, you can set your name and other information, and change PIN codes. There's several types of PIN codes: normal use, unblocking a locked card, and a third PIN code for admin operations. Changing the PIN codes is a good idea, otherwise everyone will just try the default of 123456 (admin 12345678). However, I'm skipping that in the interest of brevity.
$ gpg -card-edit
...
  • Actually move the subkeys to the card. Note that this does a move, not a copy, and the subkeys will be removed from your ~/.gnupg (check with gpg -K).
$ gpg --edit-key liw
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 1

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb* rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> keytocard
Please select where to store the key:
(1) Signature key
(3) Authentication key
Your selection? 1

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb* rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 1

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 2

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb* rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> keytocard
Please select where to store the key:
(2) Encryption key
Your selection? 2

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb* rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 2

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> key 3

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb* rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> keytocard
Please select where to store the key:
(3) Authentication key
Your selection? 3

pub rsa4096/25FB738D6EE435F7
created: 2017-05-29 expires: 2018-05-29 usage: SC
trust: ultimate validity: ultimate
ssb rsa4096/05F88308DFB71774
created: 2017-05-29 expires: 2018-05-29 usage: S
ssb rsa4096/2929E8A96CBA57C7
created: 2017-05-29 expires: 2018-05-29 usage: E
ssb* rsa4096/4477EB0AEF1C440A
created: 2017-05-29 expires: 2018-05-29 usage: A
[ultimate] (1). Lars Wirzenius (test key) <liw@liw.fi>

gpg> save
  • If you want to use several Yubikeys, or have a spare one just in case, repeat the previous four steps (starting from importing subkeys back into ~/.gnupg).

  • You're now done, as far GnuPG use is concerned. Any time you need to sign, encrypt, or decrypt something, GnuPG will look for your subkeys on the Yubikey, and will tell you to insert it in a USB port if it can't find the key.

Use subkey on Yubikey as your SSH key

  • To actually use the authentication-only subkey on the Yubikey for ssh, you need to configure your system to use gpg-agent as the SSH agent. Add the following line to .gnupg/gpg-agent.conf:

     enable-ssh-support
    
  • On a Debian stretch system with GNOME, edit /etc/xdg/autostart/gnome-keyring-ssh.desktop to have the following line, to prevent the GNOME ssh agent from starting up:

     Hidden=true
    
  • Edit /etc/X11/Xsession.options and remove or comment out the line that says use-ssh-agent. This stops a system-started ssh-agent from being started when the desktop start.

  • Create the file ~/.config/autostart/gpg-agent.desktop with the following content:

     [Desktop Entry]
     Type=Application
     Name=gpg-agent
     Comment=gpg-agent
     Exec=/usr/bin/gpg-agent --daemon
     OnlyShowIn=GNOME;Unity;MATE;
     X-GNOME-Autostart-Phase=PreDisplayServer
     X-GNOME-AutoRestart=false
     X-GNOME-Autostart-Notify=true
     X-GNOME-Bugzilla-Bugzilla=GNOME
     X-GNOME-Bugzilla-Product=gnome-keyring
     X-GNOME-Bugzilla-Component=general
     X-GNOME-Bugzilla-Version=3.20.0
    
  • To test, log out, and back in again, run the following in a terminal:

$ ssh-add -l

The output should contain a line that looks like this:

    4096 SHA256:PDCzyQPpd9tiWsELM8LwaLBsMDMm42J8/eEfezNgnVc cardno:000604626953 (RSA)
  • You need to export the authentication-only subkey in the SSH key format. You need this for adding to .ssh/authorized_keys, if nothing else.
$ gpg --export-ssh-key keyid > ssh.pub
  • Happy hacking.

See also

See also the following links. I've used them to learn enough to write the above.

Edited to fix:

  • Output of gpg -K after removing secret master key.
Posted Mon May 29 18:03:00 2017 Tags:

Distix is my distributed ticketing system. I initially wrote the core of it as a bit of programming performance art, to celebrate my 30 years as a programmer. Distix is built on top of git and emails in Maildirs. It is a silent listener to your issue and bug discussions: as long as you ensure it gets a copy of each mail, it takes care of automatically arranging things into separate tickets based on email threading. Users and customers do not need to even know Distix is being used. Only the "support staff" need ever interact with Distix, and they mostly only need to close tickets that have been dealt with.

I've been using Distix for my own stuff for some time now, and recently we've started using it at work. I slowly improve it as we find problems.

It's not a sleek, smooth, finished tool. It's clunky, weird, and probably not what you want. But it's what I want.

Changes in recent months:

  • There is a new website: http://distix.eu/. No particular good reason for a new website, but I won the domain for free a couple of years ago, so I might as well use it.

  • In addition, a ticketing system for Distix itself: http://tickets.distix.eu/. Possibly I should've called the subdomain dogfood, but I'm a serious person, not prone to trying to be funny.

  • Mails can now be imported using IMAP.

  • Importing has been optimized for speed and memory use, making my own production use more practical.

I've discussed with a friend the possibility of writing a web UI, and some day maybe that will happen. For now, distix is a command line applicaton that can generate a static HTML site.

Posted Sat May 27 11:07:00 2017 Tags: