Welcome to my web log. See the first post for an introduction. See the archive page for all posts, and comments for a feed of comments only. (There is an english language feed if you don't want to see Finnish.)

Lars Wirzenius

There is no common way to express license information in each source file at the moment. Some people embed license information in each file, others keep it in README or another file at the top of the source tree.

Worse, there is no common syntax to express the license information in a machine-parseable way. If we had this, we could have tools that, for example, tell you if you're trying to merge code that has an incompatible license.

Obviously, this kind of thing can never work perfectly. People keep inventing new licenses, and it is not possible for a computer program to fully understand any license. It is not clear humans can do that, either. However, it would be possible to do it to a number of well-known licenses, which would help most of the time. A classic 20/80 situation.

I would like to suggest a syntax, similar to Emacs's "Hey Emacs" modelines, for embedding a summary of the license, or licenses, for a source file, in such a way that it can be programmatically extracted and parsed and analysed. With this syntax, one could then write a tool to ensure that all files in a project have the same license, or that all licenses are compatible with the project's overall license.

Such a tool will, of course, rely on heuristics and assumptions. For example, it needs to assume the machine-parseable license summary is correct, and rely on a ruleset on what licenses are compatible with what licenses. Things can go wrong. That's life. Remember, this is aiming at doing the 20% of the work that will work 80% of the time, not perfection.

I don't have a tool written, but I have a suggestion for the syntax.

/*
 * Copyright 2013 Lars Wirzenius
 *
 * tl;dr =*= Licenses: GPL-3+ or Expat, and Artistic =*=
 *
 * Blah. Blah. Blah. Imagine long, boring license texts 
 * here.
 */

The important part is this:

=*= Licenses: GPL-3+ or Expat, and Artistic =*=

The =*= prefix and suffix and the word Licenses are there to make grepping reasonably reliable without too many false positives, and to allow comment characters and other text on the same line.

The actual license summary follows the syntax and semantics of the Debian copyright-format 1.0 specification, which I chose because it exists and has had a fair bit of review so far, and is reasonably expressive.

The license summaries can be extracted with the following GNU sed invocation:

sed -n '/.*=\*= [Ll]icen[cs]es\?: \(.*\)=\*=.*/s//\1/p'

I allowed various forms of the word license, since it's a word that a lot of people will get wrong, and it's easy to catch all four common forms.

So, does anyone else think this might be useful? Would you use it in your own projects?

Posted Wed Apr 10 17:46:14 2013 Tags:

I've made two new designs for Trunk Tees, my Cafepress store.

Thank you to Richard Braakman for suggesting the .* one.

Here's all the older designs as well:

Posted Sun Mar 24 14:55:23 2013 Tags:

I've just pushed out the release files for Obnam version 1.4, my backup application, and Larch, my B-tree library, which Obnam uses. They are available via my home page (http://liw.fi/). Since Debian is frozen, I am not uploading packages to Debian, but .deb files are available from my personal apt repository for intrepid explorers. (I will be uploading to Debian again after the freeze. I am afraid I'm too lazy to upload to experimental, or do backports. Help is welcome!)

From the Obnam NEWS file:

  • The`ls command now takes filenames as (optional) arguments, instead of a list of generations. Based on patch by Damien Couroussé.
  • Even more detailed progress reporting during a backup.
  • Add --fsck-skip-generations option to tell fsck to not check any generation metadata.
  • The default log level is now INFO, instead of DEBUG. This is to be considered a quantum leap in the continuing rise of the maturity level of the software. (Actually, the change is there just to save some disk space and I/O for people who don't want to be involved in Obnam development and don't want to have massive log files.)
  • The default sizes for the lru-size and upload-queue-size settings have been reduced, to reduce the memory impact of Obnam.
  • obnam restore now reports transfer statistics at the end, similarly to what obnam backup does. Suggested by "S. B.".

Bug fixes:

  • If listing extended attributes for a filesystem that does not support them, Obnam no longer crashes, just silently does not backup extended attributes. Which aren't there anyway.
  • A bug in handling stat lookup errors was fixed. Reported by Peter Palfrader. Symptom: AttributeError: 'exceptions.OSError' object has no attribute 'st_ino' in an error message or log file.
  • A bug in a restore crashing when failing to set extended attributes on the restored file was fixed. Reported by "S. B.".
  • Made it clearer what is happening when unlocking the repository due to errors, and fixed it so that a failure to unlock is also an error. Reported by andrewsh.
  • The dependency on Larch is now for 1.20121216 or newer, since that is needed for fsck to work.
  • The manual page did not document the client name arguments to the add-key and remove-key subcommands. Reported by Lars Kruse.
  • Restoring symlinks as root would fail. Reported and fixed by David Fries.
  • Only set ssh user/port if explicitily requested, otherwise let ssh select them. Reported by Michael Goetze, fixed by David Fries.
  • Fix problem with old version of paramiko and chdir. Fixed by Nick Altmann.
  • Fix problems with signed vs unsigned values for struct stat fields. Reported by Henning Verbeek.
Posted Sat Mar 16 19:28:19 2013 Tags:

I've just released, to code.liw.fi, version 1.20130313 of cliapp, my Python framework for Unix-like command line programs. It contains the typical stuff such programs need to do, such as parsing the command line for options, and iterating over input files.

Version 1.20130313

  • Add cliapp.Application.compute_setting_values method. This allows the application to have settings with values that are computed after configuration files and the command line are parsed.
  • Cliapp now logs the Python version at startup, to aid debugging.
  • cliapp.runcmd now logs much less during execution of a command. The verbose logging was useful while developing pipeline support, but has now not been useful for months.
  • More default settings and options have an option group now, making --help output prettier.
  • The --help output and the output of the help subcommand now only list summaries for subcommands. The full documentation for a subcommand can be seen by giving the name of the subcommand to help.
  • Logging setup is now more overrideable. The setup_logging method calls setup_logging_handler_for_syslog, setup_logging_handler_for_syslog, or setup_logging_handler_to_file, and the last one calls setup_logging_format and setup_logging_timestamp to create the format strings for messages and timestamps. This allows applications to add, for example, more detailed timestamps easily.
  • The process and system CPU times, and those of the child processes, and the process wall clock duration, are now logged when the memory profiling information is logged.
  • Subcommands added with add_subcommand may now have aliases. Subcommands defined using Application class methods named cmd_* cannot have aliases.
  • Settings and subcommands may now be hidden from --help and help output. New option --help-all and new subcommand help-all show everything.
  • cliapp(5) now explains how --generate-manpage is used. Thanks to Enrico Zini for the suggestion.
  • New function cliapp.ssh_runcmd for executing a command remotely over ssh. The function automatically shell-quotes the argv array given to it so that arguments with spaces and other shell meta-characters work over ssh.
  • New function cliapp.shell_quote quotes strings for passing as shell arguments.
  • cliapp.runcmd now has a new keyword argument: log_error. If set to false, errors are not logged. Defaults to true.

Bug fixes:

  • The process title is now set only if /proc/self/comm exists. Previously, on the kernel in Debian squeeze (2.6.32), setting the process title would fail, and the error would be logged to the terminal. Reported by William Boughton.
  • A setting may no longer have a default value of None.
Posted Wed Mar 13 22:02:23 2013 Tags:

I've just released version 0.22 of ttystatus, my Python library for showing progress reporting and status updates on terminals, for (Unix) command line programs. Output is automatically adapted to the width of the terminal: truncated if it does not fit, and re-sized if the terminal size changes.

Available on code.liw.fi only, due to the Debian freeze.

Version 0.22, released 2013-03-12

  • When the terminal size changes, ttystatus will now update the display at once.
Posted Wed Mar 13 08:40:05 2013 Tags:

Daniel bought a Trunk Tees "Happiness is a depilated yak" hoodie. See the picture! (Won't embed it, since it's on Google Plush, sorry.)

Posted Tue Mar 12 20:05:26 2013 Tags:

Daniel told me how to set up scroll wheel emulation on a Thinkpad. Presumably it works on other systems as well, but the magic numbers for xinput may be different. I wrote a script to set it up, and configure my GNOME session to run it upon login.

The result: pressing the middle mouse button and moving the trackpoint up/down or left/right results in the applications receiving events as if a scroll wheel had been used.

Very, very handy. Thanks, Daniel.

#!/bin/sh

set -eu

id=$(xinput list | sed -n '/TPPS\/2 IBM TrackPoint/s/.*id=\([0-9]\+\).*/\1/p')
emu=$(xinput list-props "$id" | 
    sed -n '/Evdev Wheel Emulation (/s/.*(\([0-9]\+\)).*/\1/p')
but=$(xinput list-props "$id" | 
    sed -n '/Evdev Wheel Emulation Button (/s/.*(\([0-9]\+\)).*/\1/p')
axs=$(xinput list-props "$id" | 
    sed -n '/Evdev Wheel Emulation Axes (/s/.*(\([0-9]\+\)).*/\1/p')
xinput set-int-prop "$id" "$emu" 8 1
xinput set-int-prop "$id" "$but" 8 2
#xinput set-int-prop "$id" "$axs" 8 6 7 4 5
Posted Mon Mar 11 20:50:51 2013 Tags:

Mickaël Delahaye has a plugin for Gedit 2 to force off spaces-instead-of-tabs for Makefiles.

I wrote a similar plugin for Gedit 3, since all the APIs have changed.

The source is at git://git.gitano.org.uk/personal/liw/makefiletab3 (no .deb as of this time).

I probably did something really stupid in the plugin code, since I have no idea how plugins should really be written: I wasn't very illuminated by the documentation.

Posted Sun Mar 10 11:40:04 2013 Tags:

I love programming. I was born to code. I want to save the world by writing code.

I feel very strongly about software freedom. As I get older, and my eyes open to see more of the evil in the world, I become more concerned about freedom in general.

Software freedom is, at least traditionally, about permission use, study, modify, and re-distribute software. The Free Software Foundation expresses this as the four essential freedoms. Debian expanded on that, and produced the Debian Free Software Guidelines.

This is no longer enough. It is not enough to have all the freedom when you using your own computer, to have the source code to every bit of code that runs on your hardware. We live in the era of the Internet. Much of what we use computers for involves communication over the Internet, and the Internet is being actively used to curtail the freedom of people.

For example, governments and corporations do large-scale surveillance of everyone, by eavesdropping on private communications, gathering enormous databases of personal data, and by analysing everything they can in order to find patterns and make conclusions both at the statistical level and about individuals. This ruins privacy. Without privacy, democracy cannot survive.

Another example: in the name of various strawmen, such as terrorism, copyright violations, drugs, and child pornography, governments and corporations are collaborating in limiting private people's communications. I'm a Finnish citizen living in the UK. Both countries are among the closest ones to an ideal democractic nation state. Both countries arbitrarily block access to websites based on lists provided by private organisations, assuming that those organisations produce accurate lists of sites that contain copyright violations. As a result, both countries blocked, for example, a site where musical artists promoted their own works, bypassing the large media corporations that fund the list-making organisation.

These issues transcend software freedom, though they interact with it. Software freedom is a necessary requirement for freedom, in a world where almost everything is done with, or controlled by, the use of computers. Software freedom is not enough to prevent censorship or surveillance: even if all the software in the world were free, including the national firewalls of China, Finland, and the UK, this would not prevent those countries from censoring and surveilling their citizens. The firewall would run free software, but that does not give the citizens the freedom to disable, or modify, the firewall systems.

Fixing these issues is not a coding task. It is a job for politics, and it is going to take a long time, I fear. In the mean time, is there something a hacker can do to improve things?

My main hobby project is Obnam, my backup program. Does that help people to protect their freedom? I think it does, in a small way: Obnam supports online, encrypted backups (and de-duplication even for encryption is used), which means they can make backups to servers anywhere on the Internet without having to fear their data gets read and analysed by hostile entities such as their own government, other governments, large corporations, or criminals. Obviously this does not help them if their government prohibits the use of encryption, or requires key escrow, or mandates backdoors to all encryption methods. But it's a step in the right direction.

I don't claim Obnam will save the world, but a million small such steps by a thousand individual hackers, even without any particular organisation or guidance, will make a big impact.

What step can you take?

Posted Sat Mar 9 12:58:34 2013 Tags:

Soile, my beloved, is running a second Kickstarter campaign to fund her first feature-length movie, Maria Eveliina.

This time, she and her producer are seeking funding to pay an editor to do the final, professional cut of the movie. They've found a professional editor who'd like to work on the project, but need some payment.

The Kickstarter page explains more. It also has a "Back this project" button if you feel so inclined.

Posted Sat Feb 16 19:14:19 2013 Tags: