Documentation and guides

Weekly Meeting: 2015-09-09

9th September 2015: Xen file generation, functoria and end-to-end systems


Attendees: Amir Chaudhry (chair), Thomas Gazagnaire, Thomas Leonard, Hannes Mehnert, Richard Mortier, Gabriel Radanne and Dave Scott


Outstanding patches

xl files — Last time, a patch was mentioned about better .xl files. There was some discussion on the PR (mirage/mirage#440), and this has now been merged. There are still things to do as the deployment scripts will need updating. The plan is to update the skeleton file for the unikernel rather than the generic one. Some discussion that it might be useful to think about a more generic skeleton framework for deployment — like the one we ended up with for TravisCI. Dave will think about this but part of the process is to find out where the abstractions should be made. A separate comment was that the logic for Xen file generation might be better placed in a standalone library related to the point below).

functoria — See mirage/mirage#441. The work on this is close to being ready but unfortunately the above xl patch meant that a lot of messy work is required before the functoria patch is ready to merge again (as it changes a lot of things internally to the mirage tool). Things are looking a lot tidier already though.

The goal is to integrate this fairly soon and the next release of the mirage command line tool will incorporate it. There are a three pull requests to merge before this one and perhaps a fourth one too. Note that the whole engine is in functoria so the mirage tool will have a new dependency.

In terms of impact on users, it's largely the same as before. There may be changes to how things work with TLS and there will be much better ways to do certain things. It's not really a huge, breaking change as most existing files will work (although a few will need tweaking). Should be ready to look at for merge before the next call. Gabriel will work on some new examples that will help explain the differences, as well as a blog post. Should also add a note to the breaking changes page for anything that does need updating (as well as emailing the list).

Describing an end-to-end system

We have a reasonably good toolchain that gets us from code push to deployable unikernel but one of the less clear pieces is how to set up and deploy to a Xen machine. We do have the current deployment scripts Mort put together but there's room for further simplification (Thomas also did some work to make them more parameterised). One way to think about this is to aim for a short post (<1000 words) that describes the three stages of local build, CI and deployment. Keeping it this short means we need to have enough automated infrastructure to point people at. As an example, we've already achieved this with the Travis skeleton scripts.

Ultimately, we agree that deploying to EC2 is important but a good first step would be to document (and to some extent, automate) our deployment on the current Bytemark machines. Dave has been looking at these machines anyway and it's a manual process to set things up appropriately. It would be great to improve what we have and there have been new RPMs for Xen released recently that can help with this.