The MirageOS Documentation

Agenda

• Review notes from last time
• is-mirage-broken and Docker
• mirage-dev releases
• Tracing
• mirage-tc release
• (good?) use of custom configure and make scripts
• Mirage task list for newcomers

Attendees: Amir Chaudhry (chair), Thomas Gazagnaire, Thomas Leonard, Anil Madhavapeddy, Hannes Mehnert, Mindy Preston, Dave Scott, Magnus Skjegstad and Nik Sultana

Notes

is-mirage-broken and Docker

Nik Sultana working on a new tool aims to make builds of Mirage applications on various operating systems, using the Docker images (see the repo). This is a continuation of looking at Bash scripts. It just runs a cron job everyday and lets us know if we've broken any interfaces. Currently trying to make it colourful or use images (suggestions welcome!).

At the moment we only have two repos being tested this way so it would be good if people added their own. For example, the TLS stack would be useful — Hannes can do this (and may also add the XMPP stuff).

The logs are in a README file and you can see (at time of the call) that mirage-skeleton and mirage-www both build on Ubunty Trusty (with OCaml 4.01.0) but not the others, as yet.

The aim is to merge the output to mirage-www at some point but we should also close the loop somehow by notifying people (maybe send emails). This is sort of like TravisCI but instead the builds happen on a fixed schedule (via cron), rather than on each pull request. This helps to surface a different set of issues. Should look at how to close the loop over the next few weeks (perhaps getting emails working first).

mirage-dev releases

Dave was going do these releases but found more bugs, which got in the way. On the positive side, mirage-dev is getting better and better and now the TLS work is added too.

Packaging metadata is the most awkward stuff which affects the build. Thomas is adding OPAM files to the projects he's making and using the pin and publish workflow, which is an improvement (NB: OPAM 1.2 is released now, which makes this available to everyone).

We should switch all things over to OPAM 1.2 workflows but should take note of any blog posts/docs that need to be updated (brief segue into blog aggregation — see AOB).

Tracing

ThomasL wrote a blog post about Mirage tracing. Everyone was in awe. When the work is finished, we should have a post on the Mirage website.

Some questions about when people can start using it. Would be great to get more feedback from others trying it out. At the moment it requires lots of pins and there was a thought about adding it to mirage-dev but we really should flush the queue of releases before adding more things.

If anyone wants to use it to explore the TCP stack, that would be very useful ThomasL already looked at UDP). Maybe Nik can think about this, for example looking at the behaviour of our stack during ping flood (currently we crash!). In general, looking at the network stack is a good way forward.

Mindy will try out the tracing if she can reproduce the memory leaks she's had on her unikernel blog (it does reboot fine). It's not clear if that issue is related to the network stack or the K/V store (crunch). In any case, Mindy will write up what she finds (and we should be surprised if it turns out that crunch does not have problems). In addition, it'll be interesting to see what issues crop up if traces last more than a few seconds. Let's see what happens.

mirage-tc release

ThomasG has pulled out mirage-tc from Irmin. Mirage Type-classes is a set of functors and combinators to pretty-print (using sexplib), to convert to and from and JSON and Cstruct buffers. There are some examples in the Readme on the mirage-tc repo.

It's already released in OPAM and could imagine it being useful for the TLS stuff (but that may introduce a dependency on mirage-tc). Feedback is welcome!

custom configure and make scripts

ThomasG debugged a bunch of stuff and found that make and config scripts are doing far too much magic. It would be good to have some consistent way to do this ... perhaps something like a tool for consistently describing a project! (sidenote: For the uninitiated this was a reference to Assemblage, which ThomasG and Daniel Bünzli have been working on but is not yet ready for users). Unfortunately, this means dealing with build systems which is usually complicated.

In general, config is a substantive and unsolved problem. Lots of breakages and issues because of optional dependencies (depopts). To try and help with this this, we've been removing depopts and replacing with virtual packages. These issues are cropping up just because the amount of code and number of libraries is growing. We do need a more consistent DSL for this.

There's a particular problem with pkg-config being a is bit broken on Mac OSX but it's better to try and fix that than do anything else (already been pushing fixes). Anil will fix GMP and pkg-config and notes that Yosemite latest release of OSX) has bridging support, which means not needing tuntap modules any more. This is very handy.

Mirage tasks list for newcomers

Lots of people have been asking about 'junior jobs' — we really don't like this term so if you have a better one please do suggest it on the list.

This came up again in the OPW rounds and Mirage is now an even bigger (more daunting?) project. There are actually lots of tasks like this available so Anil will collate them and make them visible to others. Suggestion of a wiki page like the kind the Compiler Hacking sessions maintain could be good(link). This works very well for them, and they have expertise and time ratings for each task, but it does take regular maintenance.

Whether people are familiar with OCaml also makes a difference as there may be multiple learning curves involved so a straight-forward rating system is unlikely to work. Mindy mentioned that when she was looking at the project, she picked an area that she knew something about (networking), which made it easier to pick up what was going on.

This led to the idea that we should split out the possible tasks/subprojects by some kind of functional area so that newcomers can pick something they're already familiar with e.g networking stack, file systems, hypervisor etc. This seems like a sensible structure to present and neatly points people to specific sets of libraries so they don't have to be overwhelmed with the project as a whole. Anil will put something together along these lines. Mort really wants to work on the training material over the next months so this will be useful. Mindy also suggested adding things to OpenHatch, which is one mechanism people search for FOSS projects to contribute to.

Amir raised the prospect of what we want from Mirage 3.0 and that we should plan this with enough time. One interesting suggestion is having a fully working Javascript backend. Will raise this next time to capture people's thoughts. Of course, we do still need to get all the things for 2.0 finished first, including the TLS stack and deciding on C library handling.

--

AoB

• TLS on Xen: Trying to upstream things to zarith but could just patch in OPAM repo instead? Separate package for things is sometimes better as it helps with debugging.

• Thoughts on folding content into the openmirage.org site. Could run syndic in unix and commit the results. Could run this on the same place as the cron job. Amir to try a manual run of syndic with people's blogs posts, taking the code from OCaml.org.

• Next call is scheduled for 12th November - Please refer to the mailing list for actual details a day or so in advance.