The MirageOS Documentation

Agenda

• Xen Hackathon summary
• Irmin update
• OCaml conduit progress
• Networking fuzz testing
• Unikernel deployment
• Call schedule and organisation

Attendees: Amir Chaudhry (chair), Thomas Gazagnaire, Thomas Leonard, Anil Madhavapeddy, Richard Mortier, Mindy Preston and David Scott

Notes

Xen Hackathon

The Xen Developer Hackathon took place a couple of weeks ago and Anil, ThomasL, Dave, Jon and Nik all attended. Apparently it did turn into a chatathon as previously predicted.

A group of people went through Amir's Jekyll to Unikernel post to get their sites building using TravisCI. They did get this working but the post was written based on a Mac workflow and some parts didn't work well on other machines, specifically involving tap interfaces (e.g. sudo mirage run didn't work as expected). In general, Amir mentioned that the blog post isn't going to get updated but it should point to somewhere that current information is available for people to follow (that can account for any quirks on different systems). The Mirage website would be good but updating this isn't as easy as a wiki.

Other than the above, there were lots of conversations with others people including those trying lots guests on Xen and seeing lots of latency. Important for us as we want to be able to run lots of unikernels on top of a single Xen instance.

Irmin

Irmin 0.8.1 was released. ThomasG has been working on an filesystem backend implementation. You can take a normal filesystem and write at the block level. Next step is to expose Irmin as a filesystem. So you can use the filesystem and connect somewhere else to get the history back. Practically speaking, you build an Irmin filesystem on top of an existing filesystem (e.g FAT). It'll be a combinator that takes the FAT implementation and make a new one that's used/exposed (so the file system is like staging). [If these notes are confusing, it's because I couldn't keep up - AC :)]

ThomasG also learned that the DOS protocol doesn't let you have two letter directories. Also had some discussion on FAT, blockstore and key value stores.

Conduit

The network stack has become a chimney, where the underlying choices are affecting things further up the stack. Anil has been thinking about different approaches and discussing things with people. One thought was around using functors but that leads to having a functorised stack and then the applications also have to deal with things this way and whole universe becomes functorised. This becomes difficult to use. Another approach is to use objects (or at least partially) and this seems like it would work much better. In any case, this is blocking other people's cohttp fixes but looks like there is a workable solution to this. Anil will have to write this down to clarify it and the details are quite involved (but are to do with existential types -- in case anyone feels like diving into it).

In general conduit will permit people to use either openSSL bindings or the new TLS library and doesn't impose those choices on anyone.

Network fuzz testing

Mindy is in a cycle of finding bugs and fixing them. Been thinking about the platform question and doing research today. Also saw Balraj's email, who has been bisecting to figure out where/when the problems arose in the stack. It turns out that it's a set of commits over Christmas/New Year by Anil.

Overall, it's a great sign that we're able to go back and pin down which set of commits introduced the problems but it does show that we need to improve the type safety in the stack and add unit tests. Any thoughts on unit test frameworks for networking protocols would be useful so please do send them to the mailing list.

Unikernel deployment

We currently have two machines, configured with Xen that we could deploy our unikernels to. Amir would like to put his unikernels there and (when stable) start moving more of the static sites we have to those machines. It should be possible to get an account for Anil and at the moment users would need to write their own config file. There is the question of DNS servers.

In the long term the idea is that we each have our own unikernel DNS servers but in the short term to have a separate DNS server that would know about the mapping between the names and the IP addresses. We have a set of IP addresses on a wiki and can grab one of those and run with it. For example Mort has one but has not deployed yet as he's blocked on filesystem issues. Essentially, his site is large enough (papers, media, etc) that he needs two FAT images, which isn't handled very well. Can achieve it locally but it requires tweaking by hand and shell scripts, so not really appropriate for deployment. Is thinking of patching the Mirage tool but has not time to get to that. Should create an issue to track this as others are likely to follow in his wake as their sites grow (e.g Amir switched from Crunch to FAT some time ago). An issue might help surface others who are having similar kinds of problems.

Call schedule and organisation

Amir points out that we should improve the call organisation. Announcement of the call was sent about an hour beforehand and we should be giving others more time. Proposed that a notice is sent on the Monday beforehand to the list to collect Agenda items, and then the call is confirmed the following day. Regular people on the call can help by adding their agenda items in advance and Amir will structure them and help keep the call focused (which it usually is anyway).

AoB

Mindy and ThomasL have written some awesome blog posts recently. The posts made it to the front pages of news aggregators and generated a lot of interesting discussion. The posts were Python to OCaml: Retrospective (HN, Reddit) and The Minnesota Goodbye (HN).

OCaml-TLS is is gearing up to an alpha release and we will point people to the Mirage mailing list for communications.

Some discussion about Mirage 2.0 release and what the requirements would be for it. Would need ARM support, Irmin and a story on distributed computing. This all seems achievable but really would like to have this by the time of OSCON.

Carmack thinks we're "really interesting". Cue lots of fist pumping.

Next call will be 24th June. Amir will send an email to the list for Agenda items on the 23rd.