and developer guides
Attendees: Amir Chaudhry (chair), Thomas Gazagnaire, Thomas Leonard, Anil Madhavapeddy, Hannes Mehnert, Mindy Preston, Dave Scott, Magnus Skjegstad and Nik Sultana
Nik Sultana working on a new tool aims to make builds of Mirage applications on various operating systems, using the Docker images (see the repo). This is a continuation of looking at Bash scripts. It just runs a cron job everyday and lets us know if we've broken any interfaces. Currently trying to make it colourful or use images (suggestions welcome!).
At the moment we only have two repos being tested this way so it would be good if people added their own. For example, the TLS stack would be useful — Hannes can do this (and may also add the XMPP stuff).
The logs are in a README file and you can see (at time of the call) that mirage-skeleton and mirage-www both build on Ubunty Trusty (with OCaml 4.01.0) but not the others, as yet.
The aim is to merge the output to
mirage-www at some point but we should
also close the loop somehow by notifying people (maybe send emails).
This is sort of like TravisCI but instead the builds happen on a fixed
schedule (via cron), rather than on each pull request. This helps to surface a
different set of issues. Should look at how to close the loop over the next
few weeks (perhaps getting emails working first).
Dave was going do these releases but found more bugs, which got in the way. On
the positive side,
mirage-dev is getting better and better and now the TLS
work is added too.
Packaging metadata is the most awkward stuff which affects the build. Thomas is adding OPAM files to the projects he's making and using the pin and publish workflow, which is an improvement (NB: OPAM 1.2 is released now, which makes this available to everyone).
We should switch all things over to OPAM 1.2 workflows but should take note of any blog posts/docs that need to be updated (brief segue into blog aggregation — see AOB).
ThomasL wrote a blog post about Mirage tracing. Everyone was in awe. When the work is finished, we should have a post on the Mirage website.
Some questions about when people can start using it. Would be great to get
more feedback from others trying it out. At the moment it requires lots of
pins and there was a thought about adding it to
mirage-dev but we really
should flush the queue of releases before adding more things.
If anyone wants to use it to explore the TCP stack, that would be very useful ThomasL already looked at UDP). Maybe Nik can think about this, for example looking at the behaviour of our stack during ping flood (currently we crash!). In general, looking at the network stack is a good way forward.
Mindy will try out the tracing if she can reproduce the memory leaks she's had on her unikernel blog (it does reboot fine). It's not clear if that issue is related to the network stack or the K/V store (crunch). In any case, Mindy will write up what she finds (and we should be surprised if it turns out that crunch does not have problems). In addition, it'll be interesting to see what issues crop up if traces last more than a few seconds. Let's see what happens.
ThomasG has pulled out
mirage-tc from Irmin. Mirage Type-classes is a set
of functors and combinators to pretty-print (using sexplib), to convert to and
from and JSON and Cstruct buffers. There are some examples in the Readme on
the mirage-tc repo.
It's already released in OPAM and could imagine it being useful for the TLS
stuff (but that may introduce a dependency on
Feedback is welcome!
ThomasG debugged a bunch of stuff and found that make and config scripts are doing far too much magic. It would be good to have some consistent way to do this ... perhaps something like a tool for consistently describing a project! (sidenote: For the uninitiated this was a reference to Assemblage, which ThomasG and Daniel Bünzli have been working on but is not yet ready for users). Unfortunately, this means dealing with build systems which is usually complicated.
In general, config is a substantive and unsolved problem. Lots of breakages and issues because of optional dependencies (depopts). To try and help with this this, we've been removing depopts and replacing with virtual packages. These issues are cropping up just because the amount of code and number of libraries is growing. We do need a more consistent DSL for this.
There's a particular problem with
pkg-config being a is bit broken on Mac
OSX but it's better to try and fix that than do anything else (already been
pushing fixes). Anil will fix GMP and pkg-config and notes that Yosemite
latest release of OSX) has bridging support, which means not needing tuntap
modules any more. This is very handy.
Lots of people have been asking about 'junior jobs' — we really don't like this term so if you have a better one please do suggest it on the list.
This came up again in the OPW rounds and Mirage is now an even bigger (more daunting?) project. There are actually lots of tasks like this available so Anil will collate them and make them visible to others. Suggestion of a wiki page like the kind the Compiler Hacking sessions maintain could be good(link). This works very well for them, and they have expertise and time ratings for each task, but it does take regular maintenance.
Whether people are familiar with OCaml also makes a difference as there may be multiple learning curves involved so a straight-forward rating system is unlikely to work. Mindy mentioned that when she was looking at the project, she picked an area that she knew something about (networking), which made it easier to pick up what was going on.
This led to the idea that we should split out the possible tasks/subprojects by some kind of functional area so that newcomers can pick something they're already familiar with e.g networking stack, file systems, hypervisor etc. This seems like a sensible structure to present and neatly points people to specific sets of libraries so they don't have to be overwhelmed with the project as a whole. Anil will put something together along these lines. Mort really wants to work on the training material over the next months so this will be useful. Mindy also suggested adding things to OpenHatch, which is one mechanism people search for FOSS projects to contribute to.
TLS on Xen: Trying to upstream things to zarith but could just patch in OPAM repo instead? Separate package for things is sometimes better as it helps with debugging.
Thoughts on folding content into the openmirage.org site. Could run syndic in unix and commit the results. Could run this on the same place as the cron job. Amir to try a manual run of syndic with people's blogs posts, taking the code from OCaml.org.
Next call is scheduled for 12th November - Please refer to the mailing list for actual details a day or so in advance.