Dune's crash message:
> I must not segfault. Uncertainty is the mind-killer. Exceptions are the little-death that brings total obliteration. I will fully express my cases. Execution will pass over me and through me. And when it has gone past, I will unwind the stack along its path. Where the cases are handled there will be nothing. Only I will remain.
They did an episode of Signals and Threads (Jane Street's technical podcast) about their build systems back in 2020¥. Highly recommended, as are the other episodes, especially The Future of Programming with Richard Eisenberg° and What is an Operating System? with Anil Madhavapeddy§.
¥https://signalsandthreads.com/build-systems/
Very interesting footnote system. Is it just for fun and aesthetic or have a deeper origin or purpose?
I have to respect Jane Street for proving that HM-typed functional programming can be production-ready and extremely fast when given enough love.
I still occasionally hear things about how the more academic-styled functional languages can't work in production, but Ocaml shows that it absolutely can work, even with high performance requirements.
NoRedInk[0] also uses some Haskell in their backend, as well as Elm for most of their frontend. They've also worked some with Roc according to a blog post from a few years back; not sure if using it.
> HM-typed functional programming
Mercury also uses Haskell for their backend https://mercury.com/
Also very cool!
I was fortunate enough to work at Jet.com before Walmart completely destroyed it, which was an F# shop. I really liked it, and I never really felt "limited" by it.
The stuff I was working on didn't have nearly the same requirements as high-frequency trading like Jane Street does though. I never did any super low-latency stuff with F#, so it's tough for me to say how well it fair with that kind of environment.
Too bad they didn't use their namesake:
Would honestly be a lot more interesting than Haskell.
Thank you for mentioning this, because—not having followed the link—I thought someone was using Mercury for a backend. And... I wasn't sure I was prepared to follow that link.
Standard Chartered also uses their variation of Haskell.
Yeah, I actually interviewed for a Haskell position there in 2023. They didn't hire me, but it was kind of fun to do technical questions in Haskell for an interview instead of the boring JavaScript or Python that I usually use.
Certainly. There's also a good amount of F# in production. It's a fantastic option for building high quality programs with broad interop with existing systems.
There's also pandoc written in Haskell. Facebook used it for it's spam-protection too, at some point (it might still do that, but I don't know).
The only thing worse than a complex codebase is a complex build system. I think CMake is an abomination. “Hey go learn this new language to compile this other language!”
The only thing worse than a complex build system is a simple build system that can't build what you need to build. :(
>> Complexity has to live somewhere. If you are lucky, it lives in well-defined places. In code where you decided a bit of complexity should go, in documentation that supports the code, in training sessions for your engineers. You give it a place without trying to hide all of it. You create ways to manage it. You know where to go to meet it when you need it. If you're unlucky and you just tried to pretend complexity could be avoided altogether, it has no place to go in this world. But it still doesn't stop existing.
>> With nowhere to go, it has to roam everywhere in your system, both in your code and in people's heads. And as people shift around and leave, our understanding of it erodes.
>> Complexity has to live somewhere. If you embrace it, give it the place it deserves, design your system and organisation knowing it exists, and focus on adapting, it might just become a strength.
- Fred Hebert, https://ferd.ca/complexity-has-to-live-somewhere.html
This means the architecture is also bad, and that is what needs to be fixed.
so simple huh? since you fancy yourself an architect can you let me know how we might fix the architecture of https://github.com/llvm/llvm-project such that CMake is no longer needed?
Not in a glib response, no.
> CMake is an abomination. “Hey go learn this new language to compile this other language!”
"go learn this awful new language"
I don't mind CMake. It's awful, but I'm already using C or C++, so the bar is set pretty low. It does do a few things right, and at least some of the things it does wrong are manageable.
I'll eat at least a bit of shit if it means I can get more than one platform's-worth of build process out of a single set of human-editable configuration files.
Have you tried meson? I used it once many years ago, and it seemed slightly better.
The ideas embodied in modern target-oriented CMake are worthwhile. It's all hobbled by a dreadful script language but that could be improved with sufficient motivation.
I think what annoys me about it the most is that for every good idea in there there's one piece of tedious crap you have to work around, one actively bad misfeature that doesn't work properly, and one terrible aspect of the awful scripting language.
(And two minor annoyances that people determined to hate CMake will never shut up about. But then, if you're determined to hate something, there are worse things than CMake to do it to! So I can't be too critical.)
It's hobbled not just by that but by the fact that it's not the thing doing the building. It's a build system for build systems, and suffers from brittleness and confusion as a result.
Whenever I see posts from Janestreet blog I laugh inside thinking how working on the financial industry at Jane is probably more fun and rewarding than some 90% of the jobs in the so-called tech industry.
I spent several years in Haskell land a while back, my first functional experience after Lisp.
Ocaml never clicked for me, I have a rare form of semicolon allergy and Haskell just looked a lot nicer to me.
But then I recently tried Reason and enjoyed it A LOT, so everything Ocaml is suddenly interesting.
F#, while missing some of the more interesting features of OCaml (e.g. the expressiveness of its module system), also has indentation-sensitive syntax that ditches a lot of the scaffolding, including semicolons in most cases:
https://learn.microsoft.com/en-us/dotnet/fsharp/language-ref...
“If your intention is to compile or interpret files as scripts, double semicolons can and should be avoided when writing in OCaml. Leaving them does not raise errors, but they are useless. The compiler tolerates them to allow copy-paste from UTop to a file without having to remove them.“ https://ocaml.org/docs/toplevel-introduction#loading-librari...
https://reasonml.github.io/ looks cool, OCaml with javascript.
i wish the reason syntax was the main one
It's funny because the thing that always turned me off from Haskell was that it feels like a stream of whitespace and identifies without good visual cues as to structure.
I'm sure once you get the zen of it, it's fine. Like Lisp, I guess, you learn to think in its structure. But looking at a screenful of Haskell to me is intimidating.
Of the bunch I found SML/NJ to be the most readable.
I wonder if anyone has used Jane Street's open source OCaml libraries and tools for somewhat serious projects.
I took a brief look at these things, and my impression is that their stuff isn't "ready" for anyone outside Jane Street, even though they put a lot of effort in building the ecosystem and open source their code.
I do! Their standard lib replacement was used on the project where I originally learned ocaml, so I have just always used it. You have to be fairly sophisticated at reading fn signatures to find things sometimes but otherwise no complaints with it. It has some very nice features too for example convert functions to/from sexps for all base types can save you a ton of time if you know it's there and plan for it.
I've used some of their other libraries too, their logging and unit test ppx are common maybe even de facto standards as much as the ocaml world has such a thing. I've also used, off the top of my head, their code formatter, one of their test frameworks, their implementations of some advanced data structures.
Sometimes you do run into one like the other commenter said, where that shit just does not work. It depends on an undocumented something they shipped separately, or needs a secret bit of config or whatever. These aren't malicious, I open a ticket and come back in a year or two often they'll be working.
It's not zero frustration but I appreciate their approach of just throwing everything over rather than spending more resources testing and polishing fewer releases. Their code quality is generally very high and even if I can't get something working directly, it provides a rigorous & vetted example implementation.
I tried to use their magic-trace tool but I get at least 4 differents crash when using it on trivial dotnet programs.
kinda surprising that it still makes sense to develop your own new build system, and then do the migration to it, instead of adopting an off-the-shelf one like buck2 (or bazel/etc...). Not saying that these fancy build tools are easy to adopt, but compared to building your own? is there something remarkable that makes dune much different?
Without getting into any specifics of it - I'm sure there's people with much more experience with these tools who can comment - I'll point out that neither buck nor bzl existed when JS decided to start building their own tool in 2012. Bazel's first release was in 2015, Buck's was in 2013.
JS does have a bit of a NIH culture, but I'm not sure if that was really at play here. There just...weren't very many good build tools available at the time, particularly for a company using an unorthodox tech stack.
> I'll point out that neither buck nor bzl existed when JS decided to start building their own tool in 2012. Bazel's first release was in 2015, Buck's was in 2013.
But Dune started (according to this blog post) in 2016 and JS started seriously improving and adopting it last year. So to me Jenga sounds like a reasonable step in 2012, but pouring significant effort into migrating from Jenga to Dune (and improving Dune) in 2024 sounds more weird
Jenga and dune are the same thing, it was just renamed.
The blog post clearly describes them as two different systems, and how Jane Street migrated from one to another.
Yes and no. This is all spelled out in the post, but it's a little thorny.
Dune is a rename of Jbuilder (2016). Jbuilder uses Jenga (2012) configuration files.
> By 2016 we had had enough of this, and decided to make a simple cross-platform tool, called Jbuilder, that would allow external users to build our code without having to adopt Jenga in full, and would release us from the obligation of rewriting our builds in OCamlbuild [...] Jbuilder understood the jbuild files that Jenga used for build configuration.
So in 2012 it made sense for them to build Jenga, because there weren't any good alternatives - Bzl etc. didn't exist, so they couldn't have solved their problems.
And in 2016 they had open-source code they wanted others to be able build; those people didn't want to use Jenga, and JS didn't want to rewrite their builds so that they could use something else. Thus, Jbuilder was a shim so that JS could still use their Jenga builds and others could build JS' code without using Jenga. Bzl etc., even though they existed, wouldn't have solved these problems either.
My bad, dune is a rename of jbuilder indeed. Not Jenga. But the other reply provides more context that's important.
I don't know ocaml particularly well but my understanding is that only with buck2 (of those mentioned) do you have a build system with the semantics to actually compile ocaml properly.
In particular I was under the impression one needed to be able to run ocamldep before hand (or compile twice) - buck2 can do this, bazel needs hacks iirc.
I loath autotools with a passion, and it's awkward to set it all up, but it handles OCaml code just fine. We use it for multi-language projects (including OCaml) like nbdkit & libguestfs.
Dune is the single best name of a build system out there, nothing even comes close to.
Indeed.
"He who controls the [build system] controls the universe."
What percentage of JS programmers actually work on this stuff I wonder?
The article said the team working on it has 5 engineers.
might get downvoted but all i think of when I see a Jane Street article is how much each programmer earns
Man I must be under a rock. Never heard of Dune!
Cmake mostly
but why?
I respectfully have to disagree. As Fedora packager for OCaml packages, dune's configuration is yet another one-off, with a weird Lisp syntax, hard to understand, difficult to patch. I'd far rather people stuck to using more familiar Linux build systems. As an upstream OCaml developer, the whole thing falls down the minute you need to integrate other programming languages into your build (or OCaml code into a code base written in another language).
(This rant more or less equally applies to other language-specific build systems.)
> more familiar Linux build systems
"Familiar" is not a property of any system. It's a relation between a system and a user.
Some Linux build systems maybe be more familiar to some users, but will be less familiar to others. When picking a build system, you can't just look at the system itself and declare it familiar or not.
It's not even enough to look at the total number of users familiar with some thing. Hindi is one of the most familiar languages in the world, but you're probably gonna have a bad time if you use it for the menu in a cafe in rural Texas.
You have to look at your actual cohort of users (and potential future users) and see what's familiar to them. This is one of the key reasons why usability is actually a deeply hard problem. So much of usability hinges on familiarity, but familiarity is a human-specific highly variable property.
As a user of Linux, I wish Linux distributions would abandon their build systems which really only work well for C and maybe C++, and also only work on one platform (not even the whole of linux!) meaning that packaging has to be duplicated for every OS, and stuck to using more familiar language-specific build systems that work everywhere.
(this rant more or less equally applies to all os-or-distro-specific build systems)
---
This rant is only semi-serious. I do see some value in the Linux distribution style packaging. In particular, I do sympathise with the need to do cross-language builds. But goodness are they a pain to work with, and probably the biggest barrier to me shipping software on Linux.
My hope is that eventually an evolution on build systems like bazel/buck2 will lead to a truly universal build system that is both cross-platform and cross-language. But unfortunately it doesn't look like it's coming soon.
If you only care about Linux and MacOS then Nix can be that universal cross language build tool.
> more familiar Linux build systems
What's a 'familiar Linux build system'? make?
They likely mean 'generally used with other languages'. make is an example.
Make is primarily used with C and C++. It is not commonly used in Java, Rust, Go, NodeJS, or hardly anything besides C and C++. Make is not "generally used with other languages".
I haven't been doing much OCaml in recent years, but I do sometimes check in to see what's going on in OCaml-land, and it seems like dune is what everyone is using now - what are the alternatives?
Using, probably, to be deprecated ocamlbuild, with _tags file which is another made up syntax.
But I'm a new OCaml user, and actively starter using ocamlbuild because dune added layers of indirection really tripped me up at first.
> This rant more or less equally applies to other language-specific build systems.
And yet those language-specific build systems are overwhelmingly winning, in pretty much every language.
> As Fedora packager for OCaml packages,...
I honestly think traditional Linux packaging is in the wrong here and the problems are essentially self-inflicted (not in the sense that individual maintainers are doing something wrong, but in the sense that the policy that traditional Linux distributions are following is inherently unsustainable. It's designed for a pre-CPAN world)
> As an upstream OCaml developer, the whole thing falls down the minute you need to integrate other programming languages into your build (or OCaml code into a code base written in another language).
True up to a point, but frankly the worst case is falling back to a terrible C-style build, and "always do terrible C-style builds in case you need to integrate with C code" is not a proposition that has much appeal.
Much as I wish the whole world would standardise on Maven or Cargo, I can't see a realistic path to there without first eliminating C, because the C people are never going to agree to follow a standard for package repositories.