Two days ago, Hex v1.0 was released. This is a major achievement, and all the reason to celebrate! We’re delighted that we were able to contribute to this success by implementing a new feature which is now readily available!
Hex is the de-facto standard when it comes to package
management in the Elixir ecosystem. Most developers don’t typically run it
directly but rather invoke it via Mix tasks. For instance, running
deps.get for fetching dependencies is a very common thing to do – however,
it’s Hex doing all the grunt work in the background!
The recent Hex v1.0 release is a significant milestone for the project: it’s
the first time the major version got bumped after having been with the
version scheme for over seven
Being an Elixir shop, we are of course very excited to see this degree of software maturity. However, we’re not just happy users of Open Source software: we also want to give something back. For Hex v1.0, we implemented a feature which improves the authentication and contributed it back upstream for everyone to enjoy. :-)
Our own Hex repository
At BetterDoc, we run our own local Hex repository, hosting Elixir packages with shared functionality used by different Elixir projects. We’ve been happy users of the awesome MiniRepo project for a long time. It truly makes running your own Hex-compatible repository a breeze!
However, time moves on, and we noticed that MiniRepo eventually got deprecated.
In fact, it was superseded by a new, simpler way to build a Hex repository:
mix hex.registry build
(available with Hex v0.21 and later).
Sticking with MiniRepo didn’t seem like a sustainable approach, so we started
mix hex.build could be used to run our repository.
Building a repository using
hex.registry task is beautifully simple: given a directory full of
tarballs as created by
hex.build, a single
$ mix hex.registry build
can be used to generate the registry, i.e. the set of meta files which represents an index of the available packages. This leaves you with a plain directory of static files which can be hosted by any HTTP server to form a perfectly fine Hex repository!
At BetterDoc, we already use Amazon S3 for a few other things, so it was only natural to host the files there. However, there was one thing which got us scratching our heads:
How do we make sure that the repository is not readable by the whole wide world?
Elixir packages are plain tarballs which, among other things, contain Elixir source code. We’d rather not share that with our everyone (including our parents) right away; who knows what profanities might have found their way into the source code! 😬
Amazon S3 does have some authentication functionality built-in, but it involves
signing and authentication S3
Doing so is far from trivial: a specially crafted token needs to be created and
passed along with the HTTP request via the
Authorization HTTP header.
Unfortunately, Hex had no way good way to do this.
Thus, we needed to find a different way to authenticate clients somehow. We decided that plain simple HTTP basic access authentication would be sufficient for us. This is well-supported by Hex: simply encode the user name and password right in the URL which is registered with the mix hex.repo task – of course, you would use a strong password:
$ mix hex.repo add https://littlebobbytables:email@example.com
Implementing this server-side was easy: instead of talking straight to a S3 bucket, we would have our repository URL point to a Amazon CloudFront distribution which uses a tiny CloudFront function to implement the authentication check. Authenticated requests are then forwarded to S3.
Storing login credentials
We had a working solution and started exchanging virtual high-fives. However, there was also some guilty conscience: we were storing sensitive login credentials in the Hex configuration file, which is typically world-readable. Also, we don’t like our credentials spread all over the place!
Unix-based systems (including macOS) used a dedicated configuration file for
this exact purpose for many decades: the
This file provides a convenient place to collect login credentials to different
machines. Furthermore, the file is typically user-readable only (and many tools
processing this file enforce that).
Since we already used the
.netrc file for other purposes, we asked ourselves:
how much work would it be to make Hex read login credentials from the
Rolling up the sleeves
At BetterDoc, we enjoy working on (and presenting!) our Friday projects - and contributing to Open Source software is a great project! Thus, we decided to give extending Hex a shot. A few days (and nights) later, we got ourselves a nice pull request and were eagerly waiting for feedback.
We didn’t have to wait long: Eric Meadows-Jönsson (of the Elixir core language team) and Wojtek Mach (a prominent Elixir developer working on Ecto among other things) were quick to give very friendly and constructive feedback.
Design decisions were discussed, test cases extended and error conditions addressed. It was only during this review phase that we realised the wide range of platforms supported by Hex: the v1.0 Release Workflow verifies that the test suite passes with all Elixir version since Elixir 1.0.5, covering OTP 17.x through OTP 24. Phew!
As we were rewriting our code to work with the very first Elixir versions, we really started to appreciate how far the language has come. 😊
Eventually, our work got merged. Right in time for the big Hex v1.0 release! 🥳
We successfully scratched our own itch: basic auth credentials used for
authenticating with Hex repositories no longer need to be encoded in the URL.
Instead, we can nicely tuck them away in our trusted
The project has been a very pleasurable experience; the BetterDoc concept of Friday projects gave us the freedom to work on this task and the Hex project maintainers have been extremely welcoming and friendly all along.