Sharp tools: direnv

28 October, 2020

In this article we'll take a look at direnv, a tool I've been using for years to manage environment variables when developing projects. It recently got new features that especially teams can benefit from.

intro

A lot of runtimes and applications can be configured using environment variables. Take git for example: no less than 3 dozen knobs are given to the user.

Same thing goes for a lot of runtimes for programming languages: there's around 7 variables showing up in my env that determine what ruby version is active in my shell and where ruby libraries are to be found and installed.

Probably the application you're building has a bunch of environment variables as well.

Direnv allows you to manage these environment variables on a per-folder basis. As it supports various shells (bash, zsh, tcsh, fish and elvish) and platforms, it's relatively easy to introduce in a team.

.envrc

Direnv is all about .envrc files. Let's take a simple example:

# ~/projects/some-project/.envrc
export DATABASE_URL=postgres://host:5432/db1

Now when you enter ~/projects/some-project (or any folder below it) direnv will make sure to update your environment. It will unset the variables once you leave the project.

stdlib

There's all sorts of helper functions from the standard library at your disposal. Take for example prepending a project-folder to $PATH:

# ~/projects/some-project/.envrc
PATH_add bin

Way more intuitive and less error-prone than export PATH=$PWD/bin:$PATH!

For PATH-like variables you can use path_add:

path_add GOPATH go

Another variable that is often handy to have is PROJECT_ROOT, let's add it:

export PROJECT_ROOT=$(expand_path .)

This way scripts don't need to construct paths, but instead can just use $PROJECT_ROOT/some/file.

We'll see some more functions from the stdlib in the tips below.

customization

While having export DATABASE_URL=postgres://localhost:5432/some_database in a project's .envrc is a great way to communicate what environment variables your application expects (and what 'shape' the value should have), the exact value will probably differ between developers on a team.
To allow any value to be overridden locally add the following to the bottom of your .envrc file:

# file: ~/projects/some-project/.envrc
export DATABASE_URL=postgres://localhost:5432/db1

if [[ -f ".envrc.local" ]]; then
  source_env ".envrc.local"
fi

# better (see note below):
source_env_if_exists ".envrc.local"

Then in case the defaults don't suffice, a developer can put overrides in .envrc.local.

UPDATE: I opened an issue with a suggestion to include this idiom in the stdlib. The author kindly delivered; so as of v2.24.0 the above would become source_env_if_exists ".envrc.local".

inheriting

By default direnv only considers the first .envrc it finds going up the folder chain starting at $PWD.
If you want to 'inherit' from a .envrc higher up use source_up.
Typically I use this when having multiple projects under a client-folder:

~/projects
└── client
    ├── project1
    └── project2

Now when you need e.g. client-specific git-settings, you'd do:

# file: ~/projects/client/.envrc
export GIT_AUTHOR_EMAIL=user@client.org

# then in ~/projects/client/project1/.envrc ^1
source_up
...other stuff

^1: for the sake of example this is .envrc, typically you'd treat this as a personal setting and put source_up in .envrc.local

You might be tempted (as I was before I RTFM) to use this to define global settings in ~/.envrc. Don't do it, these belong in ~/.config/direnv/direnvrc or ~/.config/direnv/lib/*.sh.

CI

The project provides statically linked binaries for a wide variety of platforms, so using the latest version in a CI-job is easy:

# in an alpine-container
$ apk add bash #^1
$ DIRENV_VERSION=2.23.1 sh -c 'wget https://github.com/direnv/direnv/releases/download/v${DIRENV_VERSION}/direnv.linux-amd64 -O /usr/bin/direnv' && chmod +x /usr/bin/direnv
$ direnv allow /path/to/working-directory
# ^2
$ direnv exec /path/to/working-directory setup

^1: bash is required as direnv evaluates the contents of .envrc files in a bash subshell (and copies environment variables that changed to the active shell).
^2: at this point you might want to source CI-specific values to override the defaults from the .envrc

extending

Extending the standard library with your own functions is easily done by writing a bash function and putting it under ~/.config/direnv/lib/*.sh.

Say for example that we'd like to have an expressive way to set a specific version of the JDK.
Create ~/.config/direnv/lib/java.sh containing the following:

#!/usr/bin/env bash

# NOTE this only works on MacOS
use_jdk() {
  export JAVA_HOME=$(/usr/libexec/java_home -v"$1");
}

Then in a .envrc:

use jdk 15 #^1

^1: We're using direnv's helper function use that delegates to the appropriate use_* function.

The problem of these custom functions is that they are hard to use in a team-context: in order to use this .envrc it would require everyone on the team to have the java.sh in their lib-folder.

Though, with the recent introduction of source_url we can make our .envrc self-containing again.
The only thing we need to do is store java.sh somewhere publicly accessible (e.g. as gist) and change our .envrc:

direnv_version 2.23 #^1

# ^2
source_url "https://gist.githubusercontent.com/eval/049354b2d4ebd1e6f67440e4b98fffc5/raw/939c8b0767579866d6fa2a5cec2f215c82feca5c/java.sh" "sha256-TvMUsEtbn8ruOSNNIfcMNK3XWqXTPtGpKGkV3RARGU4="

use jdk 15

^1: ensure to trigger an error when using older direnv versions that don't support all functions we use in this .envrc
^2: the script will be downloaded and its integrity-hash verified before being sourced.
NOTE: To find the integrity hash for a script, leave it out and follow the instructions that direnv allow shows.

conclusion

For years I found direnv to be an indispensable tool for development. The recently added capability to include external scripts makes it an even more powerful tool, especially for teams.