Reliably determining the user’s $PATH environment variable

Introduction

Reliably determining the current user's $PATH environment variable doesn't sound too difficult. Depending on your understanding of shell execution modes, your grasp of the unix process model, and your familiarity with the interaction between shells, processes and terminal emulators works, it might not be. I thought I had a reasonable (if rusty) handle on all three, but I came up short recently, so wanted to share what I learned.

Context

Stackman is a desktop application which acts as a process manager for your Node.js applications. It starts and stops them, restarts them if they crash, funnels their log output to a central time-ordered view, and keeps them running on predictable ports between application and system restarts. I built it to solve my own disorganisation: often drowning in too many terminal tabs when running multiple Next.js stacks at once due to the various SaaS products I've been building. At a basic level you can think of it as a smart wrapper around npm run dev, and as such it needs the correct $PATH to find node, npm, or whichever package manager each project uses.

The problem

The very first version of stackman I shared with a friend (the excellent @MrRio) didn't work. He could add his projects to stackman, but he couldn't actually run them. You know, the whole raison d'être of the application. Every time he tried to start a process it would instantly crash, pointing at a problem retrieving the correct $PATH environment variable needed to resolve the location of node and npm. If stackman was a command-line application the user invoked from a shell we'd already have a $PATH to inherit, but it's not - it's a desktop application which inherits the default system $PATH which is likely to be useless.

I'd had to tackle this to get the application running on my own machine, and hoped my rather crude solution would do the business for others, too. Here's roughly what it looked like:

import os from 'node:os'
import { execSync } from 'node:child_process'

export const getPath = () => {
  // userInfo() returns, among other things, the user's default shell
  const { shell } = os.userInfo()
  // I don't recommend execSync in production, but it keeps the example shorter
  return execSync("bash -c 'echo $PATH'", { shell }).toString().trim()
}

This function was called elsewhere like so:

const process = spawn('npm', ['run', 'dev'], {
  env: {
    ...process.env,
    // merge the hopefully correct path into the env vars passed to the child process
    PATH: getPath(),
  },
})

Okay, look; I know it looks a bit stupid going to the effort of fetching the user's default shell only to use it to launch bash, but I had my reasons: I use fish shell which uses space as a path delimiter, which is awkward to deal with when individual path entries themselves contain spaces. Bash's colon separator is more predictable and standard across other shells, and I knew bash would be available on any supported system, so my logic was as follows:

• The user's default shell almost certainly has the correct $PATH configured, since it's the shell their terminal sessions launch with
• Regardless of what that shell is, we can use it to launch bash, which will inherit the correct $PATH from the default shell and provide a normalised $PATH to use

The logic seemed sound and worked fine during my testing: fish launched bash, bash inherited its $PATH from fish, and bash gave me a colon-delimited string in return.

As an aside, it's worth understanding that exec and execSync always use a shell to execute commands, even if you don't explicitly pass one: on unix systems, they'll default to /bin/sh. We're specifying the shell to launch with here, but it's good to know that if you don't, you'll still get one, and to know which one it is. Other child process functions like spawn won't use a shell by default (but they can if you want them to), executing the command directly instead using system calls. These little details are important but very easy to lose track of.

Anyway, you may already be able to see the problem with this approach. I couldn't. And I got a bit stuck.

ChatGPT to the rescue

I've been a jack-of-all-trades developer for nearly 20 years, meaning I know enough about enough to be effective but not always entirely accurate, sometimes lacking the exact terminology to succinctly describe what I'm trying to do. Being succinct and accurate is exactly what you need to successfully Google a problem, but thankfully being vague and verbose is something ChatGPT handles incredibly well. As someone who's spent the majority of the year working alone, I've found it invaluable in situations like this where I'd otherwise progress painfully slowly. It actually gets better with a bit of back-and-forth; continuity which simply doesn't exist with a search engine. Working a problem with ChatGPT is like a turbo-charged pair programming session with the most enthusiastic developer you've ever met, and while you need to fact check the answers you get, it's a great way to sharpen your understanding of a problem and point you in the right direction to solve it.

A lot of conversation followed this opening exchange, but ChatGPT undoubtedly saved me a lot of time and frustration.

A solution

I'm certain the solution below is far from perfect, but so far, it works. As is often the case with these things, there's not actually a lot to it:

import { exec } from 'node:child_process'
import os from 'node:os'
import path from 'path'
import util from 'node:util'

const execAsync = util.promisify(exec)

const getPathArgs = (shell: string) => {
  switch (path.basename(shell)) {
    case 'fish':
      // we still have some shell-ception going on to normalise the path. Sorry ¯\_(ツ)_/¯
      return `-i -c 'bash -c "echo $PATH"'`
    case 'zsh':
      return `-il -c 'echo $PATH'`
    case 'bash':
      return `--login -i -c 'echo $PATH'`
  }
  throw new Error(`Unsupported shell ${shell}`)
}

export const getPath = async (): Promise<string> => {
  const { shell } = os.userInfo()
  const args = getPathArgs(shell)
  const cmd = `${shell} ${args}`

  // TODO: handle errors
  const { stdout } = await execAsync(cmd)
  return stdout.trim()
}

Interactive, non-interactive and login shells

The code above launches different combinations of execution modes for each shell. Interactive and login shells are different things but can be combined together: indeed, the zsh and bash invocations launch an interactive login shell. The fish invocation launches an interactive shell. Let's examine some of the differences.

Non-interactive shells

We're not actually launching any non-interactive shells anymore: all three are launched with at least an -i flag. Non-interactive shells typically don't read any userland configuration files, which is where homebrew, nvm and many others typically wrangle the user's $PATH in weird and wonderful ways. The earlier code launching a non-interactive shell worked mostly by luck, because fish does source some common configuration files in non-interactive mode which in my case was enough.

Interactive shells

All three invocations launch an interactive (-i) shell. This sources extra configuration files in which users often manipulate their $PATH environment variable which will probably look pretty familiar to you: ~/.bashrc, ~/.zshrc and the like. This was take one of the fix, but after asking James - who uses zsh, the default shell since macOS Catalina - to test out a new build of the application spawning interactive shells, things still didn't work. Until...

Login shells

The bash and zsh invocations also launch a login shell (--login and -l). Login shells source even more commonly-used configuration files like ~/.bash_profile and ~/.zprofile, and finally with these tweaks in place stackman worked for James. Phew.

Why not always launch an interactive login shell?

Login and interactive shells are slower to launch than non-interactive shells because they source more configuration, and they can also have side effects like launching extra processes. Side effects - especially unexpected ones - can introduce unpredictability into a system, so you should always start with a non-interactive shell by default. The clue is also in the name: interactive shells are designed to be interactive, but I'm using them to run a command and then exit immediately.

For stackman, launching interactive login shells where required appears to be a necessary trade-off. We want to mimic the environment a user would have if they opened their terminal, navigated to a project, and ran npm run dev themselves. If there's a better way to mimic this than the hoops I'm jumping through above, I'd love to hear about it.

Why not just use process.env.PATH?

I covered this earlier but it's worth reiterating: when a user runs an application from their desktop, on macOS at least, the system $PATH variable is likely to be useless. In my case, it's /usr/bin:/bin:/usr/sbin:/sbin, none of which contain node and friends.

Summary

In hindsight the nature of the problem and the associated solution seem obvious, but at the time I'd been away from the nitty-gritty details of shells and their execution modes for a while, which coupled with my experience building desktop applications being close to zero was enough to trip me up. Apart from the useful refresher on $PATH resolution, the other big takeaway was another reminder to always use the tools at your disposal effectively: when searching a problem comes up short, don't be afraid to ask for help. Thanks, ChatGPT!