Installing pyenv on macOS

Python is very well suited for writing AWS Lambda functions.
For local development, you might want to be able to run the whole range of supported Python runtimes on your Mac.
Unfortunately, setting this up is not as straightforward as you might think.

A nice way to manage installations of various versions of Python is to use pyenv.
Although installing pyenv itself is trivial, you might run into some trouble later.
This post is not meant as an introduction to pyenv, but as a guide for solving the following problems:

• Python versions fail to compile
• a pyenv managed Python does not play nice with Homebrew
• a Homebrew managed Python environment interferes with pyenv

This post shows how to configure your system to avoid these problems.

Setting up pyenv

Simply installing pyenv using Homebrew and making a small change to the configuration of your shell is sufficient to get pyenv itself working properly.
(Do not forget to reload your shell after changing its config.)

brew install pyenv

However, if you now try to install a Python version, it will most likely fail.
You first need to make sure you have installed the dependencies for building Python.

Trick #1: Install build dependencies

Some development headers are required for compiling Python on your Mac.

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

After installing macOS updates you will need to (re)install these headers!
See: https://github.com/pyenv/pyenv/wiki

Although not required, the pyenv wiki recommends installing some additional libraries.

brew install openssl readline sqlite3 xz zlib

Now you try to install any python version you like by executing pyenv install $VERSION. For a list of available versions, execute pyenv install --list.

pyenv install 3.7.2
pyenv install 3.6.8
pyenv install 2.7.15

Finally, configure pyenv to use one of these Python versions by default.

pyenv global 3.7.2

Let pyenv play nice with Homebrew

If you run brew doctor, it might have some complaints.

$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: "config" scripts exist outside your system or Homebrew directories.
<code>./configure scripts often look for *-config scripts to determine if
software packages are installed, and what additional flags to use when
compiling and linking.

Having additional scripts in your path can confuse software installed via
Homebrew if the config script overrides a system or Homebrew provided
script of the same name. We found the following "config" scripts:
~/.pyenv/shims/python3.6m-config
~/.pyenv/shims/python3.7-config
~/.pyenv/shims/python3.7m-config
~/.pyenv/shims/python-config
~/.pyenv/shims/python3-config
~/.pyenv/shims/python3.6-config

If you do not solve this, any Python dependent package from Homebrew might be compiled against a Python version managed by pyenv.
See the second trick to create a wrapper script to solve these issues.

Trick #2: A wrapper for brew

Create the following wrapper script for the brew executable with the following content.

#!/bin/sh
# check if pyenv is available
# edit: fixed redirect issue in earlier version
if which pyenv >/dev/null 2>&1; then
  # assumes default location of brew in <code>/usr/local/bin/brew
  /usr/bin/env PATH="${PATH//$(pyenv root)/shims:/}" /usr/local/bin/brew "$@"
else
  /usr/local/bin/brew "$@"
fi

Store it, also named brew, in a place that has a higher priority in your $PATH than the real brew executable, so that this wrapper script is executed instead of the real brew.
(Do not forget to make the wrapper script executable!)
See: https://github.com/pyenv/pyenv/issues/106

Running brew doctor with this wrapper, it is satisfied.

$ brew doctor
Your system is ready to brew.

Let brew play nice with pyenv

All Pythons managed by pyenv are nicely tucked away in some hidden place, but everything installed in a Python version managed by Homebrew (including pip) is placed in /usr/local/bin. This path might have preference over the shims managed by pyenv and ruin your experience.

Unfortunately one cannot simply uninstall Homebrew managed Python versions as some packages depend on it.
What you can do to prevent any issues is to:

• uninstall any packages installed by Homebrew managed Python versions, see the third trick
• make sure the pyenv shims have a higher priority than /usr/local/bin in your $PATH

Trick #3: Uninstall pip packages from the Homebrew managed Python versions

/usr/local/bin/pip2 freeze |xargs /usr/local/bin/pip2 uninstall -y
/usr/local/bin/pip3 freeze |xargs /usr/local/bin/pip3 uninstall -y

Repeat these commands until all packages have been removed.

Wrapping up

This post was written to help you solve some issues with pyenv, it is not meant as a guide on how to use it.
For a user guide, please refer to the readme.

Happy Pythonning!