Let’s say I have a Python project which uses the following packages:
That’s ten packages in total. I first create and activate a virtual environment for the project. I can then install these packages using pip as follows:
python -m pip install pandas geopandas numpy scipy bokeh matplotlib descartes shapely pyproj sphinx
Note
Replace python -m with py -m if using Windows. See the pip documentation for more information.
For easier package management, I can list these packages in a requirements.txt file:
# requirements.txt
pandas
geopandas
numpy
scipy
bokeh
matplotlib
descartes
shapely
pyproj
sphinx
These packages can then be easily installed using pip or Anaconda (I recommend the lightweight Miniconda):
-
using pip:
python -m pip install -r requirements.txt -
using conda:
conda install --channel conda-forge --file requirements.txt
Tip
If you are using Anaconda, use the conda-forge channel, as some packages may not be available in the default channel.
Manually keeping track of required packages isn’t the best idea; you could easily forget to add a package to the list. pip offers the functionality to list all packages installed in the environment using the pip freeze command, which will produce the following output:
$ pip freeze
alabaster==0.7.12
attrs==19.3.0
Babel==2.8.0
bokeh==2.1.1
certifi==2020.6.20
chardet==3.0.4
click==7.1.2
click-plugins==1.1.1
cligj==0.5.0
cycler==0.10.0
descartes==1.1.0
docutils==0.16
Fiona==1.8.13.post1
geopandas==0.8.1
idna==2.10
imagesize==1.2.0
Jinja2==2.11.2
kiwisolver==1.2.0
MarkupSafe==1.1.1
matplotlib==3.3.0
munch==2.5.0
numpy==1.19.1
packaging==20.4
pandas==1.0.5
Pillow==7.2.0
pip-chill==1.0.0
Pygments==2.6.1
pyparsing==2.4.7
pyproj==2.6.1.post1
python-dateutil==2.8.1
pytz==2020.1
PyYAML==5.3.1
requests==2.24.0
scipy==1.5.2
Shapely==1.7.0
six==1.15.0
snowballstemmer==2.0.0
Sphinx==3.1.2
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
tornado==6.0.4
typing-extensions==3.7.4.2
urllib3==1.25.10
This is good, but it’s worth noting that some of these packages depend on another in the list. For example, GeoPandas depends on pandas, pyproj, NumPy, and Shapely. The requirements file could be more concise, and PIP Chill is a utility that can help us with this!
Install PIP Chill in the virtual environment as follows:
python -m pip install pip-chill
For version 1.0.1, these options are available:
$ pip-chill --help
usage: pip-chill [-h] [--no-version] [--no-chill] [-a] [-v]
Like `pip freeze`, but more relaxed.
optional arguments:
-h, --help show this help message and exit
--no-version omit version numbers.
--no-chill don't show installed pip-chill.
-a, --all, --show-all
show all packages.
-v, --verbose list commented out dependencies too.
Type pip-chill and it will produce the following output:
$ pip-chill
bokeh==2.1.1
descartes==1.1.0
geopandas==0.8.1
pip-chill==1.0.0
sphinx==3.1.2
scipy==1.5.2
It’s already looking a lot better! If we want to omit the version numbers, which ensures the latest version of the package is installed, add the --no-version option:
$ pip-chill --no-version
bokeh
descartes
geopandas
pip-chill
sphinx
scipy
If I then decide to use a custom Sphinx theme, namely The PyData Sphinx Theme, I will install it in the environment as follows:
-
using pip:
python -m pip install pydata-sphinx-theme -
using conda:
conda install pydata-sphinx-theme --channel conda-forge
That’s an additional package, which will change the project’s requirements. Running pip-chill again updates the requirements list. It replaces sphinx with pydata-sphinx-theme as the former is now a dependency of the latter. Add the --verbose option to the command to list the dependencies in the form of comments:
$ pip-chill --verbose --no-version
bokeh
descartes
geopandas
pip-chill
pydata-sphinx-theme
scipy
# alabaster # Installed as dependency for sphinx
# attrs # Installed as dependency for fiona
# babel # Installed as dependency for sphinx
# certifi # Installed as dependency for requests
# chardet # Installed as dependency for requests
# click # Installed as dependency for cligj, click-plugins, fiona
# click-plugins # Installed as dependency for fiona
# cligj # Installed as dependency for fiona
# cycler # Installed as dependency for matplotlib
# docutils # Installed as dependency for sphinx
# fiona # Installed as dependency for geopandas
# idna # Installed as dependency for requests
# imagesize # Installed as dependency for sphinx
# jinja2 # Installed as dependency for sphinx, bokeh
# kiwisolver # Installed as dependency for matplotlib
# markupsafe # Installed as dependency for jinja2
# matplotlib # Installed as dependency for descartes
# munch # Installed as dependency for fiona
# numpy # Installed as dependency for bokeh, pandas, scipy, matplotlib
# packaging # Installed as dependency for sphinx, bokeh
# pandas # Installed as dependency for geopandas
# pillow # Installed as dependency for bokeh, matplotlib
# pygments # Installed as dependency for sphinx
# pyparsing # Installed as dependency for packaging, matplotlib
# pyproj # Installed as dependency for geopandas
# python-dateutil # Installed as dependency for pandas, bokeh, matplotlib
# pytz # Installed as dependency for pandas, babel
# pyyaml # Installed as dependency for bokeh
# requests # Installed as dependency for sphinx
# shapely # Installed as dependency for geopandas
# six # Installed as dependency for fiona, munch, python-dateutil, packaging, cycler
# snowballstemmer # Installed as dependency for sphinx
# sphinx # Installed as dependency for pydata-sphinx-theme
# sphinxcontrib-applehelp # Installed as dependency for sphinx
# sphinxcontrib-devhelp # Installed as dependency for sphinx
# sphinxcontrib-htmlhelp # Installed as dependency for sphinx
# sphinxcontrib-jsmath # Installed as dependency for sphinx
# sphinxcontrib-qthelp # Installed as dependency for sphinx
# sphinxcontrib-serializinghtml # Installed as dependency for sphinx
# tornado # Installed as dependency for bokeh
# typing-extensions # Installed as dependency for bokeh
# urllib3 # Installed as dependency for requests
The last thing to do is to save the output of the terminal to a requirements.txt file. pip-chill isn’t considered a dependency of the project; it’s just a utility for creating this file. So, I removed it from the list using the --no-chill option.
pip-chill --no-version --verbose --no-chill > requirements.txt