Installing Python Libraries and Packages

Overview

When installing Python libraries, there are two general approaches. One will install packages into the local user library directory, using the pip command, while the other involves creating virtual environments. In the first, packages are available to all Python scripts run by that user, without having to activate a specific environment. This is a quick way to install packages with minimal effort. Using the local library approach is generally sufficient if you are working on only a single project, but not robust enough when working on multiple projects that might require different versions or dependencies. The second approach, using virtual environments, creates an isolated Python environment that allows packages to be installed for use by a particular application or for a particular project.

Common tools for Python package installation and environment management include:

• pip - the Python package installer, can be used on it's own or within a virtual environment
• venv - an environment manager within which pip is used to install packages
• conda - a Python-based package installer and environment manager that does not rely on pip
• mamba - a C++ based drop-in replacement for conda that provides numerous advantages leading to substantially faster performance

Before getting started, remember to load an appropriate module for the Python distribution version that you want to use. The following example shows how to load a miniforge module that provides all of the above tools along with Python version 3.12.12 (the $ is the prompt and should not be typed):

$ module purge
$ module load miniforge/25.11.0-py3.12

If you wish to use any additional packages that are not included with the pre-installed libraries, you can easily install those Python packages either locally, with pip, or you can create a virtual environment with either venv, conda, or mamba.

Pip

To install to your local Python library, use pip as follows:

pip install --user <package>

For example, to install the PyVCF package, you would enter:

pip install --user pyvcf

A local package installation (--user) will, by default, be placed in the following path:

~/.local/lib/python*/site-packages`

where * indicates the python version. The library will then be available to you for this and future sessions.

Venv

Venv is a tool included with Python that allows you to create lightweight virtual environments. The Python venv module uses the same Python interpreter (and, therefore, the same Python version) with which it is executed as the base for the new environment. It does this via a symbolic link that is created in the bin directory of the virtual environment. By default, pip is provided with all venv virtual environments. Installing packages into an active venv is done via the pip command. In this case, however, you should not include the --user option.

Create a virtual environment in your project folder as follows:

$ python -m venv /path/to/new/virtual/environment/myenv

Here, myenv is the name of the environment folder. You can replace it with any name. Alternatively, you can change into the directory of the project you are working on and simply provide a name for the virtual environment in place of the full path:

$ cd /path/to/my/project
$ python -m venv myenv

The next step is to activate the virtual environment, as follows:

$ source myenv/bin/activate

If you are not in your project directory, then you must provide the full path to the virtual environment:

$ source /path/to/my/project/myenv/bin/activate

When activated, the virtual environment's bin directory is prepended to the PATH environment variable. An activated virtual environment will also change your prompt to have the name of the environment in parentheses before the rest of the prompt. Your prompt will likely look like the sample below (the hostname may be different).

(myenv) [yourNetID@vacc-login1 ~]$ which pip
/path/to/my/project/myenv/bin/pip

At this point, you would use the pip command to install any needed packages. Packages that you install using pip while in a virtual environment will be placed in the myenv folder, isolated from the global Python installation, and only available to you from within the virtual environment.

Install packages into your environment with pip as shown below.

$ pip install numpy

When installing into a virtual environment using the pip command, it is important that you do not include the --user option. That option is only used to install to your local library.

You can deactivate the environment, when done, simply by typing deactivate in your terminal:

$ deactivate

The venv tool is straightforward and efficient for basic Python projects. It’s lightweight, and it’s included by default with Python installations.

Conda/Mamba

Conda is a widely-used package and environment manager for Python, R, and other languages. Mamba is a reimplementation of Conda in C++ with parallel operations and other efficiencies. Mamba typically provides faster downloads and superior handling of large environments due to the fact that Mamba is a complete C++ implementation, while Conda is still largely Python-based.

The subcommands in mamba for creating and managing environments are the same as those for conda. If you are already familiar with the use of Conda, then transitioning to Mamba is as simple as swapping out the use of the conda command for mamba. Additionally, Mamba is fully compatible with the Conda ecosystem and you can use both tools interchangeably.

Note, for configuring environments, mamba config options are similar, but not exact replacements for conda config commands. For example, instead of

$ conda config --add channels bioconda

you would use

$ mamba config prepend channels bioconda

For configuration, it's probably best to use the more well known and documented conda config commands. To see more info on mamba config options try mamba config -h to print a help message, or reference the official Mamba documentation.

In the example below, we show a common sequence of commands using Mamba. If you chose, instead, to use Conda you would simply replace mamba with conda in each command. Unlike the venv environment manager, when you create a Python environment using Conda or Mamba, the new environment is created with its own Python interpreter installed into the environment. It is not limited to the version of Python that is used to create the environment. Therefore, you can explicitly specify the desired Python version when creating your environment. Since Conda/Mamba can be used to create environments for other languages, you need to specify that Python itself should be installed when creating the environment. If you do not specify a version, the latest available Python from the configured channel will be installed. To ensure predictable results, it is recommended to always explicitly specify the desired Python version.

To create a new Python environment, specifying it's version, and simultaneously install the numpy package:

$ mamba create -n myenv python=3.13 numpy

You can and, ideally, should list all packages that you want to include in an environment at the same time in order to avoid dependency conflicts. You can do this when creating the environment. If you need to install additional packages to an already existing environment, you must first activate the environment.

To activate the myenv environment:

$ mamba activate myenv

Similar to venv virtual environments, the activate command modifies your session's PATH to prepend the environment's bin directory. Additionally, the command prompt will update to show the environment name in parentheses.

You can install additional packages into an activated environment as follows:

$ mamba install scipy requests

Finally, to deactivate the environment:

$ mamba deactivate

Virtual environments created with mamba/conda when using a VACC module reside, by default, in the envs directory found in the following path:

$HOME/.conda/envs

If you are, instead, using mamba/conda provided by your own installation of Miniforge or other distribution, the default envs directory will likely be located in the root of that installation.