Installing ipkernel to access virtual environments in Jupyter
The VACC Open OnDemand Jupyter app uses Python version 3.12 and provides only a very limited set of packages. In order to use additional packages in the Jupyter app, we recommend that you create virtual environments that provide the required Python packages for each project and then access those environments via a Jupyter kernel.
This page gives a step-by-step guide to installing a particular set of
Python packages in a Conda virtual environment and then setting that
virtual environment up so it can be used within our Open OnDemand
JupyterLab/Jupyter Notebook application. Similarly, a virtual Python
environment that was created with venv can be accessed in Jupyter
via a kernel.
Create a Conda virtual environment and install the kernel¶
These instructions assume that you are starting with a 'clean slate',
by which we mean there are no active Conda environments. This also
assumes that you want to create the virtual environment in the
standard path ~/.conda/envs. If you want to save it elsewhere, see
the official
documentation
for details on setting a --prefix. All of the commands that follow
are preceded by a $ prompt. If you copy and paste, do not copy the
prompt. Lines that start with a # are comments that simply provide
information. To be extra safe, we deactivate any Conda
environments. If you don't have an active Conda environment, running
the deactivate command is harmless and will simply tell you that
The command: conda could not be found. This is a safety measure for
those for whom it is found.
$ conda deactivate
The first thing we want to do is to insure that only the module(s) needed for our project, which is biology for this example, are loaded. To do that we first remove all loaded modules, then load the one(s) we need.
Next, we create the environment, making sure to specify the Python
version that we want to use explicitly and to install the ipykernel
package. In this example, we name our environment my_bio_env. You
can substitute any name that is appropriate for your
environment. Additionally, note that the name given to the kernel does
not have to match the name of the Conda environment. We use the same
name in our example below for simplicity. Note, we use the mamba
command in place of conda because it is generally much faster at
resolving and installing the environment than conda. The mamba
command is essentially a drop-in for the conda command. The end
result will be the same.
$ mamba create --name my_bio_env python=3.12 ipykernel
# Enter 'y' when asked: Proceed ([y]/n)?
# Activate the environment and install the required packages
$ mamba activate my_bio_env
$ mamba install numpy pySlim scikit-allel msprime
# Enter 'y' again, when asked to proceed
# Installing the kernel
$ python -m ipykernel install --user --name=my_bio_env
# Deactivate when finished
$ mamba deactivate
Using the new Conda environment from a terminal or in a SLURM batch job¶
To use the new Conda virtual environment, in this case named
my_bio_env, from within a terminal, enter the following commands:
If you want to run a batch job that uses the Conda environment, copy
the above commands into your batch script. Do not include the $ at
the beginning of each command.
Accessing Conda environments via a Jupyter kernel¶
Once you have a virtual environment with the kernel installed as described above, you can access that environment from within the Jupyter application in our Open OnDemand web interface to the VACC. The process of accessing the Jupter kernel is slightly different in JupyterLab versus Jupyter Notebook. We provide separate documentation for each at the links below:
Using a kernel in JupyterLab
Using a kernel in Jupyter Notebook