Contents


Preamble


Installing Python packages locally

The process of setting up Python for your personal use and needs consists of first choosing a Python distribution and setting up the environment using modules, and second adding any custom packages to your environment locally. These two steps are discussed below.

Choosing a Python distribution and setting up the environment

As with all user-selectable system-supplied software at the HPCC, modules are used to select the Python distribution to be used and set up a user's environment. Typically users initialize their environment when they log in by setting environment information for every application they will reference during the session. The HPCC uses the Environment Modules package as a tool to simplify shell initialization and allow users the ability to easily modify their environment during the session with module files.

For more information about how to load and maintain your software environment using modules, please refer to the user guide "Software Environment Setup".

In order to select a distribution and set the environment variables for Python you will first need to check which versions of Python are available using the "module spider python" command.  This command will return a description of the software and which versions are currently installable through the modules system.

The default version of python at this time loaded by "module load intel python" or "module load gnu python" is Python 2. To load the most recent available system-installed version of python you can run "module load intel python3" or "module load gnu python3" command. You can then run the "module list" command to verify that the Python module has been successfully loaded. This can be done using one of the following sets of commands:

#To load the Intel compiled version of Python run the following:
module load intel python # For older version

# OR
module load intel python3 # for newer version

# check which is loaded
module list
python --version

# OR

#To load the GNU compiled version of R run the following:
module load gnu python # For older version

# OR
module load gnu python3 # for newer version

# check which is loaded
module list
python --version

In general, the Intel distribution of Python is more fully featured and has tools available to improve multi-threaded coding, which can be important to optimize the performance of long-running Python applications. The Gnu distribution may be more portable for moving your application to other systems, as it does not require a commercial license. Most Python code will function equally well with either distribution. You can alternatively use the Conda package manager as described below to install a custom version of Python and manage packages in user-controlled environments.

A good starting point if you don't know how to choose from the above options is Python 3 in the Intel distribution, which can be selected using "module load intel python3".

Installing packages locally to supplement or replace a system distribution

By default, Python packages require the installation be performed by the 'root' user.  However, most python package installers and managers will also allow the user to install the package into their HOME folder to supplement the features of these distributions.