Installing Pyinteraph for python 2.7 with Conda

Pyinteraph is currently distributed as a python 2.7 package, although the authors are working on a python3 version. This raises a few problems, since python 2.7 will be phased out shortly and python3 is not backward compatible. As Python 2.7 packages are still an issue in Academia, you can find a walk-through for Pyinteraph installation below. The general steps are valid also for other packages.

what to expect

The typical installation procedure for legacy software.

Install locally

We will install this on the cluster since there you already have some of the requirements in the moduli. The same procedure should work on your laptop too with minimal modification (i.e. you will have to install hdf5 and netcdf by yourselves).

First of all, download the source got from its github repo. In order for the distributed pyinteraph to work, you need to make it believe it nothing has changed around it since the last published version. Check the dates and modifications on the commits history of the repo.

When you download the software, you will notice in the INSTALL file a list of dependencies (look at the instructions for Linux). Sometimes you do not have the proper version number of a module or library, so you will have to take a guess at the right one. To do so, go check pypi, search the package you are interested in, and look for its release history. You will have to take note of the last version of the package prior to the distribution date of pyinteraph.

To save you some work, this is the correct list of software you will need, set up to create a python environment simulating late 2013.

Non-python libraries needed

In the cluster, you can get those from moduli. On your laptop, check the apt-get instructions from the INSTALL file (or equivalent for MacOs).

  • HDF5
  • Netcdf4
  • Gcc These are available as modules on the cluster. You can load them with:
    module load netcdf-4.7.0--gcc-9.1.0
    module load hdf5-1.8.18
    

Python & python packages

As written above, you will need python2.7 plus

  • scipy
  • matplotlib
  • networkx
  • cython
  • biopython
  • netCDF4
  • MDAnalysis

pyinteraph scheme

Pyinteraph is unfortunately still implemented in python2.7 (a new version is going to come out soon).

To install the correct version of them (2013..) we will have to create another environment with conda, and then install things there with pip, the python package manager.

To create a conda environment named pyint and based on python 2.7, run:

Conda create -n pyint python=2.7

Or create a .yml file as follows,

name: pyint
channels:
    - conda-forge
dependencies:
    - python=2.7
    - jupyter
    - pip

And run it with

conda env create --file myfile.yml # substitute the correct name of course

The second step is to install the missing packages. To do so, first you need to activate your environment:

conda activate pyint

Once you are in, you have to create two requirements files for pip. The first will have numpy, as it is a requirement for everything else.

numpy==1.7.0
GridDataFormats==0.2.2

Then you have to create a file requirement-1.txt, containing:

scipy==0.12.0
matplotlib==1.3.0
networkx==1.8
cython==0.19
biopython==1.61
netCDF4==1.0.8
MDAnalysis==0.7.6

And install everything with pip install -r requirements-1.txt.

What errors do you get? What do they mean?

You have to create the requested environment variable to tell the compiler where to find the headers (the .h files) it needs. With module you can find out where the header files are stored using

module show <module-name> #again, substitute the correct name

Once you found the directory, you have to export the variables and re-run pip.

OK. Now, we have all the correct modules installed! We can thus install the package. You have to run setup.py telling it to install things locally.

python setup.py install —user
#install in home/name.surname/.local

We can now try to run the examples in the examples folder. Of course, we will have to modify the tutorial.sh file. In two ways. One, to let it know where to find pyinteraph and libinteract, as per instructions of the developers; two to make it so that bash knows where to look for the python environment you created.

export PYTHONPATH=$PYTHONPATH:$HOME/.local/lib/python2.7
export PYINTERAPH=$HOME/.local/pyinteraph
export PATH=$PATH:$PYINTERAPH
# making conda visible
source /home/luca.tubiana/usr/src/miniconda3/etc/profile.d/conda.sh
# activating the environment
conda activate pyint

why do you need to make the script aware of conda?

Ok, we corrected this issue as well. Let’s run the script (again).

Question: why does it crash now?

Suggestion: add which python to tutorial.sh before the first call. Then open pyinteraph with vim and check the very first line.

#!/usr/bin/python  #never do this!

Check all the files called by tutorial.sh. Open them with a text editor and change #!/usr/bin/python with #!/usr/bin/env python. Install again the package.

The first line of a script, when it starts with a #! is called a “shebang” and tells the system what interpreter to use. The difference between the two lines is that the original one tells the system to use a specific python installation, the system-wide one; the second line specifies instead to use whatever python has been activated in the current environment, i.e. conda in your case.

Extra points: upload everything to your gitlab repo!