Installing Pyinteraph for python 2.7 with Conda
Pyinteraph is currently distributed as a python 2.7 package, although the authors are working on a python3 version. This raises a few problems, since python 2.7 will be phased out shortly and python3 is not backward compatible. As Python 2.7 packages are still an issue in Academia, you can find a walk-through for Pyinteraph installation below. The general steps are valid also for other packages.
The typical installation procedure for legacy software.
Install locally
We will install this on the cluster since there you already have some of the requirements in the moduli. The same procedure should work on your laptop too with minimal modification (i.e. you will have to install hdf5 and netcdf by yourselves).
First of all, download the source got from its github repo. In order for the distributed pyinteraph to work, you need to make it believe it nothing has changed around it since the last published version. Check the dates and modifications on the commits history of the repo.
When you download the software, you will notice in the INSTALL
file a list of
dependencies (look at the instructions for Linux). Sometimes you do not have
the proper version number of a module or library, so you will have to take a
guess at the right one. To do so, go check pypi, search the
package you are interested in, and look for its release history. You will have
to take note of the last version of the package prior to the distribution date
of pyinteraph.
To save you some work, this is the correct list of software you will need, set up to create a python environment simulating late 2013.
Non-python libraries needed
In the cluster, you can get those from moduli. On your laptop, check the
apt-get instructions from the INSTALL
file (or equivalent for MacOs).
- HDF5
- Netcdf4
- Gcc
These are available as modules on the cluster. You can load them with:
module load netcdf-4.7.0--gcc-9.1.0 module load hdf5-1.8.18
Python & python packages
As written above, you will need python2.7
plus
- scipy
- matplotlib
- networkx
- cython
- biopython
- netCDF4
- MDAnalysis
Pyinteraph is unfortunately still implemented in python2.7 (a new version is going to come out soon).
To install the correct version of them (2013..) we will have to create another environment with conda, and then install things there with pip, the python package manager.
To create a conda environment named pyint
and based on python 2.7, run:
Conda create -n pyint python=2.7
Or create a .yml
file as follows,
name: pyint
channels:
- conda-forge
dependencies:
- python=2.7
- jupyter
- pip
And run it with
conda env create --file myfile.yml # substitute the correct name of course
The second step is to install the missing packages. To do so, first you need to activate your environment:
conda activate pyint
Once you are in, you have to create two requirements files for pip
.
The first will have numpy, as it is a requirement for everything else.
numpy==1.7.0
GridDataFormats==0.2.2
Then you have to create a file requirement-1.txt
, containing:
scipy==0.12.0
matplotlib==1.3.0
networkx==1.8
cython==0.19
biopython==1.61
netCDF4==1.0.8
MDAnalysis==0.7.6
And install everything with pip install -r requirements-1.txt
.
What errors do you get? What do they mean?
You have to create the requested environment variable to tell the compiler where
to find the headers (the .h files) it needs. With module
you can find out
where the header files are stored using
module show <module-name> #again, substitute the correct name
Once you found the directory, you have to export
the variables and re-run pip
.
OK. Now, we have all the correct modules installed! We can thus install the
package. You have to run setup.py
telling it to install things locally.
python setup.py install —user
#install in home/name.surname/.local
We can now try to run the examples in the examples
folder. Of course, we will have
to modify the tutorial.sh
file. In two ways. One, to let it know where to
find pyinteraph and libinteract, as per instructions of the developers; two to
make it so that bash knows where to look for the python environment you
created.
export PYTHONPATH=$PYTHONPATH:$HOME/.local/lib/python2.7
export PYINTERAPH=$HOME/.local/pyinteraph
export PATH=$PATH:$PYINTERAPH
# making conda visible
source /home/luca.tubiana/usr/src/miniconda3/etc/profile.d/conda.sh
# activating the environment
conda activate pyint
why do you need to make the script aware of conda?
Ok, we corrected this issue as well. Let’s run the script (again).
Question: why does it crash now?
Suggestion: add which python to tutorial.sh before the first call. Then open pyinteraph with vim and check the very first line.
#!/usr/bin/python #never do this!
Check all the files called by tutorial.sh
. Open them with a text editor and
change #!/usr/bin/python
with #!/usr/bin/env python
. Install again the
package.
The first line of a script, when it starts with a #!
is called a “shebang” and
tells the system what interpreter to use. The difference between the two lines is
that the original one tells the system to use a specific python installation, the
system-wide one; the second line specifies instead to use whatever python has been
activated in the current environment, i.e. conda in your case.
Extra points: upload everything to your gitlab repo!