You are here

Installing software yourself on DICE

Printer-friendly versionPrinter-friendly version
This document is a work in progress. If you think we could add anything useful to this page please get in touch using the usual contact form.

Although DICE prohibits its users from becoming root, this is not usually a limitation to installing software yourself. Most normal installations, including those where software must be built from source, are able to install themselves into your home directory with a few modifications to the build process.

Most software installation follows a similar pattern. In general the only thing that must be changed is that the install prefix (that is, the base directory into which all files are placed) be changed from /usr or /usr/local to $HOME/usr/ or similar. Alternatively, some software packaging environments have their own special flags to very simply allow per-user installations.

Automated installation

Several tools exist for users of different languages to distribute library packages, particularly where full compilation is not required. In these cases software needn't even be downloaded or unpacked: one command should do it all.

Emacs: the MELPA repository

Emacs users can install their own packages from the comprehensive MELPA repository. To set this up, put the following lines into your preferred emacs startup file (~/.emacs or ~/.emacs.d/init.el) then restart emacs:

(require 'package)
(add-to-list 'package-archives (cons "melpa-stable" "https://stable.melpa.org/packages/") t)
(package-initialize)

To browse the repository and install packages:

M-x list-packages

Python: pip or easy_install

See also the more general information on our python environment on DICE for instructions including the use of alternative python versions and the virtualenv tools.

DICE carries both the pip and easy_install tools which allow automated download, build and installation of python packages to your home directory. This is usually as simple as typing one of:

$ pip install --user packagename

It's much the same in python 3:

$ pip3 install --user packagename 

— or —

 
$ easy_install --prefix=$HOME/.local packagename

to install your package for local use. Remember, if the package includes an executable you may wish to add $HOME/.local/bin to your PATH (see below).

Ruby: rubygems

DICE carries a version of rubygems which allows automated user installation of ruby packages. Typically all that is required is

$ gem install --user-install packagename

See gem help install for full details.

Perl: CPAN

Using CPAN is rather unfortunately a skill in itself, but in theory it is very simple to install new software in this way. Simply type

$ cpan packagename

for the package of your choosing, and follow the prompts while CPAN sets up your environment, then downloads and installs the software for you.

Haskell: cabal

Cabal is a package database for GHC which automates fetching, configuration, compilation and installation of Haskell libraries and programs. On DICE it is installed as part of the Haskell Platform. Only modules prepared using Cabal can be installed this way and they must appear in Hackage. To install a package first update the database with

$ cabal update

You can then list the available packages with

$ cabal list

and install them with

$ cabal install 

If there are dependencies then cabal will automatically try to fill them by download/compile if required. You can see what packages are currently installed by running

$ ghc-pkg list

By default cabal installs packages locally (i.e. in your home directory under ~/.cabal) so you will need to have some free quota for them to install properly. For more details see the cabal install page.

Building it yourself

If the repositories above don't have the software you require, it is assumed that you will have instead downloaded a piece of software, typically as a .zip or .tar.gz or similar, and unpacked it into a temporary directory. It is also assumed that you have downloaded the software from a trusted source. Step one is to open a terminal and cd to that directory.

The next step is to identify what sort of software and build system the software uses. This can be determined by the presence or absence of a few files in the root of the unpacked files (or by reading the installation guide, typically on the application website or in a file named INSTALL or README.

The autoconf / automake process

A great deal of software is distributed as source files and uses a loosely standardised process for compilation and installation, sometimes known as the autoconf toolchain. You can usually tell this process by the presence of files such as 'configure', 'autoconf.*', 'Makefile.in' or the like. It generally doesn't hurt (but see the note below about --dry-run to try a sequence along the lines of:

$ autoconf     # (usually not required)
$ ./configure --prefix=$HOME/usr/
$ make         # sometimes with the PREFIX variable set
$ make install # sometimes with the PREFIX or INSTALL_DIR variables set

checking carefully for errors along the way. If the ./configure portion of the process is not possible or necessary, then it is usually necessary to set the PREFIX parameter in the Makefile to a directory in your home directory. It would also be wise to check first what the install process is planning on writing, since poorly-configured scripts could cause widespread damage to your home directory. You can do this with the --dry-run argument to make / make install.

Python: with setuptools

If your downloaded software takes the form of a python library, very often it will use the setuptools system to automate installation. It's easy to see if the package uses setuptools: look for a file called setup.py in the root of the pacakge. If this is the case, installation is usually very simple.

$ python setup.py install --user

this will install a copy of the software into your home directory and automatically add it to your python path. If the package includes executables, for example in ~/.local/bin, you might need to add this directory to your PATH (see below).

Lua: with luarocks

Users can install their own packages using the lua "rocks" system:

$ luarocks install --local twitter

remember you will need to add $HOME/.luarocks/bin to your path if your package includes an executable.

After Installation

Paths

Following installation, you might find that the executables you want to run aren't accessible from your shell. This is usually because the path to which the files have been installed, for example ~/.local/bin or ~/usr/bin/, haven't been added to your system path. The good news is that this only needs to be fixed once: simply add the relevant bin path in a line to your ~/.brc file, like:

PATH=$PATH:$HOME/path/to/local/bin/dir

or

PATH=$HOME/path/to/local/bin/dir:$PATH

if you'd prefer the new binary to be executed in preference to a system-provided file.


Haskell (ghc)


If the Haskell platform installed on DICE is not appropriate it is possible to install a different version of Haskell in a local directory (~ or /disk/scratch or /group/...) and configure the package management to use this in preference to the rpm install.

ghc

First download the appropriate binary for ghc, into a temporary directory, untar the tar file, cd into the directory this creates and then run ./configure pointing it at the directory you want to install ghc into

$ tar jxvf ghc-7.8.3-x86_64-unknown-linux-centos65.tar.bz2 
$ cd ghc-7.8.3/
$ mkdir /disk/scratch/ghc-latest
$ ./configure --prefix /disk/scratch/ghc-latest/

This will set ghc up to install into a series of directories under /disk/scratch/ghc-latest but you can use any directory you have write permissions on. Then run

$ make install

to install the binaries. Finally add the <prefix>/bin directory to your PATH environmental variable by putting

$ export PATH=/bin:$PATH

in your .brc file.

cabal-install

The haskell interpreter/compiler installed above doesn't come with all of the packages in haskell-platform the best way to manage packages is with cabal-install.

First you need to find the version of the Cabal library that has been shipped with your binary copy of ghc

$  ghc-pkg list|grep Cabal
    Cabal-1.18.1.3
$

You then need to pull down, build and install a matching version of cabal-install. Normally this will be the latest version but if you have issues choose the version of cabal-install closest to the version of the Cabal library you have installed.

First download the source tarball and untar it

$ wget http://www.haskell.org/cabal/release/cabal-install-1.20.0.3/cabal-install-1.20.0.3.tar.gz
--2014-08-18 12:18:01--  http://www.haskell.org/cabal/release/cabal-install-1.20.0.3/cabal-install-1.20.0.3.tar.gz
Resolving www.haskell.org... 88.198.224.243
Connecting to www.haskell.org|88.198.224.243|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 274779 (268K) [application/x-gzip]
Saving to: `cabal-install-1.20.0.3.tar.gz'

100%[======================================>] 274,779     1.42M/s   in 0.2s    

2014-08-18 12:18:02 (1.42 MB/s) - `cabal-install-1.20.0.3.tar.gz' saved [274779/274779]

$ tar zxf cabal-install-1.20.0.3.tar.gz 

cd into the resulting directory and using an editor edit the bootstrap.sh file. Change the DEFAULT_PREFIX path to match where you have installed ghc previsously

...
do
  case "${arg}" in
    "--user")
      SCOPE_OF_INSTALLATION=${arg}
      shift;;
    "--global")
      SCOPE_OF_INSTALLATION=${arg}
      DEFAULT_PREFIX="/disk/scratch/ghc-latest"
      shift;;
    "--no-doc")
      NO_DOCUMENTATION=1
      shift;;
...

then build cabal by running

$ ./bootstrap.sh --global

This will download and build any package dependencies that are required and may take some time. At the end of the process you should have a cabal binary installed alongside the ghc and ghci binaries you've already installed.

Finally edit ~/.cabal/config and change the -- prefix line in the install-dirs global section to point at where you've installed ghc, in our case this looks like

install-dirs global
  -- prefix: /disk/scratch/ghc-latest
  -- bindir: $prefix/bin
  -- libdir: $prefix/lib
  -- libsubdir: $pkgid/$compiler
  -- libexecdir: $prefix/libexec

you should now be able to install ghc packages for global use via cabal.


See also more general information on software within informatics.

Last reviewed: 
22/04/2016

System Status

Home dirs (AFS)
Network
Mail
Other services
Scheduled downtime

Choose a topic