Singularity

Singularity is a container platform, perhaps the most well-supported Docker alternative. It allows you to run containers that package up pieces of software in a way that is portable and reproducible. You can build a container using Singularity on your laptop, self-managed desktop or a virtual machine and then run it on a DICE desktop or server or even move it to the cloud or a HPC cluster.

Singularity is currently available on all Ubuntu Teaching Cluster nodes (landonia nodes)

Full documentation can be found here: Singularity user's guide

Using Singularity on DICE

Singularity is only available on DICE Ubuntu.

Check that Singularity is installed by simply using:

singularity --version

If this returns command not found please raise a ticket requesting for its installation on your DICE machine.

The most common usage of singularity is to download pre-built images from an external resource like the Container Library or Docker Hub. You can use search command to locate groups, collections, and containers of interest on the Container Library.

This will output an extensive list of all containers on the Container Library built around tensorflow.

singularity search tensorflow

You can use the pull command to download pre-built images.

singularity pull library://lolcow

for Container Library images or

singularity pull docker://sylabsio/lolcow

for Docker Hub images

Singularity and AFS

When pulling images from Container Library a single image file (.sif) is downloaded. When pulling images from Docker Hub the image will be built on the fly. For this reason Singularity uses temporary and cache folders. As the default location for these folders is inside your home folder (~/.singularity) this might result in running out of AFS space. Singularity offers a solution to this by using two environmental variables SINGULARITY_TMPDIR and SINGULARITY_CACHEDIR. Please consider adding these to your .brc or .bashrc file. The below is just an example:

export SINGULARITY_TMPDIR=/disk/scratch/singularity/temp
export SINGULARITY_CACHEDIR=/disk/scratch/singularity/cache

Interacting with images

The shell command allows you to spawn a new shell within your container and interact with it as though it were a small virtual machine.

singularity pull library://lolcow
singularity shell lolcow_latest.sif 
Singularity>

The change in prompt indicates that you have entered the container.
Once inside of a Singularity container, you are the same user as you are on the DICE machine.

The exec command allows you to execute a custom command within a container by specifying the image file. For instance, to execute the cowsay program within the lolcow_latest.sif container:

singularity exec lolcow_latest.sif cowsay moo
 _____
< moo >
 -----
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Singularity containers contain runscripts. These are user defined scripts that define the actions a container should perform when someone runs it. The runscript can be triggered with the run command, or simply by calling the container as though it were an executable. Both commands below will give the same result.

singularity run lolcow_latest.sif

or

./lolcow_latest.sif

Working with files

By default Singularity bind mounts your home folder, /tmp and current working directory into the container at runtime. You can specify additional directories to bind mount into your container with the --bind option. e.g:

singularity shell lolcow_latest.sif 
Singularity> ls /disk/scratch
ls: cannot access '/disk/scratch': No such file or directory
Singularity> exit
singularity shell --bind /disk/scratch lolcow_latest.sif 
Singularity> ls /disk/scratch/
lost+found  miniconda3

Working with GPUs

Singularity does not bind GPU(s) to containers by default. Please use --nv parameter to do so.

singularity shell --nv lolcow_latest.sif 
Singularity> nvidia-smi
Thu Jun 30 13:39:59 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.129.06   Driver Version: 470.129.06   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |

                                      (...)
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Build images from scratch

To build containers requires a machine you have root privileges for. As this is not the scope of this quick start guide please refer to Singularity user's guide.

For any issues or further help please use the computing support form.