Department of Engineering

IT Services

Using Python on CUED ts-access Servers

Scope

This document written by mal84 in 2019 with a few adjustments by support staff is intended as a quick guide to aid using Python on the Engineering Department servers (which run CUED's central computing service), in order to provide access to additional computing resources for projects. It based around Python3 in mac OSX, however where appropriate links to instructions for using Windows and Linux are provided.

Considerate use

As these servers run the computers around the department, this service is mainly aimed at usage outside of teaching hours (i.e. outside when you would expect students to be in the department using terminals in term time). Even outside teaching hours, the servers can sometimes be needed by Part II undergraduates doing projects. We depend on community spirit (and user complaints) to deal with excess use.

General Info

The department provides some general information -

Getting Started

The following method of accessing the servers on macOS described is a non GUI method and operates through a Linux terminal interface. Windows and Linux access is described at the links provided.

  1. If you want to use graphics, check that you have X11 or Xquartz installed (if it is, it will be located in the Utilities folder in the Applications dock). If not download and install it here.
  2. Start the application (a window may not pop up), and open up an xterm window (shortcut ⌘+N) which will look like this.
  3. Type slogin -Y usr123@gate.eng.cam.ac.uk and press return (note that you should use your CRSiD instead of usr123).
  4. You will then be prompted to enter your Engineering Department computer password, do so and press enter.
  5. Then type slogin -Y ts-access and press return, and then again enter your password. This will take you to a ts-access server, however if you wish to access a specific server, replace ts-access with the name of said server.
  6. At this stage you are now logged on to the server and the xterm window now acts as a Linux terminal window for the server. The steps thus far will look like this

Which Python?

By default on the central system, typing python runs Python 3.

Installing your own Anaconda distribution

If you feel you need to have your own private version of Anaconda on the CUED linux machines, you can download the latest version from here, and install it (preferably in the standard location).

Now we need to alter the PATH in the terminal window in order to access this version of Python. First we need to locate our installation of Anaconda. Use the ls command to look at your files.

Now we need to adjust the PATH.

Enter the command export PATH=/homes/usr123/anaconda3/bin:$PATH

Note that the path you require may be different depending on where you’ve installed your anaconda. I’ve installed it in my home folder for simplicity.

Now we can check that all has gone well by using the commands which python and which conda

python --version and conda --version can be used to confirm which versions of Python and Anaconda you have just installed.

Also we can check which modules (and which versions) we have installed with this Anaconda install by using the conda list command.

From this PATH you can now use all conda commands as normal.

If you require a newer version of any given module than is installed you can use

pip install modulename --upgrade to do so. Similarly, you can pip install new modules by omitting the --upgrade.

We are now in a position where we can run our scripts as usual by using cd to navigate to the appropriate folder and python filename.py to run.

Checking Modules

Check whether you have all the modules you require by entering the command pydoc3 modules into the console, which will return a list of all the modules the server has installed.

If you are having issues with your functions not behaving as expected, this may be due to your code being written with a newer version of the module.

If however the server has all you require, you can now run python scripts as you would normally through terminal, navigating to them using the cd and ls commands, and executing them using python filename.py

Transferring Files

Files can be transferred to the server using the command scp filename.extension usr123@gate.eng.cam.ac.uk:. and retrieved from the the server using scp usr123@gate.eng.cam.ac.uk:filename.extension . This process is detail at the bottom of this page.

However it is highly recommended that you download the Filezilla client, which is a convenient graphical tool for transferring files to the server and makes life much easier and quicker, as you can drag and drop files from your machine.

At this point it is important to note that the servers are the department computing service, and so the file structure on the server is that of your account space. Hence if you wish files can be loaded via Google Drive or whichever preferred method, you can do so by logging on to one of the DPO computers and saving them in your area.

Filezilla has a quick connection function and should be filled out as follows,

Once you’ve logged in once, the circled button can be used to reconnect automatically.

Drag and drop files either way to upload and download. You can delete files via Filezilla, however be careful as there is no trash (that I’ve found).

For more detailed instructions for Filezilla see here.

Standard Usage

Once we have done the setup, a normal login and usage of files will look like this,

Saving Figures and Files

Whilst Xquartz will give you pop-ups through which you can interact with your code outputs, it is sensible to save any figures or data generated to the server.

When using matplotlib use pyplot.savefig - documentation here. It is important to note that this must be placed before plt.show() for it to work properly.

When working with numpy data structures use numpy.save - documentation here.

GUI Interaction

The CUED information details methods of getting graphical interfaces with the servers, which operate much like a CUED terminal. However, apps can be opened with the method detailed. The Anaconda navigator can be opened using the command anaconda-navigator, however care must be taken with setting the path to ensure the right version of Anaconda is accessed.

Further, using the apps this way can be very laggy and I do not recommend it.

Retaining Connection

If Python files are run in this way, then as soon as the xterm window is closed, the server will stop the running code. To get round this the nohup command can be used, see this StackExchange thread.

Getting Help

There are many ways to get help round the department:

  1. Read through the detailed instructions written by the department here
  2. Go to the computing helpdesk in the DPO.
  3. Email me at mal84
  4. Email IT Services at helpdesk@eng.cam.ac.uk

Acknowledgements

Many thanks to John Sloan from IT Services and The Computing Helpdesk for their invaluable help and advice.

Errors and Corrections

If you spot an error or think there’s an area of instructions that could be improved, please email me at mal84 and inform the person from whom you received this document.