University of Bristol Computer Science Society

SSH and Snowy

Ross Gardiner on May 19, 2016

You are likely to find at some point that you need to work on a department system remotely. In this tutorial, we’ll explore connecting to Snowy, the departmental server.

If you’re familiar with SSH, here are the details you need:

Snowy SSH details

SSH into lab machines

All the lab machines in MVB 2.11 are available when using snowy as a jump box, this involves first connecting to snowy then to the lab machine in a chain like fashion making use of SSH’s ProxyCommand. This step can be skipped if you are using the university VPN.

Lab machine addresses: it<ID>.wks.bris.ac.uk where ID is in the following range:

SSH

Mac/Linux

Open a Terminal/Console window, and type the following command:

ssh [email protected]

(where ab12345 is replaced by your username)

Hit return, and wait for the password prompt. Type in your password and press return again. You are now logged in.

Windows

PuTTY is the easiest solution for SSH on Windows. You can either download the installer (putty-*.**-installer.msi) or putty.exe, which will run standalone.

Open PuTTY, and enter snowy.cs.bris.ac.uk in the ‘Host Name (or IP address)’ field. Make sure the ‘Port’ is set to 22. Click ‘Open’. If you are asked to trust a signature, click yes. You will be greeted by a prompt that says:

login as:

Type your university username and press return. Then type your password and press return.

From a terminal

You will probably end up working in a terminal a lot of the time, even on Windows. Unfortunately, the set of text-based utilities that comes with Windows is rather lacking. Several efforts exist to make these available on Windows:

Figuring out how to get one of these to work is a good exercise for any first year CS student.

Getting a graphical environment

Snowy (like most current Linux systems) supports a feature called X Forwarding. This lets you run graphical applications remotely but use them as if they were running natively(-ish) on your desktop.

Mac/Linux

To enable X Forwarding, add the -X flag when running OpenSSH:

ssh [email protected] -X

Once logged in, you will be able to launch utilities like nautilus (the GNOME file manager) and gedit (the GNOME text editor).

Windows

Windows requires additional steps for X Forwading. Firstly, you need to install Ximg, this allows graphical interface on server to be displayed. If you are using putty, all you should have to do tick the Enable X11 forewarding box under Connections\SSH\X11 in your putty connection. As for Bash on Windows on Ubuntu, Cygwin… etc, you need to add -X flag when running OpenSSH like in Mac/Linux:

ssh [email protected] -X

Remote Desktop (NoMachine)

If you would like to access lab machines or university softwares at home, then SSH with X11 forwarding might not be a good option. This is because X11 forwarding does not compress data, so the graphical interface will be very slow. NoMachine client allows graphical remote desktop with less lag. The server snowy can be accessed with NoMachine, if you desire to use a lab machine, you can still SSH into them using snowy with or without X11 forwarding.

Copying files with scp

Mac/Linux

To copy from your computer to Snowy:

scp sourcefile [email protected]:targetfile

For example, scp ~/main.c [email protected]:~/main.c.

To copy from Snowy to your computer:

scp [email protected]:sourcefile targetfile

For example, scp [email protected]:~/main.c ~/main.c

Windows

On Windows, the easiest graphical method to transfer files is to use SFTP. There are many SFTP clients available. FileZilla is free and works nicely. Use the following settings:

In a terminal

If you want to do this from the terminal, PuTTY comes with a utility called psftp. Alternatively, if you have something like cygwin installed, you probably have access to scp already.

Passwordless login

If you get tired of having to enter your password every time you connect to Snowy, don’t worry! You can create an SSH keypair to verify your identity automatically: Snowy has a copy of your public key, and anyone with access to your private key (i.e. only you) can use it to automatically authenticate.

Mac/Linux

Open a terminal and enter the following:

ssh-keygen -t rsa

The tool will walk you through the process of creating a key. By default, the keypair is saved to the .ssh directory in your home directory - for example /home/myname/.ssh/. The private key is normally called id_rsa, the public key id_rsa.pub - though you can change these if you want. You will be asked to provide a passphrase. This protects your key from viruses, theft etc., so choose something strong.

Once your key is generated, create the .ssh folder on Snowy:

ssh [email protected] 'mkdir -p .ssh'

Then append your public key to the list of authorised keys:

cat ~/.ssh/id_rsa.pub | ssh [email protected] 'cat >> .ssh/authorized_keys'

Windows

PuTTY comes with a tool called PuTTYgen for creating new SSH keys. If you have something like cygwin installed, ssh-keygen is probably available in your shell.

You can copy the public key across using FileZilla, as described above.

Example SSH config

An example SSH config is given below:

# ~/.ssh/config

Host *
    ForwardAgent yes
    ForwardX11 yes
    ForwardX11Trusted yes

#######################################
#         University machines         #
#######################################
# University of Bristol machines
# See http://www.bristol.ac.uk/it-services/locations/zones/zonee/cosc/remote-access

Host snowy
    Hostname snowy.cs.bris.ac.uk
    Port 22
    User <USERNAME>
    IdentityFile ~/.ssh/id_rsa
    ForwardX11 no
    ForwardX11Trusted no

# To use only for Cadence tools
Host frosty
    Hostname frosty.fen.bris.ac.uk
    Port 22
    User <USERNAME>
    IdentityFile ~/.ssh/id_rsa
    ProxyCommand ssh snowy nc %h %p 2> /dev/null

# CS lab machines live on the follow ranges
#
#   - 025956--025995
#   - 052556--052595
#
# CentOS 6 machines:
#
#   - 052569--052572
#   - 052579--052582

Host cs-desktop-01
    HostName it025980.users.bris.ac.uk
    User <USERNAME>
    ProxyCommand ssh snowy nc %h %p 2> /dev/null

Host cs-desktop-02
    HostName it052573.users.bris.ac.uk
    User <USERNAME>
    ProxyCommand ssh snowy nc %h %p 2> /dev/null

Host cs-desktop-03
    HostName it052574.users.bris.ac.uk
    User <USERNAME>
    ProxyCommand ssh snowy nc %h %p 2> /dev/null

Resources

Writing a post for cssbristol.co.uk

Ross Gardiner on May 21, 2016

So you want to write a post for the CSS website? Great! This tutorial will walk you through the process. Basic familiarity with git is assumed.

Getting the code

First, you will need to create a GitHub account. If you have one already, log in.

Log into GitHub

Now, navigate to the website repository. In the top-right hand corner of the page, you will see a ‘Fork’ button. Click this to create a copy of the repository in your own account.

Fork the repository

After a few seconds, you will have your own fork of the repository.

Forked repository

Now, if you’re familiar with git, clone the repository and make your changes. I’ll show you how to make a post without leaving the GitHub website - but the principles are the same whether you clone the repository or use the web interface.

Writing a post

Let’s try writing a new blog post. First, click on the _posts folder to see the existing blog posts on the site. If you click on one, you can read and modify it.

Editing a post

Now let’s add a new post - click the ‘Create new file’ button:

Creating a file

First, name the file. The format of the name should be yyyy-mm-dd-title-of-the-post.markdown, for example 2016-05-21-pining-for-the-fjords.markdown. With the file named, we first need to provide some metadata about the post. The very first thing in the file should be something like this:

---
layout: post
title:  "Pining for the fjords"
date:   2016-05-21 21:51:00 +0100
categories:
author: "John Cleese"
---

On the next line, you can start writing your post. If you hadn’t already guessed, posts are written in a very simple formatting language called Markdown. If you haven’t used it before, here’s a tutorial.

Writing a new post

If you want to preview your post, you can click the ‘Preview’ tab at the top of the editor. Once you’re happy with the content, it’s time to commit it to your local version of the site (don’t worry, it won’t be published to the live site yet!).

Scroll down, and write a short description for your commit. Choose to commit directly to master. Now click ‘Commit new file’.

Starting a pull request

You will be taken back to the _posts folder. Notice that there is now a message: ‘This branch is 1 commit ahead of cssbristol:master’. That means we can make a pull request to have our changes merged back into the main site. Click the ‘Pull request’ button to begin the process.

Starting a pull request

GitHub should choose to do the right thing automatically. At the top, you can see a ‘base’ and ‘head’. The ‘base’ is the repository that you want to merge into (the main CSS website repository), and the ‘head’ is your local copy. Below, in green, you can see you additions.

Submitting a pull request

Click ‘Create pull request’, and write a short description of your changes in the box provided. The clearer the information you provide, the easier it will be for us to merge your changes.

Describing a pull request

And now you’re done! We’ll see to it as soon as possible.

A submitted pull request

University IT

Ross Gardiner on May 21, 2016

IT Services hosts a huge range of systems, many of which are well kept secrets! We’ll have a look at some of them in this article.

Snowy

Every CS student has an account on the departmental server, snowy. Read the snowy tutorial.

IT Services maintain a page with other remote access information, including the addresses of lab machines that you can remote into.

Frosty

If you’re doing Design Verification (COMS31700), you’ll probably need to use Frosty for remote access to Cadence tools. This machine supports NoMachine, which makes remoting a lot more pleasant.

VPN

The university VPN allows you to access university resources (e.g. journals, firewalled internal systems) from anywhere.

Setup instructions

Remote Desktop

You can also remote into a standard university desktop from anywhere. Some commercial software that you cannot install on personal machines is available on remote desktop. IT Services record the status of each piece of software in Topdesk.

Connection instructions

Network Filestore - MyFiles

Every student has a ‘MyFiles’ filestore, which you can mount on your own device while connected through the university network (i.e. eduroam, VPN, Remote Desktop).

Setup instructions

Mediasite

Rewatching (ahem) lectures for revision? Wouldn’t it be nice if you could speed them up? Well, you can - just not with Flash enabled. Append &usehtml5=true to the URL of a Mediasite video and voila: you can speed up playback.

If you’re tired of having to do this manually, then use the HTML5 mediasite userscript to automatically append it (using an extension like TamperMonkey)

Free Software

While studying, you have access to various university-licensed software. This includes:

Complete list

Laptop Clinic

If you need help with a computer, phone or software, you can get help from the Student Laptop and Mobile Clinic, which runs in Senate House at 2pm to 5pm, Monday to Friday.

The clinic is staffed largely by current computer science students. Towards the end of the academic year, IT Services advertises for new students to staff the clinic - so keep an eye out!

Wikis

The university hosts an Atlassian Confluence install, which you can use to create your own wikis.

Online Tools

Password tools - check that your password is working correctly. Sometimes UoB and Google passwords become desynced - you can fix it here.

Find a study space

Ross Gardiner on January 12, 2017

The university has a couple of pages for finding study spaces. Here’s the main one.

New: find a free desk

Library staff count the number of people in each library hourly. In the exam periods this year, the university is trialling ‘Find a study desk’, which is based on this data.

Hot tip: there’s a JSON API! Here’s the building information, and here are the study space stats.

There’s also a page for finding a free computer.

Where else can I work?

Some places that aren’t officially documented. Found an obscure place to work? Open an issue and we’ll add it to this page

Study areas

If you’ve done or are doing Software Product Engineering, you should have access to the Student Project Lab (‘backlab’), MVB 2.09 (just beyond 2.11). If you need a second monitor, this is the place to go. Third years and above should also have access to the Device Lab, MVB 2.10 (with the curved glass wall).

There is also a ‘quiet pod’ for one or two people on level 2 of the MVB atrium.

In Queen’s, there are a few non-library areas with seating:

Workshop spaces

MVB Hack Space

The Hack Space in MVB can be used by all students who have done a short induction. Sign up for a Hack Space induction. The equipment available includes a laser cutter, 3D printer, soldering irons and a 3D carving machine. A component vending machine is being installed outside the Hack Space.

Student Workshop

The student workshop has equipment including a laser cutter; 3D printers; a lathe and one of these. You’ll need to be trained on some of the equipment first but if you’re planning to take part in BEEES Robot Wars you’re probably going to spend a lot of time in here.

BIG Lab

The Bristol Interaction and Graphics group has a lab on level 2 of the MVB Atrium. If you’re taking Interactive Devices, you can use this lab to work on your device.

Humanities - Woodland Road

There are a few common rooms accessible to everyone between 3 and 19 Woodland Road. These houses are joined by a corridor at the back, so you can enter at one end and walk along the whole thing.

Install Nvidia CUDA Drivers on Ubuntu

John Griffith on February 20, 2017

Notes: This guide was written using an Ubuntu 16.04 image, therefore it may not work on other systems

  1. Connect to your AWS Instance, for instructions follow this guide
  2. Next we install the dependencies for the CUDA installation
    1. First update the package manage
    2. sudo apt-get update
    3. Install build essentials
    4. sudo apt-get install build-essential
    5. The newer versions of gcc and g++ can cause issues with popular machine learning libraries, so we’ll install older versions:
    6. sudo apt-get install gcc-4.9
    7. sudo apt-get install g++-4.8
    8. Remove the original version and replace them with symbolic links to the older versions:
    9. sudo rm -f /usr/bin/gcc
    10. sudo rm -f /usr/bin/g++
    11. sudo ln -s /usr/bin/g++-4.8 /usr/bin/g++
    12. sudo ln -s /usr/bin/gcc-4.9 /usr/bin/gcc
    13. Install the missing Linux header files:
    14. sudo apt-get install linux-headers-generic
    15. Next, reboot the server: sudo reboot
  3. Download the CUDA toolkit
    1. In your webrowser navigate to the CUDA download section
    2. Navigate through to your target platform
    3. For our target platform of Ubuntu 16.04, navigate: Linux > x86_64 > Ubuntu > 16.04 > run file (local)
    4. Right click on the download link and select copy link
    5. Back in the terrainal window type wget <your_link> where <your_link> is the link you copied in the previous step
    6. After the file has downloaded, change the permissions using chmod 751 <file_name> so we can run it
  4. Next run the installation ./<file_name>
    1. After reading the EULA press q to begin the installation
    2. Follow the installation steps
  5. Next install NVIDIA Toolkit sudo apt-get install nvidia-cuda-toolkit
  6. Reboot the server sudo reboot
  7. If you have installed the Nvidia sample pack, build and the deviceQuery program located in <sample_dir>/1_Utilities/deviceQuery

First Steps with Git

David Bernhard on October 18, 2017

Git is a way to store, back up and organise your code. It is one of those things that you wish you’d been taught in your first term if you discover it later on.

This tutorial assumes that you are moderately familiar with using a terminal/shell.

Installing

The lab machines should have git installed already. On your own machine, type “git” in a terminal and if you’re lucky you’ll get the help text, showing it’s installed. If you need to install it yourself:

Create a repository

When you start on a new project or unit, create an empty folder, go to it in your terminal and type git init. This creates a hidden subfolder .git which is your repository - it will store the entire history of your project and every change you make, as long as you remember to commit these changes (which we’ll do soon). This not only gets you unlimited undo/redo, so you can try things out and undo them if they don’t work, but you can also maintain several different versions of your code and switch between them.

The folder containing the .git folder is your working tree - one particular version of your project.

Making commits

Let’s say you’ve written some code in a file hello.c:

#include <stdio.h>

int main() {
  puts("Hello World!");
  return 0;
}

Type git status in your terminal. This will show that there’s an untracked change in file hello.c. If colours are set up correctly, untracked changes are red and tracked ones are green.

To tell git to track this change, run git add hello.c. Run git status again and you’ll see “new file: hello.c”, possibly in green. This means you’ve told git about this change, but it’s not committed yet (not saved in the repository). Run git commit -m 'New file'. Each commit must have a message and the -m <message> option is a way to provide this. The single quotes around the message are to do with how your shell parses arguments - basically if you put single quotes around a commit message then you can use spaces, punctuation etc. and it should work as long as there’s no single quotes in your message itself. Do git status again and you should see “nothing to commit, working tree clean” which is a good message to see.

The simplest way to work with git is this: make changes, commit changes, repeat. Git lets you undo/redo changes at the commit level so it’s helpful to make lots of small commits instead of few large ones; commit messages help you find commits when you want to e.g. revert to a particular commit.

If you make a change to your source file - perhaps add a comma after Hello - you’ll see “modified: hello.c” under a heading “Changes not staged for commit” (in red if colours are working). git add hello.c tracks the change (it now appears under “Changes to be committed” in the status text), git commit -m 'Added a comma' commits it:

$ git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   hello.c

no changes added to commit (use "git add" and/or "git commit -a")

$ git add hello.c

$ git status

On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        modified:   hello.c

$ git commit -m 'Added a comma.'
[master 6e7e782] Added a comma.
1 file changed, 1 insertion(+), 1 deletion(-)

$ git status
On branch master
nothing to commit, working tree clean

In summary:

If “git status” shows “working tree clean” then all your changes have been saved.

Reverting changes

As you might have gathered already from the help text, if you’ve made changes to a file that you’ve not committed yet and you want to undo them back to the last commit, git checkout -- FILENAME reverts the file. There is a space both before and after the -- in that command.

If you’ve committed changes and want to go back to an earlier commit, the first thing you do is git log which shows you a list of all commits. If there are a lot, git log | less lets you view them one page at a time - advance with SPACE, exit with Q. git log --oneline shows just one line per commit and you can stick | less on the end of that too. Each commit has a hash value which is a long hexadecimal string. When referencing a commit, you usually only need to type the first few characters of the hash.

Let’s say I decide I don’t want a comma after “Hello” after all.

$ git log --oneline
6e7e782 Added a comma.
4f251ef New file

$ git checkout 4f25 hello.c`

The command git checkout HASH FILENAME reverts the file to the state in the given commit. The file is now in “changed and not tracked” state as if you’d edited it by hand. You can also git checkout HASH to revert all files back to an older commit.

Excluding files

Let’s compile our file: gcc hello.c -Wall -Werror -std=c99 -o hello. You do use -Wall -Werror on all your C programs to catch possible bugs, right?

We now have two files in the working tree: hello.c and hello (or hello.exe if we’re on windows). However we don’t want to track hello in the repository - you want your sources but not your compiled files in the repository. You can just compile the sources again whenever you need to, after all.

Create a file called .gitignore (yes, it starts with a dot) with one line hello (or hello.exe) in it. Each line in this file is treated as a pattern that matches one or more files (you can do *.exe to exclude all windows executables for example) that git should ignore. Adding the name of a folder to a gitignore file ignores that folder and everything in it. When you do git status again you’ll see that git ignores your executable file, but notices the new file .gitignore. Add that file to the repository and commit.

Remote repositories

So far we’ve just saved our project’s history in a subfolder on our own machine. The next step is to save it in the cloud so that several people can work on it together, or you can work on it from different machines (for example, home and lab PC).

There are many free providers of git services, including: GitLab, GitHub and Bitbucket. All three provide provide unlimited (within reason) free private repositories, allowing you to control who can read/write to them, in addition to unlimited free public repositories for open-source projects. GitHub’s Student Pack can be obtained, using your university email, to give you access to private repositories in addition to other interesting offers.

WARNING: do not under any circumstances place code that you have submitted or intend to submit as assessed coursework to a public repository on any provider. If another student finds this code and submits a copy of it, you are both in trouble for plagiarism. Yes, this has happened in the past.

The next few steps in this tutorial are based on the gitlab UI to create a remote repository, but github and bitbucket offer the same features.

You get to see your project page. The git address of your project is https://gitlab.com/USERNAME/PROJECTNAME.git, so for example my username is david-bristol and I have a project called coconut that lives at https://gitlab.com/david-bristol/coconut.git. If you access this link on the web, it’ll remove the “.git” part at the end but you need it in your terminal.

The command git clone ADDRESS downloads a repository and sets up some information so that you can synchronise the local and remote copies. For example, git clone https://gitlab.com/david-bristol/coconut.git will create a subfolder coconut with the contents of my coconut project. Since this is a public project, the files will just appear - for a private project, git will ask for your username and password first.

You can now use the following two commands to synchronise the local and remote copies:

It is advised to use these commands only when your working tree is clean.

The rough outline of setting up a git project to work together:

The commands git add and git commit add changes from your working tree to the local copy of the repository. The commands git push and git pull move committed changes from the local copy of the repository to the remote copy of the repository and back again. After pushing/pulling, all copies of the repository should be identical.

Your daily coding routine:

You can save yourself a lot of trouble by only doing push/pull operations with a clean working tree, e.g. you’ve added and committed all changes to the local copy of your repository. In theory, that’s all you need. In practice, what happens if two people edit the same file?

You do not have to agree via e-mail or some other system who is currently editing which files. Git is set up to handle this situation!

What happens is that your git push will fail if someone else has pushed since your last pull. In this case, do a git pull. If you’ve both been editing different files, git will update the ones you haven’t changed with the new versions from the remote copy. Check that everything still works/compiles and then try and push again.

If you’ve both been editing the same files, git will see if there’s an obvious way to combine your changes (for example you’ve both edited different functions in the same file). If so, all is well. If not, you have a conflict. The line CONFLICT ... when you try and pull will alert you to this and tell you which files are affected (so will git status). When you open these files, you’ll see markers as follows in places where there’s a conflict (search for <<< in your text editor to find them):

<<<<<<< HEAD
puts("Hello, world!");
=======
puts("Hi world!");
>>>>>>> 31be48e8e2a86fb71b0b0b61bb08019314d87a1b

This means that you tried to change a line to read puts("Hello, world!"); but someone else changed the same line to puts("Hi world!"); since your last pull. Fix the conflict by choosing what to do with the line(s) in question and then delete the <<< === >>> marker lines and check that everything compiles and works again. Then you can add and commit the file and try to push again.

Conflicts, although annoying, are nothing to be afraid of. They can happen as part of normal git working practice in a team and you deal with them as they happen.

Our lovely sponsors

Cookpad logo Dyson logo Just Eat logo Bloomberg logo Jump Trading logo

This could be you!

Get in touch