What’s happened to big-box HPC in UK Universities?

October 27th, 2010 | Categories: walking randomly | Tags:

I was having a quick browse through the current Top-500 supercomputer list and it appears to me that UK Universities hardly feature at all.  We’ve got Hector at Edinburgh University in at number 26 but that’s the UK’s National facility so we’d expect it to be good.

Other than that, we have The University of Southampton at Number 83, The University of Bristol at Number 378 and that seems to be about it (please correct me if I’m wrong).  There are a few more UK-based installations in the top-500 but none of them are in Universities as far as I can tell.

I’m a little surprised by this since I thought that we (UK academia) would give a better account of ourselves.  Does it matter that we don’t have more boxes in the top-500 within UK Academia?  Is everyone doing research on smaller scale Beowulf clusters, condor pools, GP-GPUs and fully tricked-out 12 core desktops these days?  Is the need for big boxes receding?  Any insight anyone?

  1. October 27th, 2010 at 14:11
    Reply | Quote | #1

    I work in a BBSRC centre, and the BBSRC offer time on Hector. I have considered applying for time on there, but I am put off by a few things. There is a full-on application procedure with review panels to get access. It doesn’t appear to be easy to get any specific libraries, packages or software that I might need for a particular application installed on Hector. We already have access to 2 fairly large clusters, and I do have a nicely tricked out desktop. These suit my workflows better since I can just try things out on them much more easily, before committing to firing off a huge computational job. I think big-box HPC is too inflexible to suit computational biology, or at least simulation modelling, high-throughput image analysis and inference which are the things I am interested in.

  2. October 27th, 2010 at 15:12
    Reply | Quote | #2

    Thanks for that Conor. When you say ‘huge computational job’ do you tend to mean one big problem that needs 512 cores and 256Gb RAM (say) or do you tend to mean lots of smaller jobs? In other words, would a Condor HTC facility be more use to you than Hector say?

  3. Mike Dewar
    October 27th, 2010 at 16:48
    Reply | Quote | #3

    Hi Conor, I’m responsible for the Computational Science and Engineering support team for HECToR. Part of our job is to install, port and tune software for the machine so as long as there aren’t any licensing issues we can handle that side of things for you. In addition, there are a number of lightweight mechanisms to allow users to try out HECToR before going through the full review process. If you’re interested then I suggest that you contact BBSRC to see which are most appropriate to you (contact details are at http://www.bbsrc.ac.uk/funding/facilities/facilities.aspx). Alternatively please contact the CSE team on hector-cse@nag.co.uk and we’ll do our best to help you out.

  4. October 27th, 2010 at 17:07
    Reply | Quote | #4

    I find that I don’t tend to have one big problem that needs many cores, and that Condor facilities suit me. But I also feel that this might be largely because I only work on problems that I think I can resolve with the resources that are available to me! For example, I sometimes think about modelling interacting populations of (millions of) cells. I am familiar with the difficulty of writing code to carry out individual simulations in such a way that the code is shareable and efficient, using standards etc. I find that difficult enough. The extra inertia involved in applying for and winning time on Hector for instance, and adapting (maybe completely rewriting) code to simulate millions of interactions it makes it seem like a risky proposition to me. Maybe I’m just chicken!

  5. October 27th, 2010 at 17:09
    Reply | Quote | #5

    Thanks Mike, I should do that really instead of complaining without giving it a good try! It just seems a bit intimidating at the moment.