Archive for August, 2018
Over the years I’ve been blogging, I have run a few recurring series of blogposts. In the early days, there was the shortlived Problem of The Week. Sometime later I inherited The Carnival of Maths which I looked after for a couple of years before passing it over to Aperiodical.com who have looked after it ever since. I also ran a series called A Month of Math Software for 2 and a half years before my enthusiasm for the topic ran out.
I am currently the Head of Research Computing at The University of Leeds — a senior management role that puts me in the fortunate position of being reasonably well-informed about the world of research computing. Software, hardware, cloud, data science, dealing with sensitive data…everyone, it seems, has something to tell me. I’m also continuing with my EPSRC fellowship part-time which means that I’m rather more hands on than a typical member of an executive leadership team.
While at JuliaCon 2018, I had the extremely flattering experience of a few people telling me that they had been long time readers of WalkingRandomly and that they were disappointed that I didn’t post as often
All of this has led to the desire to start a new regular series. One where I look at all aspects of research computing and compile it into a series of monthly posts. If you have anything you’d like including in next months’ post — contact me via the usual channels.
Botched code causes seven-year scientific argument
For the last couple of years, I have given a talk around the UK and Europe called ‘Is your Research Software Correct‘ (unlike this other talk of mine, it has not yet been recorded but I’ll soon remedy that! Let me know if you can offer a venue with good recording facilities).
I start off by asking the audience to Imagine….imagine that the results of your latest simulation or data analysis are in and they are amazing. Your heart beats faster, this result changes everything and you know it. This is why you entered science, this is what you always hoped for. Papers in journals like Nature or Science — no problem. Huge grant to follow up this work…professorship….maybe, you dare to dream, this could lead to a nobel prize. Only one minor problem — the code is completely wrong and you just haven’t figured it out yet.
In the talk (based originally on an old blog post here) I go on to suggest and discuss some simple practices that might help the situation. Scripting and coding instead of pointy-clicky solutions, version control, testing, open source your software, software citation etc. None of it is mind blowing but I firmly believe that if all of the advice was taken, we’d have fewer situations like this one…..
Long story short, Two groups were investigating what happens when you super-freeze water. They disagreed and much shouting happened for 7 years. There was a bug in the code of one group. A great article discussing the saga is available over at Physics Today:
https://physicstoday.scitation.org/do/10.1063/PT.6.1.20180822a/full/
Standout quotes that may well end up in a future version of Is Your Research Software Correct include
“One of the real travesties,” he says, is that “there’s no way you could have reproduced [the Berkeley team’s] algorithm—the way they had implemented their code—from reading their paper.” Presumably, he adds, “if this had been disclosed, this saga might not have gone on for seven years”
and
Limmer maintains that he and his mentor weren’t trying to hide anything. “I had and was very willing to share the code,” he says. What he didn’t have, he says, was the time or personnel to prepare the code in a form that could be useful to an outsider. “When Debenedetti’s group was making their code available,” Limmer explains, “he brought people in to clean up the code and document and run tests on it. He had resources available to him to be able to do that.” At Berkeley, “it was just me trying to get that done myself.”
Which is a case study for asking for Research Software Engineer support in a grant if ever I saw one.
Julia gets all grown up — version 1.0 released at JuliaCon 2018
One of the highlights of the JuliaCon 2018 conference was the release of Julia version 1.0 — a milestone that signifies that the new-language-on-the block has reached a certain level of maturity. We celebrated the release at University of Leeds by installing it on our most recent HPC system – ARC3.
In case you don’t know, Julia is a relatively new free and open source language for technical computing. It works on everything from Raspberry Pi up to HPC systems with thousands of Cores. It’s the reason for the letters Ju in Project Jupyter and aims to be an easy to use language (along the lines of Python, R or MATLAB) with the performance of languages like Fortran or C.
UK Research Software Engineering Association Webinar series
The UK Research Software Engineering Association is starting a new webinar series this month with a series of planned topics including an Introduction to Object-Oriented Design for Scientists, Interfacing to/from Python with C, FORTRAN or C++ and Meltdown for Dummies.
These webinars are free to join, and you do not need to register in advance. Full details including the link to join the webinar are available below.
For more information on the RSE webinar series, including information on how to propose a webinar and information on upcoming webinars, please see:
https://rse.ac.uk/events/rse-webinar-series
This page will also have links to recordings of past webinars when they become available.
Verification and Modernisation of Fortran Codes using the NAG Fortran Compiler
There is still a huge amount of research software written in Fortran. Indeed, software written in Fortran are, by far, the most popular codes run on the UK’s national supercomputer service, Archer (See http://www.archer.ac.uk/status/codes/ for up to date stats).
Fortran compilers are not created equally and many professional Fortran developers will suggest that you develop against more than one. Gfortran is probably essential if you want your code to be usable by all but the Intel Fortran Compiler can often produce the fastest executables on x86 hardware and the NAG Compiler is useful for ensuring correctness.
This webinar by NAG’s Wadud Miah promises to show what the NAG Fortran Compiler can do for your Fortran code.
New Macbook Pro has 6 CPU cores but….
Apple’s new Macbook Pro laptops have a fantastic looking CPUs in them with the top of the line boasting 6 cores and turbo boost up to 4.8Ghz. Sounds amazing for simulation work but it seems that there are some thermal issues that prevent it from running at top speed for long.
Contact me to get your news included next month
That’s all I have for this first article in the series. If you have any research computing news that you’d like included in the next edition, contact me.
Audiences can be brutal
I still have nightmares about the first talk I ever gave as a PhD student. I was not a strong presenter, my grasp of the subject matter was still very tenuous and I was as nervous as hell. Six months or so into my studentship, I was to give a survey of the field I was studying to a bunch of very experienced researchers. I spent weeks preparing…practicing…honing my slides…hoping that it would all be good enough.
The audience was not kind to me! Even though it was only a small group of around 12 people, they were brutal! I felt like they leaped upon every mistake I made, relished in pointing out every misunderstanding I had and all-round gave me a very hard time. I had nothing like the robustness I have now and very nearly quit my PhD the very next day. I can only thank my office mates and enough beer to kill a pony for collectively talking me out of quitting.
I remember stopping three quarters of the way through saying ‘That’s all I want to say on the subject’ only for one of the senior members of the audience to point out that ‘You have not talked about all the topics you promised’. He made me go back to the slide that said something like ‘Things I will talk about’ or ‘Agenda’ or whatever else I called the stupid thing and say ‘Look….you’ve not mentioned points X,Y and Z’ [1].
Everyone agreed and so my torture continued for another 15 minutes or so.
Practice makes you tougher
Since that horrible day, I have given hundreds of talks to audiences that range in size from 5 up to 300+ and this amount of practice has very much changed how I view these events. I always enjoy them…always! Even when they go badly!
In the worst case scenario, the most that can happen is that I get given a mildly bad time for an hour or so of my life but I know I’ll get over it. I’ve gotten over it before. No big deal! Academic presentations on topics such as research computing rarely lead to life threatening outcomes.
But what if it was recorded?!
Anyone who has worked with me for an appreciable amount of time will know of my pathological fear of having one of my talks recorded. Point a camera at me and the confident, experienced speaker vanishes and is replaced by someone much closer to the terrified PhD student of my youth.
I struggle to put into words what I’m so afraid of but I wonder if it ultimately comes down to the fact that if that PhD talk had been recorded and put online, I would never have been able to get away from it. My humiliation would be there for all to see…forever.
JuliaCon 2018 and Rise of the Research Software Engineer
When the organizers of JuliaCon 2018 invited me to be a keynote speaker on the topic of Research Software Engineering, my answer was an enthusiastic ‘Yes’. As soon as I learned that they would be live streaming and recording all talks, however, my enthusiasm was greatly dampened.
‘Would you mind if my talk wasn’t live streamed and recorded’ I asked them. ‘Sure, no problem’ was the answer….
Problem averted. No need to face my fears this week!
A fellow delegate of the conference pointed out to me that my talk would be the only one that wouldn’t be on the live stream. That would look weird and not in a good way.
‘Can I just be live streamed but not recorded’ I asked the organisers. ‘Sure, no problem’ [2] was the reply….
Later on the technician told me that I could have it recorded but it would be instantly hidden from the world until I had watched it and agreed it wasn’t too terrible. Maybe this would be a nice first step in my record-a-talk-a-phobia therapy he suggested.
So…on I went and it turned out not to be as terrible as I had imagined it might be. So we published it. I learned that I say ‘err’ and ‘um’ a lot [3] which I find a little embarrassing but perhaps now that I know I have that problem, it’s something I can work on.
Rise of the Research Software Engineer
Anyway, here’s the video of the talk. It’s about some of the history of The Research Software Engineering movement and how I worked with some awesome people at The University of Sheffield to create a RSE group. If you are the computer-person in your research group who likes software more than papers, you may be one of us. Come join the tribe!
Slide deck at mikecroucher.github.io/juliacon2018/
Feel free to talk to me on twitter about it: @walkingrandomly
Thanks to the infinitely patient and wonderful organisers of JuliaCon 2018 for the opportunity to beat one of my long standing fears.
Footnotes
[1] Pro-Tip: Never do one of these ‘Agenda’ slides…give yourself leeway to alter the course of your presentation midway through depending on how well it is going.
[2] So patient! Such a lovely team!
[3] Like A LOT! My mum watched the video and said ‘No idea what you were talking about but OMG can you cut out the ummms and ahhs’
Technological development in software is more like a cliff-face than a ladder – there are many routes to the top, to a solution. Further, the cliff face is dynamic – constantly and quickly changing as new technologies emerge and decline. Determining which technologies to deploy and how best to deploy them is in itself a specialist domain, with many features of traditional research.
Researchers need empowerment and training to give them confidence with the available equipment and the challenges they face. This role, akin to that of an Alpine guide, involves support, guidance, and load carrying. When optimally performed it results in a researcher who knows what challenges they can attack alone, and where they need appropriate support. Guides can help decide whether to exploit well-trodden paths or explore new possibilities as they navigate through this dynamic environment.
These guides are highly trained, technology-centric, research-aware individuals who have a curiosity driven nature dedicated to supporting researchers by forging a research software support career. Such Research Software Engineers (RSEs) guide researchers through the technological landscape and form a human interface between scientist and computer. A well-functioning RSE group will not just add to an organisation’s effectiveness, it will have a multiplicative effect since it will make every individual researcher more effective. It has the potential to improve the quality of research done across all University departments and faculties.
Further Reading