Walking Randomly

Almost every computer you buy these days contains more than one processor core – even my laptop has 2 – and I have access to relatively inexpensive desktops that have as many as 8.  So, it is hardly surprising that I am steadily receiving more and more queries from people interested in making their calculations run over as many simultaneous cores as possible.

The big three Ms of the mathematical software world, Mathematica, Maple and MATLAB, all allow you to explicitly parallelise your own programs (although you have to pay extra if you want to do this in MATLAB without resorting to hand crafted C-code) but parallel programming isn’t easy and so tutorials are invaluable.

Fortunately, Darin Ohashi, a senior kernel developer at Maplesoft has taken the time to write some parallel Maple tutorials for us and he has presented his work as a series of blog posts over at MaplePrime.  Here’s an idex:

• The Parallel Programming Blog
• Why Go Parallel?
• What Makes Parallel Programming Hard?
• Thread Safety
• Load Balancing and Thread Management
• The Task Programming Model
• Parallel Programming Blog Feedback

If you are interested in parallel programming in Maple then Darin’s posts are a great place to start.

Other articles from Walking Randomly you may be interested in

• Parallel MATLAB with openmp mex files
• A parallel programming tip for Mathematica
• Parallelisation: Is MATLAB doing it wrong?

Will Robertson and I have released a new version of our colorbarplot package for Mathematica 6 and above.  Colorbarplot has only one aim and that is to allow Mathematica users to add a MATLAB-like colorbar to their graphs.  Something like this for example:

Version 0.5 is a very small upgrade from 0.4 in that it allows you to change the font style of the colorbar ticks as well as it’s label – as requested by Murat Havzalı.  Here are a couple of examples of the new functionality:

ColorbarPlot[Sin[#1 #2] &, {0, 2}, {0, 2},
 PlotType -> "Contour",
 CTicksStyle -> Bold]

ColorbarPlot[Sin[#1 #2] &, {0, 2}, {0, 2},
 PlotType -> "3D",
 CLabel -> "Hello",
 ColorFunction -> "TemperatureMap",
 CLabelStyle -> Directive[Red, Italic, Bold, FontSize -> 20],
 CTicksStyle -> Directive[Green, FontFamily -> "Helvetica"]
 ]

Colorbarplot comes with an extensive examples file and you can download version 0.5 either from me or from Will’s github repository. We hope you find it useful and comments are welcomed.

• UPDATE 12th Novermber 2010. Update to version 0.6

Slowly but surely more and more MATLAB functions are becoming able to take advantage of multi-core processors.  For example, in MATLAB 2009b, functions such as sort, bsxfun, filter and erf (among others) gained the ability to spread the calculational load across several processor cores.  This is good news because if your code uses these functions, and if you have a multi-core processor, then you will get faster execution times without having to modify your program.   This kind of parallelism is called implicit parallelism because it doesn’t require any special commands in order to take advantage of multiple cores – MATLAB just does it automagically.  Faster code for free!

For many people though, this isn’t enough and they find themselves needing to use explicit parallelism where it becomes the programmer’s responsibility to tell MATLAB exactly how to spread the work between the multiple processing cores available.  The standard way of doing this in MATLAB is to use the Parallel Computing Toolbox but, unlike packages such as Mathematica and Maple, this functionality is going to cost you extra.

There is another way though…

I’ve recently been working with David Szotten, a postgraduate student at Manchester University, on writing parallel mex files for MATLAB using openmp and C.  Granted, it’s not as easy as using the Parallel Computing Toolbox but it does work and the results can be blisteringly fast since we are working in C.  It’s also not quite as difficult as we originally thought it might be.

There is one golden rule you need to observe:

Never, ever call any of the mex api functions inside the portion of your code that you are trying to parallelise using openmp.  Don’t try to interact with MATLAB at all during the parallel portion of your code.

The reason for the golden rule is because, at the time of writing at least (MATLAB 2009b), all mex api functions are not thread-safe and that includes the printf function since printf is defined to be mexPrintf in the mex.h header file.  Stick to the golden rule and you’ll be fine.  Move away from the golden rule,however, and you’ll find dragons.  Trust me on this!

Let’s start with an example and find the sum of foo(x) where foo is a moderately complicated function and x is a large number of values.  We have implemented such an example in the mex function mex_sum_openmp.

I assume you are running MATLAB 2009b on 32bit Ubuntu Linux version 9.04 – just like me.  If your setup differs from this then you may need to modify the following instructions accordingly.

• Before you start.  Close all currently running instances of MATLAB.
• Download and unzip the file  mex_sum_openmp.zip to give you mex_sum_openmp.c
• Open up a bash terminal and enter export OMP_NUM_THREADS=2 (replace 2 with however many cores your machine has)
• start MATLAB from this terminal by entering matlab.
• Inside MATLAB, navigate to the directory containing mex_sum_openmp.c
• Inside MATLAB, enter the following to compile the .c file

  mex mex_sum_openmp.c CFLAGS="\$CFLAGS -fopenmp" LDFLAGS="\$LDFLAGS -fopenmp"

• Inside MATLAB, test the code by entering

  x=rand(1,9999999); %create an array of random x values
  mex_sum_openmp(x) %calculate the sum of foo(x(i))

• If all goes well, this calculation will be done in parallel and you will be rewarded with a single number.  Time the calculation as follows.

  tic;mex_sum_openmp(x);toc

• My dual core laptop did this in about 0.6 seconds on average.  To see how much faster this is compared to single core mode just repeat all of the steps above but start off with export OMP_NUM_THREADS=1 (You won’t need to recompile the mex function).  On doing this, my laptop did it in 1.17 seconds and so the 2 core mode is almost exactly 2 times faster.

Let’s take a look at the code.

#include "mex.h"
#include <math.h>
#include <omp.h>

double spawn_threads(double x[],int cols)
{
	double sum=0;
	int count;
        #pragma omp parallel for shared(x, cols) private(count) reduction(+: sum)
	for(count = 0;count<cols;count++)
	{
	     sum += sin(x[count]*x[count]*x[count])/cos(x[count]+1.0);;
	}

	return sum;
}

void mexFunction(int nlhs,mxArray* plhs[], int nrhs, const mxArray* prhs[])
{
	double sum;
	 /* Check for proper number of arguments. */
 	if(nrhs!=1) {
    	mexErrMsgTxt("One input required.");
  	} else if(nlhs>1) {
    	mexErrMsgTxt("Too many output arguments.");
  	}

	/*Find number of rows and columns in input data*/
	int rows = mxGetM(prhs[0]);
	int cols = mxGetN(prhs[0]);		

	/*I'm only going to consider row matrices in this code so ensure that rows isn't more than 1*/
	if(rows>1){
	mexErrMsgTxt("Input data has too many Rows.  Just the one please");
	}

	/*Get pointer to input data*/
	double* x = mxGetPr(prhs[0]);

	/*Do the calculation*/
	sum = spawn_threads(x,cols);

	/*create space for output*/
 	 plhs[0] = mxCreateDoubleMatrix(1,1, mxREAL);

	/*Get pointer to output array*/
	double* y = mxGetPr(plhs[0]);

	/*Return sum to output array*/
	y[0]=sum;

}

That’s pretty much it – the golden rule in action. I hope you find this example useful and thanks again to David Szotten who helped me figure all of this out.  Comments welcomed.

Other articles you may find useful

•  MATLAB mex functions using the NAG C Library in Windows – This article includes an OpenMP example compiled on Visual Studio 2008 in Windows.

Back in August 2008, Joel Spolsky and Jeff Atwood launched a question and answer site aimed squarely at programmers and it has been a massive hit.  Called Stack Overflow, the site currently contains over 330,000 questions and answers covering pretty much every aspect of programming you care to mention from What IDE to use for Python through to smart pointers in C++ and everything in between.  With a userbase running into the thousands, you can be sure that someone, somewhere will know the answer to your programming question no matter how obscure.  All this and yet, in true internet tradition, it won’t cost you a penny.

Well, time moves on and Stack Overflow spawned other sites based on the same web technology.  Server Fault, for example, is aimed at system administrators and finally, to complete the trilogy, we have Super User which is geared towards general computer enthusiasts.  Although neither of these have been as popular as the original programming site (Server Fault currently has 17,000 questions and Super User has just over 12,000) they are still very useful resources.  More importantly, they demonstrate that you can use the Stack Overflow technology to build any community question and answer site.

I guess it was inevitable that someone would eventually build such a site for mathematics.  Math Overflow is currently in the beta stage of development and only contains just under 400 questions at the time of writing but it has potential.  It just needs a bigger audience, so head over there and get questioning.

I guess you could think of it as a people-powered version of Wolfram Alpha!

I’m usually a big fan of Wolfram Research and their products but on reading the news that they had released a $50 iPhone app that hooked in to the Wolfram Alpha website I wondered if they had taken leave of their senses.  Let’s see why…

Wolfram Alpha is a web service that is based on the extremely powerful computer algebra system, Mathematica, and it is very VERY cool. I won’t go into detail about what it can do here since many,many people have done that before (including me) but for the uninitiated think of it as a super-powerful graphical calculator that (partially) understands plain English and also has access to massive datasets from areas as diverse as the stock market, chemistry, popular music, language and physics.

Many students I know focus on the fact that it can eat their math homework for breakfast – not only giving the answer but also giving all of the steps that you’d need to get full marks on your coursework. Mathematics educators in the know are scrambling to come up with ways to harness this new freely available computational power while those not in the know are wondering why their students all get straight As in their coursework and yet flunk their exams.

All in all it’s very good stuff and you may be forgiven for thinking that it’s worth $50 for an app that can do all that. I would agree with you except for one, tiny little detail.

You can get all of this stuff for free! From Wolfram themselves!

Don’t believe me? OK, let’s prove it. Mashable have just reviewed the shiny new Wolfram iPhone app and have given it a big thumbs up because the reviewer has focused on all of the awesome features that Wolfram Alpha has. Choose any one of those nice examples shown in the mashable review – any one you like. Now, fire up your favourite web browser, head over to the Wolfram Alpha website and type in the exact same query.

Tell you what. I’ll do it for you. One of the examples given for the $50 app solves a simple differential equation – click on this link (from your iPhone if it suits you) to see exactly the same results for free.

I did exactly this on my Android phone  and have included a screenshot for your delight and delectation (well, technically speaking I did it on the Android emulator since getting screenshots from an actual Android phone is tricky).  The Wolfram Alpha query below cost me nothing:

I would have had to pay $50 to get this next one using the new iPhone app.

So, let’s be very clear here, Wolfram aren’t selling you an app that puts the power of Wolfram Alpha in your iPhone – they are just selling you a browser alternative that can only hit one website! For $50.

Of course they don’t JUST give you a browser alternative – according to the comments section over at mashable, the new alternative to the ‘I am Rich‘ app also gives you a more math-friendly onscreen keyboard and a query-history as well as formatting the results a little nicer than the built in browser.

Maybe that’s worth $50 to you but if so then you have a lot more money than me.

Update 2nd June 2010: As pointed out in the comments, Wolfram have since reduced the price of the Wolfram Alpha app significantly.   At the time of writing it cost just £1.19 for iPhone and iPad versions of the software – now that’s more like it!

There’s a new CAS in town and this one runs on the iPhone! PocketCAS is the brainchild of Daniel Alm and is based on the powerful open source computer algebra system, XCas.  Functionality includes symbolic calculus, linear algebra, the numerical solution of equations and more – click here to see a more detailed list.

It currently lacks any support for graphics but Daniel tells me that he has plans for the inclusion of both 2 and 3D plotting in the future.  At the time of writing it is selling for just $4.99 which seems a bit of a bargain to me and so much more useful than the 1001 tip calculators that you can find for the iPhone.

I haven’t been able to try it out myself since I don’t have either an iPhone or an iPod touch and, unfortunately for me, Daniel has no plans for an Android version.  However, if these screen shots are anything to go by, it is well worth your time and I would be interested to hear the opinion of anyone who has bought it.

In my previous post I asked how people felt about the future of the Carnival of Maths since it had become poorly organised lately.  Well, a group of us got together, rolled up our sleeves and sorted something out.  The result should see the Carnival going for a couple more years yet at least.  Here are the measures we have taken

• I have taken over the running of the main carnival submission page and have updated it.
• I have created a Twitter feed for the carnival.
• Rod Carvalho has created a google group for people who have contributed to the carnival over the years.  A group of us have been discussing the carnival’s future there and, between us, we will be able to ensure that it stays well organised in the future.  As far as I am concerned, this group runs the show and if they vote to have me replaced as carnival co-ordinator at any time for any reason then I will hand over the reigns.
• The Carnival of Math is now monthly and will be published on the first Friday of every month.  The next one will be on Friday 6th November and will be over at Jason Dyer’s Number Warrior.
• From November, The Math Teachers at Play Carnival will be running on the 3rd Friday of every month in order to co-ordinate with the new Carnival of Math schedule.  This means that you are never far away from your next fix of mathematics blog posts.

That’s pretty much it.  Thanks to everyone who has contributed to the disucssion and to those who helped set things up.  Here’s to the next carnival :)

The Carnival of maths has been around for a while now, recently reaching it’s 58th edition, but its organisation has become erratic of late. The first carnival was put together by Alon Levy who organised it ever since, continuing even after he officially hung up his blogging hat. The mathematics blogging community owes Alon a big debt of gratitude so I hope you all don’t mind if I say “Thank You” to Alon on all our behalves.

Recently, however, it seems that Alon’s other life commitments have caught up with him since the main Carnival page has not been updated since Februrary and the Carnival submission form indicates that the carnival has closed.  I’ve also tried to email him a couple of times (before I hosted the 58th carnival for example) but as yet have received no response.  Alon – if you are reading this then I hope you are doing OK.

So, onto the question posed in this post’s title.  What should the future hold for the Carnival of Maths?  I have thought of a few possible options as follows

• Let it end here.  The community has Math Teachers at Play and that may be enough carnival for us all.
• Continue the carnival in it’s current form but co-ordinated by someone else.
• Start a new carnival series with a title like ‘New Carnival of Maths’.
• None of the above?  Your suggestions would be welcome.

Personally, I favour option 2 – to continue the carnival in its present from but to reduce posting frequency to once a month.  As for co-ordinator – I would be happy to do it if there were no objections (and host a co-ordinating page here) – but if it is felt that this is inappropriate then I would happily step aside and let someone else do it.

It’s your carnival so what do you think?  Comments welcomed.