{"id":2678,"date":"2010-06-26T10:07:42","date_gmt":"2010-06-26T09:07:42","guid":{"rendered":"http:\/\/www.walkingrandomly.com\/?p=2678"},"modified":"2010-06-26T10:08:58","modified_gmt":"2010-06-26T09:08:58","slug":"a-free-version-of-the-pdist-command-for-matlab","status":"publish","type":"post","link":"https:\/\/walkingrandomly.com\/?p=2678","title":{"rendered":"A free version of the pdist command for MATLAB"},"content":{"rendered":"

MATLAB contains a function called pdist<\/a> that calculates the ‘Pairwise distance between pairs of objects’. Typical usage is<\/p>\n

X=rand(10,2);\r\ndists=pdist(X,'euclidean');\r\n<\/pre>\n
It’s a nice function but the problem with it is that it is part of the Statistics Toolbox<\/a> and that costs extra.  I was recently approached by a user who needed access to the pdist function but all of the statistics toolbox license tokens on our network were in use and this led to the error message<\/p>\n
??? License checkout failed.\r\nLicense Manager Error -4\r\nMaximum number of users for Statistics_Toolbox reached.\r\nTry again later.\r\nTo see a list of current users use the lmstat utility or contact your License Administrator<\/pre>\n
One option, of course, is to buy more licenses for the statistics toolbox but there is another way.  You may have heard of GNU Octave<\/a>, a free,open-source MATLAB-like program that has been in development for many years.\u00a0 Well, Octave has a sister project called Octave-Forge<\/a> which aims to provide a set of free toolboxes for Octave.\u00a0 It turns out that not only does Octave-forge contain a statistics toolbox but that toolbox contains an pdist function.\u00a0 I wondered how hard it would be to take Octave-forge’s pdist function and modify it so that it ran on MATLAB.<\/p>\n
Not very!\u00a0 There is a script called oct2mat<\/a> that is designed to automate some of the translation but I chose not to use it – I prefer to get my hands dirty you see.\u00a0 I named the resulting function octave_pdist to help clearly identify the fact that you are using an Octave function rather than a\u00a0 MATLAB function.\u00a0 This may matter if one or the other turns out to have bugs.\u00a0 It appears to work rather nicely:<\/p>\n
dists_oct = octave_pdist(X,'euclidean');\r\n% let's see if it agrees with the stats toolbox version\r\nall( abs(dists_oct-dists)<1e-10)\r\n\r\nans =\r\n     1\r\n<\/pre>\n
Let’s look at timings on a slightly bigger problem.<\/p>\n
>> X=rand(1000,2);\r\n>> tic;matdists=pdist(X,'euclidean');toc\r\nElapsed time is 0.018972 seconds.\r\n>> tic;octdists=octave_pdist(X,'euclidean');toc\r\nElapsed time is 6.644317 seconds.\r\n<\/pre>\n
Uh-oh!  The Octave version is 350 times slower (for this problem) than the MATLAB version.  Ouch!   As far as I can tell, this isn’t down to my dodgy porting efforts, the original Octave pdist really does take that long on my machine (Ubuntu 9.10, Octave 3.0.5).<\/p>\n
This was far too slow to be of practical use and we didn’t want to be modifying algorithms so we ditched this function and went with the NAG Toolbox for MATLAB<\/a> instead (routine g03ea<\/a> in case you are interested) since Manchester effectively has an infinite number of licenses for that product.<\/p>\n
If,however, you’d like to play with my MATLAB port of Octave’s pdist then download it below.<\/p>\n