Archive for May, 2009
I’m impressed! That sums up my initial feelings when I first started playing with Wolfram Alpha. As I played with it some more though it became apparent that this was wonderful project still needs a little work.
I knew that my friend Maria would get a crack at using Wolfram Alpha before I did so I checked her twitter feed to see what she thought about it. Astonishingly she uncovered a basic maths error – Wolfram Alpha tells us (at the time of writing) that the period of the tangent function is 2*pi when it is, in fact, pi. Digging a little further, I discovered that although it gives this incorrect result for the input tan(x),it gets it right for the input tan period.
A minor bug (that I have no doubt will be fixed soon) to be sure but it does show that you can’t always trust the output of any computational system as they’ll always have bugs. Check, check and check again.
It’s fun to see that even though Wolfram Alpha is a major project for WR, it doesn’t always take itself too seriously. For example, it was inevitable that legions of geeks around the world asked the question ‘What is the meaning of life?”. WA gamely returns what we all expect.
I’d love to know how many times that was searched for.
References to popular geek culture continue. For example, you get the following error message when Wolfram Alpha gets overloaded.
I was astonished by the number of different data-sets you can search. For example, it contains US birth data so I can learn that use of the name Michael for new babies is in decline and yet it is currently the 2nd most popular baby’s name in the US today.
Looking at the chart that WA provides me I see that Michael started to become a popular name in the mid-1930s and I suddenly wonder what happened around 1935 to make it so popular. Can any US Historians help me out?
Right, so I know that Michael is the 2nd most common name in the US according to WA but what is the most popular name? How can I get it to give me that? The following searches (and a load of others) didn’t help me (at the time of writing).
This was getting frustrating. So I googled it and eventually came to a page from the social secuity administration. It agreed with WA that the 2nd most popular was Michael and it also tells me that Jacob is the most popular (WA agrees). They disagree on the years though (WA-2007, SSA-2008).
I am bringing this up, not because I am obsessed with names, but because I felt that Wolfram Alpha should be able to compute this sort of thing for me with ease. I’m sure it can but I simply can’t come up with the right search term. As a related question – try getting Wolfram Alpha to tell you which food in it’s database contains the most Vitamin C. I couldn’t do it at the time of writing – let me know if you succeed.
Looking back at this post it seems that my first impressions have been all negative. On the contrary, I am astonished at what it can do and think that it is a major new technology – it just has some growing pains. The mathematical implications are so great IMHO that I’ll leave that to a separate post.
More first impressions of Wolfram Alpha I have found (list will grow with time):
- Maria is worried about the implications that Wolfram Alpha has for teaching math. It has sparked quite a discussion.
- A Disappointing First Look At WolframAlpha
- WolframAlpha – Not a Google Killer, But is a Serious Threat.
- Most Humorous Wolfram Alpha Response (as of yet)
- Wolfram Alpha – Should have called it Deep Thought! – Screencast of one user’s first experiences.
- Sound bites from the BBC.
Update(21st May 2009): The ‘period of tan(x) bug’ has been fixed. That was quick!
SAGE math is probably the best open-source maths package out there at the moment and slowly but surely I have been learning some of its tricks. Combining 2D Plots is particularly easy for example. Let’s plot a cosine curve in red
p1=plot(cos(x),x,-6,6,rgbcolor='red') show(p1)
Now let’s plot a sine curve in green
p2=plot(sin(x),x,-6,6,rgbcolor='green') show(p2)
Combining these two plots into one single plot is extremely intuitive – just add them together
show(p1+p2)
You don’t get much easier than that. Powerful, easy to use and free – loving it!
The Wolfram alpha blog has posted a set of Wolfram Alpha examples where it does some pretty complicated mathematics. For example it can calculate the eigenvalues and eigenvectors of a 3×3 symbolic matrix – just the sort of thing you might turn to Mathematica for.
In another example it shows the result of a symbolic integral (which Mathematica can do) and it gives the steps that a human would need to do to get the same result (which Mathematica can’t do).
I know that Mathematica can do inifnitely more than one-liner calculations such as these but it makes me wonder. Will there be less call for the new, cheap(er) home editition of Mathematica once Wolfram Alpha goes live?
Update (5th November 2009): PyNAG has been updated several times since this post was written. The latest version will always be available from https://www.walkingrandomly.com/?page_id=1662
This is part 6 of a series of articles devoted to demonstrating how to call the Numerical Algorithms Group (NAG) C library from Python. Click here for the index to this series.
Over the last few articles I have demonstrated that it is relatively straightforward to interface Python to the NAG C library. The resulting code, however, looks a bit ugly and overly verbose since you have to include a lot of boiler plate code such as telling Python where the C library is, defining pythonic versions of C structures, defining various constants and so on. I found myself doing a lot of copying and pasting in the early stages of working all this stuff out.
Just recently I made a small step forward by moving all of this boilerplate into a Python package which I have called PyNAG. It’s a relatively simple package since all it does is define Pythonic versions of various C-structures, link to the NAG library, perform restypes on the function names, define a few often used NAG variables such as NAGERR_DEFAULT and so on.
The practical upshot is that you need to write a lot less code when interfacing with the C library. For example here is the Python code to minimise a 2D function via the simplex method using e04ccc of the NAG library.
#!/usr/bin/env python #Mike Croucher - April 2008 #A minimisation problem solved using e04ccc from PyNAG import e04ccc,NAGERR_DEFAULT,E04_DEFAULT,Nag_Comm from ctypes import * from math import * #Create the communication structure comm = Nag_Comm() #set up the objective function def py_obj_func(n,xc,objf,comm): objf[0] = exp(xc[0]) * (xc[0] * 4.0 * ( xc[0]+xc[1] ) + xc[1] * 2.0 * (xc[1] +1.0) +1.0 ) C_FUNC = CFUNCTYPE(None,c_int,POINTER(c_double),POINTER(c_double),POINTER(Nag_Comm) ) c_obj_func = C_FUNC(py_obj_func) #Set up the starting point n=2 x=(c_double*2)() x[0]=c_double(0.4) x[1]=c_double(-0.8) #Set up other variables objf = c_double() e04ccc(n, c_obj_func, byref(x), byref(objf),E04_DEFAULT,comm,NAGERR_DEFAULT)
It’s still not as pretty as it would it if it were written in pure python but it’s a (small) step in the right direction I think.
The good
- PyNAG is free and always will be. You’ll obviously need to buy a copy of the NAG C library though.
- It includes a definition of one of the largest C-Structures I have ever seen – Nag_E04_Opt – which has over 130 members. This took me quite a while to get working and now you don’t need to go through the same pain I did. All the C structures mentioned in these tutorials have been included to save your typing fingers.
- PyNAG has been tested on 32bit and 64bit flavours of Linux and seems to work fine.
- It is relatively easy to add definitions for NAG functions that are not already currently included. In the best case scenario all you need to do is add two lines of code per NAG function. (They’re not all this easy though!)
- It comes with a set of fully functional examples (15 so far with more on the way) along with expected output. This output has been checked against the original C versions of the code to ensure that the results are correct
The bad
- It only works on Linux at the moment. Depending on interest levels (mine and yours) I may get it working on Mac OS X and Windows.
- It only includes all the boilerplate needed to interface with 22 NAG functions at the time of writing. This is a very small percentage of the library as a whole.
- It doesn’t include all of the structures you might need to use throughout the entire library at the moment.
- It doesn’t include any of the enumerations that the NAG C library uses. I plan to include some of the most commonly used ones but will simply have to define them as variables since ctypes doesn’t support enumerations as far as I know.
- This is my first ever Python package so things may be a bit rough around the edges. I welcome advice from more experienced Pythonistas concerning things like package deployment.
- The Python version of the NagError structure has a member called pprint rather than the original print. This is because print is a reserved keyword in Python so I couldn’t use it.
- It has only been tested on Python versions 2.5.x and 2.6.x. I have no idea if it will work on other versions at the moment.
- Documentation is rather sparse at the moment. I haven’t had time to do a better job.
TheUgly
- All PyNAG really does is save you from having to type in a load of boilerplate. It doesn’t properly ‘pythonise’ the NAG C library and so it is painfully obvious that your code is talking to a C library and not a Python library. Ideally I would like to change this and make the user interface a lot more user-friendly. Any input on how best to go about this would be gratefully received.
The Installation
The first public release of PyNAG is version 0.12 and is currently available here (Linux only). Installation instructions are
- Download the tar gzip file above and run the following commands
- tar -xvzf ./PyNAG-0.12.tar.gz
cd PyNAG-0.12/src - edit the following line in the file __init__.py so that it points to your installation of the library
C_libnag= ctypes.cdll.LoadLibrary(“/opt/NAG/cllux08dgl/lib/libnagc_nag.so.8”) - Move back into the root PyNAG-0.12 directory:
cd.. - run the following command with administrator priviledges
python setup.py install
The installer will scatter the PyNAG files where ever your machine is set up to put them. On my machine it was put in the following location but your mileage may vary
/usr/local/lib/python2.6/dist-packages/PyNAG
The Examples
I have written a set of example scripts that show PyNAG in use and these can be found in the src/examples directory of the PyNAG distribution. Mathematically speaking they are all rather dull but they do show how you can call the routines I have included so far. More interesting examples will be appearing soon.
The Future
Getting the NAG C library to work with Python has always been a side project for me and I never expected to take it even this far. Most of this work has been done on the train while commuting to and from work and, although it is far from perfect, I am happy with it so far. There is much to do though and the rate it gets done depends on user-interest. Things that spring to mind (in no particular order) are
- Get PyNAG on a proper CVS system such as github so that other people can easily contribute.
- Add support for Numpy matrices.
- Add support for more of the most commonly used NAG functions.
- Get a windows version up and running.
- Write some more interesting demos.
- Write some sort of automated test script that runs all the examples and compares the results with a set of expected results.
- See if I get any users and take requests :)
Final points.
I do not work for NAG – I just like their products. My professional involvement with them is via the University of Manchester where I am the site-license administrator for NAG products (among many other things). I have a vested interest in getting the best out of their products as well as getting the best out of the likes of Wolfram, Mathworks and a shed-load of other software vendors.
One of my hobbies is pointing out to NAG where their library could do with extra functionality. I am aided in this venture by a load of academics who use the library in it’s various guises and who contact me to say ‘I like NAG but it would be much better if it had the ability to do XYZ’. I am very grateful to NAG for accepting my steady stream of emails with good grace and also for implementing some of our suggestions in the latest version of their library. Feel free to point out any of your favourite algorithms that are missing from the library and why it should have them.
Finally, I’d like to thank the various people at NAG who helped me get my head around their library and for helping me test my initial tentative steps into the NAG/Python world. I owe many of you several beverages of your choice.
As always, comments are welcomed.
Update (5th November 2009): PyNAG has been updated several times since this post was written. The latest version will always be available from https://www.walkingrandomly.com/?page_id=1662
I’ve just been sent some screenshots of Wolfram Alpha in action. There are plenty of other sites that have done this now but here are some from my favourite Wolfram Research informant :)
Dieting advice….how to add apples to oranges.
Power series for sin(x)
Search the human genome
Facts about Polyhedra
Might be useful if you are learning to play the piano (like me)
Looking good so far :) I can’t wait for this to be launched.
Update: Stephen Wolfram has just published a screencast where he demos Wolfram Alpha. Check it out for more juicy details.
Further update: Well Wolfram are a brave lot – they are going to do a live webcast when they launch on Friday. Take a look at the details.
In the old days of computing, life was relatively simple as far as processor speed was concerned. If your current computer ran at 1Ghz and you were about to buy one that ran at 2Ghz then, as a rough rule of thumb, you expected it to be about twice as fast. Of course there was more to it than that (there always is) and other things such as amount of memory, hard disk speed etc etc also had an impact on how fast you got the results of your calculations. It also didn’t help that not all Ghz were created equally – two chips that ran at the same clock speed but with different architectures could exhibit radically different calculation performance.
I did say life was ‘relatively simple’ not just ‘simple’.
These days we still have all of the above to worry about, but we also have the added complication that a typical modern computer contains not one, but two processors. It’s almost like having two machines for the price of one and higher end desktops can contain as many as 8. Which is a lot!
Back in the old, single processor, days we could take our slow code from an old machine, put it on a new machine and it would go faster. That was it – no worries, no hassle. If only life were so simple today….
These days we need to actually rewrite our code so that it makes better use of all the idle processors lying around in our new fangled machines or, to put it more technically, we need to parallelize our code. Sometimes this is simple, often it’s not but one thing is for sure and that is that the environment we are programming in has to provide a set of parallel programming commands to help us do this extra work.
Old style languages such as C and FORTRAN have had this sort of stuff for years in the form of libraries such as Open MPI and Open MP but mathematical programming languages such as Mathematica and Maple have only recently gotten in on the act.
Version 7 of Mathematica and Version 13 of Maple* both included a set of tools for enabling parallel programming as part of their basic installs. In other words – you don’t need to pay any extra money to get these programs to use your multi-processor computers to their full potential.
MATLAB, on the other hand, expects you to pay extra to be able to parallelize your code by requiring you to purchase the Parallel Computing Toolbox. It’s not cheap either! Of course, Mathworks are free to charge whatever they like for their products but I don’t think that this particular policy is doing them any favours.
It’s not all bad – certain functions in basic MATLAB make full use of your multi core processors. As of version 2009a, for example, their Fast Fourier Transform function (fft) will perform much faster on multi-processor machines but if you want to parallelize your own code then you still have to buy the Parallel toolbox. Your alternatives include messing around with batch processing (if your problem can be solved this way) or use another package such as Mathematica or Maple which has full parallel support out of the box.
At my University we have several hundred network licenses for MATLAB itself and only 2 or 3 licenses for the Parallel toolbox. That doesn’t mean that people aren’t interested in parallel MATLAB, quite the opposite, they simply do not want to have to pay extra to parallelize code on desktops and will go to great lengths (all legal I hasten to add) to avoid the extra cost.
Unfortunately for Mathworks one of the easiest options for someone starting on a new project is simply to use one of their competitors and this is starting to happen. Manchester has a site license for Mathematica and I saw a big spike in interest after version 7 thanks to the parallel programming features it contains.
In summary I think that The Mathworks would do themselves a great favour by including the parallel toolbox as part of their basic package.
Any thoughts?
*Maple actually had parallel support before version 13 but it has been vastly improved in this latest version
I have got a long list entitled blog articles I want to write this year and about halfway down it are the words Write a tutorial for working with Excel Spreadsheets in MATLAB. Thanks to the guys at Blinkdagger I have crossed off that particular item without writing a thing. They have produced two fantastic tutorials at the following links
If you have any topics that you would like me to add to the list of blog articles to write then let me know in the comments section.
I recently tried to get some screenshots of Wolfram Alpha from my contacts at Wolfram Research and failed miserably. Iddo Friedberg, however, had a lot more luck so head over to his blog, Byte Sized Biology, for some juicy Wolfram Alpha pre-release screenshot goodness.
Yes…I Tweet :)
Feel free to follow me on MikeCr if you like.