Archive for the ‘python’ Category
xkcd is a popular webcomic that sometimes includes hand drawn graphs in a distinctive style. Here’s a typical example
In a recent Mathematica StackExchange question, someone asked how such graphs could be automatically produced in Mathematica and code was quickly whipped up by the community. Since then, various individuals and communities have developed code to do the same thing in a range of languages. Here’s the list of examples I’ve found so far
- xkcd style graphs in Mathematica. There is also a Wolfram blog post on this subject.
- xkcd style graphs in R. A follow up blog post at vistat.
- xkcd style graphs in LaTeX
- xkcd style graphs in Python using matplotlib
- xkcd style graphs in MATLAB. There is now some code on the File Exchange that does this with your own plots.
- xkcd style graphs in javascript using D3
- xkcd style graphs in Euler
- xkcd style graphs in Fortran
Any I’ve missed?
If you have an interest in mathematics, you’ve almost certainly stumbled across The Wolfram Demonstrations Project at some time or other. Based upon Wolfram Research’s proprietary Mathematica software and containing over 8000 interactive demonstrations, The Wolfram Demonstrations Project is a fantastic resource for anyone interested in mathematics and related sciences; and now it has some competition.
Sage is a free, open source alternative to software such as Mathematica and, thanks to its interact function, it is fully capable of producing advanced, interactive mathematical demonstrations with just a few lines of code. The Sage language is based on Python and is incredibly easy to learn.
The Sage Interactive Database has been launched to showcase this functionality and its looking great. There’s currently only 31 demonstrations available but, since anyone can sign up and contribute, I expect this number to increase rapidly. For example, I took the simple applet I created back in 2009 and had it up on the database in less than 10 minutes! Unlike the Wolfram Demonstrations Project, you don’t need to purchase an expensive piece of software before you can start writing Sage Interactions….Sage is free to everyone.
Not everything is perfect, however. For example, there is no native Windows version of Sage. Windows users have to make use of a Virtualbox virtual machine which puts off many people from trying this great piece of software. Furthermore, the interactive ‘applets’ produced from Sage’s interact function are not as smooth running as those produced by Mathematica’s Manipulate function. Finally, Sage’s interact doesn’t have as many control options as Mathematica’s Manipulate (There’s no Locator control for example and my bounty still stands).
The Sage Interactive Database is a great new project and I encourage all of you to head over there, take a look around and maybe contribute something.
My attention was recently drawn to a Google+ post by JerWei Zhang where he evaluates 2^3^4 in various packages and notes that they don’t always agree. For example, in MATLAB 2010a we have 2^3^4 = 4096 which is equivalent to putting (2^3)^4 whereas Mathematica 8 gives 2^3^4 = 2417851639229258349412352 which is the same as putting 2^(3^4). JerWei’s post gives many more examples including Excel, Python and Google and the result is always one of these two (although to varying degrees of precision).
What surprised me was the fact that they disagreed at all since I thought that the operator precendence rules were an agreed standard across all software packages. In this case I’d always use brackets since _I_ am not sure what the correct interpretation of 2^3^4 should be but I would have taken it for granted that there is a standard somewhere and that all of the big hitters in the numerical world would adhere to it.
Looks like I was wrong!
Late last year I was asked to give a talk at my University to a group of academics studying financial mathematics (Black-Scholes, monte-carlo simulations, time series analysis – that sort of thing). The talk was about the NAG Numerical Library, what it could do, what environments you can use it from, licensing and how they could get their hands on it via our site license.
I also spent a few minutes talking about Python including why I thought it was a good language for numerical computing and how to interface it with the NAG C library using my tutorials and PyNAG code. I finished off with a simple graphical demonstration that minimised the Rosenbrock function using Python, Matplotlib and the NAG C-library running on Ubuntu Linux.
“So!”, I asked, “Any questions?”
The very first reply was “Does your Python-NAG interface work on Windows machines?” to which the answer at the time was “No!” I took the opportunity to ask the audience how many of them did their numerical computing in Windows (Most of the room of around 50+ people), how many people did it using Mac OS X (A small group at the back), and how many people did it in Linux (about 3).
So, if I wanted the majority of that audience to use PyNAG then I had to get it working on Windows. Fortunately, thanks to the portability of Python and the consistency of the NAG library across platforms, this proved to be rather easy to do and the result is now available for download on the main PyNAG webpage.
Let’s look at the main differences between the Linux and Windows versions
Loading the NAG Library
In the examples given in my Linux NAG-Python tutorials, the cllux08dgl NAG C-library was loaded using the line
libnag = cdll.LoadLibrary("/opt/NAG/cllux08dgl/lib/libnagc_nag.so.8")
In Windows the call is slightly different. Here it is for CLDLL084ZL (Mark 8 of the C library for Windows)
C_libnag = ctypes.windll.CLDLL084Z_mkl
If you are modifying any of the codes shown in my NAG-Python tutorials then you’ll need to change this line yourself. If you are using PyNAG, however, then it will already be done for you.
Callback functions
For my full introduction to NAG callback functions on Linux click here. The main difference between using callbacks on Windows compared to Linux is that on Linux we have
C_FUNC = CFUNCTYPE(c_double,c_double )
but on Windows we have
C_FUNC = WINFUNCTYPE(c_double,c_double )
There are several callback examples in the PyNAG distribution.
That’s pretty much it I think. Working through all of the PyNAG examples and making sure that they ran on Windows uncovered one or two bugs in my codes that didn’t affect Linux for one reason or another so it was a useful exercise all in all.
So, now you head over to the main PyNAG page and download the Windows version of my Python/NAG interface which includes a set of example codes. I also took the opportunity to throw in a couple of extra features and so upgraded PyNAG to version 0.16, check out the readme for more details. Thanks to several employees at NAG for all of their help with this including Matt, John, David, Marcin and Sorin.
Over at Mathrecreation, Dan Mackinnon has been plotting two dimensional polygonal numbers on quadratic number spirals using a piece of software called Fathom. Today’s train-journey home project for me was to reproduce this work using Mathematica and Python.
I’ll leave the mathematics to Mathrecreation and Numberspiral.com and just get straight to the code. Here’s the Mathematica version:
- Download the Mathematica Notebook
- Download a fully interactive version that will run on the free Mathematica Player
p[k_, n_] := ((k - 2) n (n + 1))/2 - (k - 3) n;
Manipulate[
nums = Table[p[k, n], {n, 1, numpoints}];
If[showpoints,
ListPolarPlot[Table[{2*Pi*Sqrt[n], Sqrt[n]}, {n, nums}]
, Joined -> joinpoints, Axes -> False,
PlotMarkers -> {Automatic, Small}, ImageSize -> {400, 400}],
ListPolarPlot[Table[{2*Pi*Sqrt[n], Sqrt[n]}, {n, nums}]
, Joined -> joinpoints, Axes -> False, ImageSize -> {400, 400}
]
]
, {k, 2, 50, Appearance -> "Labeled"}
, {{numpoints, 100, "Number of points"}, {100, 250, 500}}
, {{showpoints, True, "Show points"}, {True, False}}
, {{joinpoints, True, "Join points"}, {True, False}}
, SaveDefinitions :> True
]
A set of 4 screenshots is shown below (click the image for a higher resolution version). The formula for polygonal numbers, p(kn,n), is only really valid for integer k and so I made a mistake when I allowed k to be any real value in the code above. However, I liked some of the resulting spirals so much that I decided to leave the mistake in.
When Dan showed the images he produced from Fathom, he mentioned that he would one day like to implement these systems in a more open platform. Well, you can’t get much more open than Python so here’s a piece of code that makes use of Python and the matplotlib package (download the file – polygonal_spiral.py).
#!/usr/bin/env python import matplotlib.pyplot as plt from math import * def p(k,n): return(((k-2)*n*(n+1))/2 -(k-3)*n) k=13 polygonal_nums = [p(k,n) for n in range(100)] theta = [2*pi*sqrt(n) for n in polygonal_nums] r = [sqrt(n) for n in polygonal_nums] myplot = plt.polar(theta,r,'.-') plt.gca().axis('off') plt.show() |
Now, something that I haven’t figured out yet is why the above python code works just fine for 100 points but if you increase it to 101 then you just get a single point. Does anyone have any ideas?
Unlike my Mathematica version, the above Python program does not give you an interactive GUI but I’ll leave that as an exercise for the reader.
If you enjoyed this post then you might also like:
Earlier today I was chatting to a lecturer over coffee about various mathematical packages that he might use for an upcoming Masters course (note – offer me food or drink and I’m happy to talk about pretty much anything). He was mainly interested in Mathematica and so we spent most of our time discussing that but it is part of my job to make sure that he considers all of the alternatives – both commercial and open source. The course he was planning on running (which I’ll keep to myself out of respect for his confidentiality) was definitely a good fit for Mathematica but I felt that SAGE might suite him nicely as well.
“Does it have nice, interactive functionality like Mathematica’s Manipulate function?” he asked
Oh yes! Here is a toy example that I coded up in about the same amount of time that it took to write the introductory paragraph above (but hopefully it has no mistakes). With just a bit of effort pretty much anyone can make fully interactive mathematical demonstrations using completely free software. For more examples of SAGE’s interactive functionality check out their wiki.
Here’s the code:
def ftermSquare(n):
return(1/n*sin(n*x*pi/3))
def ftermSawtooth(n):
return(1/n*sin(n*x*pi/3))
def ftermParabola(n):
return((-1)^n/n^2 * cos(n*x))
def fseriesSquare(n):
return(4/pi*sum(ftermSquare(i) for i in range (1,2*n,2)))
def fseriesSawtooth(n):
return(1/2-1/pi*sum(ftermSawtooth(i) for i in range (1,n)))
def fseriesParabola(n):
return(pi^2/3 + 4*sum(ftermParabola(i) for i in range(1,n)))
@interact
def plotFourier(n=slider(1, 30,1,10,'Number of terms')
,plotpoints=('Value of plot_points',[100,500,1000]),Function=['Saw Tooth','Square Wave','Periodic Parabola']):
if Function=='Saw Tooth':
show(plot(fseriesSawtooth(n),x,-6,6,plot_points=plotpoints))
if Function=='Square Wave':
show(plot(fseriesSquare(n),x,-6,6,plot_points=plotpoints))
if Function=='Periodic Parabola':
show(plot(fseriesParabola(n),x,-6,6,plot_points=plotpoints))
Just a quick note to say that I’ve released version 0.15 of PyNAG – my Python interface to the NAG C library and given it its own dedicated page on Walking Randomly. New features include
- Basic support for 165 functions from Mark 8 of the NAG C library.
- 29 simple but fully functional examples.
- Added initial support for NAG’s Complex structure.
If you have learned how to program then the odds are that you have written a program that displays a list of prime numbers. A prime number generator is a programming rite of passage that is almost up there with the ubiquitous ‘Hello World‘ application (for programmers who are going on to do numerical work at least).
There are various ways you might choose to do it but I bet that your first (or even your second or third) thought wouldn’t be ‘I know – I’ll use a regular expression‘. Someone known only as ‘Abigail’ obviously thought that the world desperately needed such a regex since he / she came up with the following incantation
^1?$|^(11+?)\1+$
Which is a regular expression that I assure you can be used to generate a list of all the prime numbers. If you are in the mood for a puzzle then sit back, grab a cup of coffee and try to work out HOW it can generate the primes.
If you give up then head over to Python Programming for a full explanation along with Python code that demonstrates that it really does work.
Can anyone come up with regular expressions for other interesting series of numbers?
Update (27th August 2009): Someone pointed me to an alternative explanation here.
I’ll be attending the 2009 EuroSciPy scientific Python conference in Leipzig next month – July 25th and 26th to be precise – and would love to hear from anyone else who’ll be attending.
Back in 1997 I was a 2nd year undergraduate of Physics and I was taught how to program in Fortran, a language that has survived over 40 years due to several facts including
- It is very good at what it does. Well written Fortran code, pushed through the right compiler is screamingly fast.
- There are millions of lines of legacy code still being used in the wild. If you end up doing research in subjects such as Chemistry, Physics or Engineering then you will almost certainly bump into Fortran code (I did!).
- A beginner’s course in Fortran has been part of the staple diet in degrees in Physics, Chemistry and various engineering disciplines (among others) for decades.
- It constantly re-invents itself to include new features. I was taught Fortran 77 (despite it being 1997) but you can now also have your pick of Fortran 90, 95, 2003, and soon 2008.
Almost everyone I knew hated that 1997 Fortran course and the reasons for the hatred essentially boiled down to one of two points depending on your past experience.
- Fortran was too hard! So much work for such small gains (First time – programmers)
- The course was far too easy. It was just a matter of learning Fortran syntax and blitzing through the exercises. (People with prior experience)
The course was followed by a numerical methods course which culminated in a set of projects that had to be solved in Fortran. People hated the follow on course for one of two reasons
- They didn’t have a clue what was going on in the first course and now they were completely lost.
- The problems given were very dull and could be solved too easily. In Excel! Fortran was then used to pass the course.
Do you see a pattern here?
Fast forward to 2009 and I see that Fortran is still being taught to many undergraduates all over the world as their first ever introduction to programming. Bear in mind that these students are used to being able to get interactive 3D plots from the likes of Mathematica, Maple or MATLAB and can solve complex differential equations simply by typing them into Wolfram Alpha on the web. They can solve problems infinitely more complicated then the ones I was faced with in even my most advanced Fortran courses with just a couple of lines of code.
Learn Fortran – spend a semester achieving not very much
These students study subjects such as physics and chemistry because they are interested in the subject matter and computers are just a way of crunching through numbers (and the algebra for that matter) as far as they are concerned. Despite having access to untold amounts of computational and visualisation power coupled with easy programming languages thanks to languages such as MATLAB and Python, these enthusiastic, young potential programmers get forced to bend their minds around the foibles of Fortran.
For many it’s their first ever introduction to programming and they get forced to work with one of the most painful programming languages in existence (in my opinion at least). Looking at a typical one-semester course it seems that by the time they have finished they will be able to produce command line only programs that do things like multiply matrices together (slowly), solve the quadratic equation and find the mean and standard deviation of a list of numbers.
Not particularity impressive for an introduction to the power of computation in their subject is it?
Fortran syntax is rather unforgiving compared to something like Python and it takes many lines of code to achieve even relatively simple results. Don’t believe me? OK, write a program in pure Fortran that gives a plot of a Sin(n*x) for integer n and x ranging from -2*pi to 2*pi. Now connect that plot up to a slider control which will control the value of n. Done? Ok – now get it working on another operating system (eg if you originally used Windows, get it to work on Mac OS X).
Now try the same exercise in Python or Mathematica.
This is still a very basic program but it would give a much greater sense of achievement compared to finding the mean of list of numbers and could easily be extended for more able students (Fourier Series perhaps). I believe that many introductory Fortran programming courses end up teaching students that programming means ‘Calculating things the hard way’ when they should be leaving an introductory course with the opposite impression.
Learn Fortran – and never use it again.
Did you learn Fortran at University? Are you still using it? If you answer yes to both of those questions then there is a high probability that you are still involved in research or that advanced numerical analysis is the mainstay of your job. I know a lot of people in the (non-academic) IT industry – many of them were undergraduates in Physics or Chemistry and so they learned Fortran. They don’t use it anymore. In fact they never used it since completing their 1st semester, 2nd year exam and that includes the computational projects they chose to do as part of their degrees.
The fact of the matter is that most undergraduates in subjects such as Physics end up in careers that have nothing to do with their degree subject and so most of them will never use Fortran ever again. The ‘programming concepts’ they learned might be useful if they end up learning Java, Python or something along those lines but that’s about it. Would it not be more sensible to teach a language that can support the computational concepts required in the underlying subject that also has an outside chance of being used outside of academia?
Teach Fortran – and spend a fortune on compilers
There are free Fortran compilers available but in my experience these are not the ones that get used the most for teaching or research and this is because the commercial Fortran compilers tend to be (or at least,they are perceived to be) much better. The problem is that when you are involved with looking after the software portfolio of a large University (and I am) then no one will agree on what ‘the best’ compiler is. (Very) roughly paraphrased, here are some comments I have received from Fortran programmers and teachers over the last four years or so.
- When you teach Fortran, you MUST use the NAG Fortran compiler since it is the most standards compliant. The site license allows students to have it on their own machines which is useful.
- When you teach Fortran, you MUST use the Silverfrost compiler because it supports Windows GUI programming via Clearwin.
- We MUST avoid the Silverfrost compiler for teaching because it is not available on platforms such as Mac and Linux.
- We MUST avoid the NAG compiler for teaching because although it has a great user interface for Windows, it is command line only for Linux and Mac. This confuses students.
- If we are going to teach Fortran then we simply MUST use the Intel Fortran Compiler. It’s the best and the fastest.
- We need the Intel Fortran Compiler for research because it is the only one compatible with Abaqus.
- A site license for Absoft Fortran is essential. It’s the only one that will compile <insert application here> which is essential for my group’s work.
- We should only ever use free Fortran compilers – relying on commercial offerings is wrong.
I’m only getting started! Choose a compiler, any compiler – I’ll get back to you within the week with an advocate who thinks we should get a site license for it and another who hates it with a passion. Let’s say you had a blank cheque book and you gave everyone exactly what they wanted – your institution would be spending tens of thousands of pounds on Fortran compilers and you’d still be annoying the free-software advocate.
Needless to say, most people don’t have a blank cheque book. At Manchester University (my workplace) we support 2 site-wide Fortran compilers
- The NAG Compiler – recommended for teaching.
- The Intel Fortran Compiler – recommended for research
We have a truly unlimited site license for NAG (every student can have a copy on their own machine if they wish) which makes it perfect from a licensing point of view and many people like to use it to teach. The Widows version has a nice GUI and help system for example – perfect for beginners. There are Mac and Linux versions too and although these are command-line only, they are better than nothing.
The Intel Fortran Compiler licenses we have are in the form of network licenses and we have a very limited number of these – enough to support the research of every (Windows and Linux) Fortran programmer on campus but nowhere near enough for teaching. To get enough for teaching would cost a lot.
There are also pockets of usage of various other compilers but nothing on a site-wide basis.
Of course, not everyone is happy with this set-up, and I was only recently on the receiving end of a very nasty email from someone because I had to inform him that we didn’t have the money to buy his compiler of choice. It’s not my fault that he couldn’t have it (I don’t have a budget and never have had) but he felt that I was due an ear lashing I guess and blamed all of his entire department’s woes on me personally. Anway, I digress….
The upshot is that Fortran compilers are expensive and no one can agree on which one(s) you should get. If you get them all to please (almost) everybody then you will be very cash poor and it won’t be long before the C++ guys start knocking on your door to discuss the issue of a commercial C++ compiler or three….
Learn Fortran – because it is still very useful
Fortran has been around in one form or another for significantly longer than I have been alive for a very good reason. It’s good at what it does. I know a few High Performance Computing specialists and have been reliably informed that if every second counts in your code execution – if it absolutely, positively, definitely has to run as fast as humanly possible then it is hard to beat Fortran. I take their word for it because they know what they are talking about.
Organisations such as the Numerical Algorithm’s Group (NAG) seem to agree with this stance since their core product is a Fortran Library and NAG have a reputation that is hard to ignore. When it comes to numerics – they know their stuff and they work in Fortran so it MUST be a good choice for certain applications.
If you find yourself using research-level applications such as Abaqus or Gaussian then you’ll probably end up needing to use Fortran, just as you will if you end up having to modify one of the thousands of legacy applications out there. Fortran is a fact of life for many graduate students. It was a fact of life for me too once and I hated it.
I know that it is used a lot at Manchester for research because ,as mentioned earlier, we have network licenses for Intel Fortran and when I am particularly bored I grep the usage logs to see how many unique user names have used it to compile something. There are many.
I am not for a second suggesting that Fortran is irrelevant because it so obviously isn’t. It is heavily used in certain, specialist areas. What I am suggesting is that it is not the ideal language to use as a first exposure to programming. All of the good reasons for using Fortran seem to come up at a time in your career when you are a reasonably advanced programmer not when you first come across the concepts loops and arrays.
Put bluntly I feel that the correct place for learning Fortran is in grad school and only if it is needed.
If not Introductory Fortran then Introductory what?
If you have got this far then you can probably guess what I am going to suggest – Python. Python is infinitely more suitable for beginner programmers in my opinion and with even just a smattering of the language it is possible to achieve a great deal. Standard python modules such as matplotlib and numpy help take care of plotting and heavy-duty numerics with ease and there are no licensing problems to speak of . It’s free!
You can use it interactively which helps with learning the basics and even a beginner can produce impressive looking results with relatively little effort. Not only is it more fun than Fortran but it is probably a lot more useful for an undergraduate because they might actually use it. Its use in the computational sciences is growing very quickly and it also has a lot of applications in areas such as the web, games and general task automation.
If and when a student is ready to move onto graduate level problems then he/she may well find that Python still fulfills their needs but if they really do need raw speed then they can either hook into existing Fortran libraries using Python modules such as ctypes and f2py or they can roll up their sleeves and learn Fortran syntax.
Using Fortran to teach raw beginners is a bit like teaching complex numbers to kindergarten children before they can count to ten. Sure, some of them will need complex numbers one day but if it was complex numbers or nothing then a lot of people would really struggle to count.
So, I am (finally) done. Do I have it all wrong? Is Fortran so essential that Universities would be short changing students of numerate degrees if they didn’t teach it or are people just teaching what they were taught themselves because that’s the way it has always been done? I am particularly interested in hearing from either students or teachers of Fortran but, as always, comments from everyone are welcome – even if you disagree with me.
If you are happy to talk in public then please use the comments section but if you would prefer a private discussion then feel free to email me.
(I am skating closer to my job than I usually do on this blog so here is the hopefully unnecessary disclaimer. These are my opinions alone and do not necessarily reflect the policy of my employer, The Univeristy of Manchester. If you find yourself coming to me for Fortran compiler support there then I will do the very best I can for you – no matter what my personal programming prejudices may be. However, if you are up for a friendly chat over coffee concerning this then feel free to drop me a line. )