{"id":1977,"date":"2009-12-14T13:10:37","date_gmt":"2009-12-14T12:10:37","guid":{"rendered":"http:\/\/www.walkingrandomly.com\/?p=1977"},"modified":"2009-12-14T13:10:37","modified_gmt":"2009-12-14T12:10:37","slug":"should-matlab-have-a-validation-suite","status":"publish","type":"post","link":"https:\/\/walkingrandomly.com\/?p=1977","title":{"rendered":"Should MATLAB have a validation suite?"},"content":{"rendered":"

Earlier this month I had a query from a colleague who had installed MATLAB on some very large and powerful workstations which are used by many researchers at our University. This colleague is a sysadmin of the old school and he doesn’t like installing any program that doesn’t have some sort of validation suite since he wants to be sure that the program is working OK on his hardware for his people to the best of his ability.<\/p>\n

He can do this for the NAG library<\/a> and he can do it for SAGE math<\/a> (among others) but not, as far as he knew,for MATLAB. So he asked me the question “Does MATLAB have a validation suite?<\/strong>” and since I didn’t have a clue I passed it onto Mathworks Tech support who quickly informed us that the answer was “No.”<\/p>\n

The essential gist of the reply was that they assume that MATLAB produces the same results on different machines except, of course, when it doesn’t<\/a>!<\/p>\n

Of course, people such as my colleague feel that ASSUMING that MATLAB produces the same result on different machines is not good enough which is why he wanted a validation suite in the first place. Life is short though and we didn’t pursue it further – partly because we didn’t have an example which proved Mathwork’s assumption wrong.<\/p>\n

We do now though!<\/p>\n

On a 64bit Windows 7 install of MATLAB 2009b we get the following (incorrect) behaviour:<\/p>\n

 p = [1,11-i;2,2;3,3;4,4;5,5;6,6;7,7;8,8;9,9;10,10;11,11];\r\nsort(p)\r\n\r\nans =\r\n\r\n   1.0000             2.0000 - 1.0000i\r\n   2.0000             3.0000\r\n   3.0000             4.0000\r\n   4.0000             5.0000\r\n   5.0000             6.0000\r\n   6.0000             7.0000\r\n   7.0000             8.0000\r\n   8.0000             9.0000\r\n   9.0000            10.0000\r\n  10.0000            11.0000\r\n  11.0000            11.0000<\/pre>\n
On the same machine but booted into 64bit Ubuntu 9.10 we get the correct behaviour:<\/p>\n
 p = [1,11-i;2,2;3,3;4,4;5,5;6,6;7,7;8,8;9,9;10,10;11,11];\r\nsort(p)\r\n\r\nans =\r\n\r\n   1.0000             2.0000\r\n   2.0000             3.0000\r\n   3.0000             4.0000\r\n   4.0000             5.0000\r\n   5.0000             6.0000\r\n   6.0000             7.0000\r\n   7.0000             8.0000\r\n   8.0000             9.0000\r\n   9.0000            10.0000\r\n  10.0000            11.0000\r\n  11.0000            11.0000 - 1.0000i<\/pre>\n
Is this exactly the type of bug that might have been caught in a validation suite?  Would a user-runnable validation suite be useful?<\/p>\n
Thanks to Michal Kvasnicka who alerted me to this bug which was first described in a discussion thread on MATLAB central<\/a>.  Mathworks are aware of the bug and have logged it as issue number 594221.<\/p>\n","protected":false},"excerpt":{"rendered":"
Earlier this month I had a query from a colleague who had installed MATLAB on some very large and powerful workstations which are used by many researchers at our University. This colleague is a sysadmin of the old school and he doesn’t like installing any program that doesn’t have some sort of validation suite since […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[4,11],"tags":[],"class_list":["post-1977","post","type-post","status-publish","format-standard","hentry","category-math-software","category-matlab"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p3swhs-vT","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/walkingrandomly.com\/index.php?rest_route=\/wp\/v2\/posts\/1977","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/walkingrandomly.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/walkingrandomly.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/walkingrandomly.com\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/walkingrandomly.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1977"}],"version-history":[{"count":19,"href":"https:\/\/walkingrandomly.com\/index.php?rest_route=\/wp\/v2\/posts\/1977\/revisions"}],"predecessor-version":[{"id":2051,"href":"https:\/\/walkingrandomly.com\/index.php?rest_route=\/wp\/v2\/posts\/1977\/revisions\/2051"}],"wp:attachment":[{"href":"https:\/\/walkingrandomly.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1977"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/walkingrandomly.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1977"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/walkingrandomly.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1977"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}