hackerific

During the course of my PhD, I’ve spent quite a lot of time examining methods of interpolating scattered data. By scattered, I mean something that looks like this:

Uploaded with plasq’s Skitch!

Or alternatively, like a very, very gappy image. This type of data is very common in geoscience, and so interpolating it so it looks more like this:

Uploaded with plasq’s Skitch!

Is a common activity. As well as examining common methods of interpolating scattered data, such as cubic, linear and nearest neighbour methods, I’ve looked at natural neighbour interpolation, radial basis function interpolation, everyone’s favourite kriging and a less common methods known as adaptive normalised convolution (ANC).

In order to do meaningful comparisons, I needed working implementations of these, and so ended up writing my own versions of ANC, radial basis function interpolation and kriging. I also wrote a wrapper to Pavel Sakov’s natural neighbour interpolation.

I’ve just released these functions as a toolkit for Matlab, with a BSD license. You can find the download links on the toolkit’s homepage, which also gives the commands you need to clone the repository.

To build the toolkit, you’ll need a working build environment, with make, gcc and python 2.5 as well as a copy of Matlab. I’ve tested the kit on Debian/Ubuntu Linux with Matlab 2007a and Mac OS X (10.5). If you’re using a Mac, you’ll need XCode installed.

You’ll probably need to tell make where your mex binary is (MEX is Matlab’s external interface builder). To do this, run:

make MEX=/path/to/mex

You’ll find mex inside your Matlab installation, under the bin/ directory. On linux, it may be symlinked into /usr/local/bin.

Once make has run, you’ll find a directory called toolkit, which contains the interpolation toolkit. Copy this folder somewhere appropriate, and add it to your Matlab path, by running:

addpath('/path/to/toolkit')

Within Matlab. You can then run help toolkit, for some help and examples of how to test the methods.

I’d recommend using natural neighbour interpolation everywhere you would previously have used matlab’s griddata function. You can find more information on interpolation in my publications, which you can read and download here.

I’m fairly sure that aesthetics play a large part in how people view your work – ugly correct results are probably viewed as being worse than pretty, but wrong results. Aesthetics must play a fairly big role in how people perceive your work.

I’ve just finished generating a lot of quiver plots for my thesis. A quiver plot is a graph of vectors, illustrating a flow field. My results typically show an image with motion vectors overlaid, all plotted with Matlab, a piece of software which is very heavily used in academia in the UK (and probably all over the world).

Matlab quiver plots are ugly, and all of my plotting got me wondering how other system’s quiver plots look. In particular, I’ve started playing with numpy and scipy, so I decided to do a quick comparison between matlab, and matplotlib, the python close of matlab’s plotting tools. The results are very interesting.

I generated a quick example motion field, and plotted it with quiver. Then I took screenshots. I used the Gimp on my Linux box, for the matlab example) and Skitch on my MacBook for the python example.

First, here’s the matlab example. This was generated on linux using Matlab 2007a:

Uploaded with plasq’s Skitch!

Here’s the matplotlib python example, generated using the enthought python distribution. I used colour here because I could: matplotlib’s quiver supports it out of the box. Matlab’s doesn’t.

Uploaded with plasq’s Skitch!

I think the difference between the results is abundantly clear. Matplotlib’s arrows look great, they’re shapely and well scaled. Matlab’s look like jaggedy lines in comparison. However, I think the main difference is down to anti-aliasing. The matplotlib’s result looks better because they’re smoother and look far more modern the 90s throwback that is matlab’s interface (on linux and Mac at least – I can’t speak for Windows).

These results have pretty much convinced me to try my next project using Python and scipy instead of Matlab. Not only is it completely free (EPD is free for academic use, and matplotlib, scipy and numpy are just plan free), the aesthetic is there too. It’s almost like python’s elegant simplicity ideals spilled over into their plotting engines.

Well, I’m now back from UWIC, in Cardiff, so I thought I’d write down my thoughts of what’s currently happening in the world of image processing for remote sensing.

First of all though, an aside. The venue totally sucked. It was hard to get to, and was in the scummy outskirts of what’s an otherwise really pretty city. I hope they didn’t put anyone off what’s an otherwise really beautiful country!

Anyway, now that the rant is done with, I found the actual conference quite interesting, especially since there was more image processing content that at IGARSS 2008. The conference chair, Lorenzo Bruzzone gave an interesting introduction, which summed up how the content was distributed.

I made a quick pie chart to illustrate:

I’m pretty sure I’m in that teeny slice! He also gave a quick breakdown of the number of abstract submitted (63), along with how many oral presentations there were (31, minus a few who weren’t there), and how many posters (11).

I’ve put my slides online at Slideshare so if you’re so inclined you can have a look (no audio).

The exhibition didn’t have a lot for those of us who weren’t looking to buy very high resolution hyperspectral imagers, and the like, but it was still interesting.

Overall, it was worth going, but it was hard to get to (even from Bath), using public transport, and the location left a lot to be desired.

I’ve just submitted this paper for the SPIE Europe Remote Sensing Conference due to be help in Cardiff on 16th September, 2008.

It can be posted online provided I display the following notice:

Copyright 2008 Society of Photo-Optical Instrumentation Engineers. This paper was (will be) published in Proceedings SPIE Europe Remote Sensing 2008 and is made available as an electronic reprint (preprint) with permission of SPIE. One print or electronic copy may be made for personal use only. Systematic or multiple reproduction, distribution to multiple locations via electronic or other means, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.

You can find the paper online at: Segmentation and tracking of ionospheric storm enhancements M. P. Foster and A. N. Evans ~1 MiB

Every now and then, I get really annoyed with MATLAB/Octave. The clumsiness of the language, and the way it makes you write so much throw away code are two particular annoyances, but the main thing that irks me is that it couldn’t be more different than my favourite language, Ruby.

So every now any then, I go on a foray, to see whether the tools I need are around yet, so I can move closer to dumping MATLAB, and bask in OO glory. I’d love to have the time to write everything I need from scratch, but unfortunately I don’t, so instead I end up hacking on things that interest me, and try not to embark on projects I don’t have time for.

To give some idea of what I need, here’s an example of typical workflow, with MATLAB:

• Create or load up some image data. This gives me matrices out.
• Do some processing, using image processing toolbox, including image morphology etc.
• Extract boundaries, and convert them into some handy spline representation for manipulation.
• Plot some image data with overlaid boundaries.
• etc…

Whilst I realise that nothing here is particularly complex, it’s fairly fast to do in MATLAB, and very slow to do in compiled languages. Things like ruby-gsl are starting to close the gap, in terms of scientific functionality for ruby, but there are still big gaps in other functionality. Gaps that seem to have been filled by python.

So maybe it’s time for me to look at python, and sage, which already has these things.

I’ll report back if anything happens!

It’s probably impossible to go to a conference without picking up some kind of idea of the current trends in the areas being discussed. IGARSS 2008 was certainly no exception. Here’s a list of what I think were probably the most trendy techniques in remote sensing. You can probably make a list like this just by graphing the frequency of word use:

• Classification
  • Support Vector Machines
  • Markov segmentation
• geostatistics - Last time I looked at wikipedia’s geostatistics entry it was very highly biased. Watch out!
• Kriging – less mentioned, but heavily used.
• Pan-sharpening
• mathematical morphology - Although interestingly, nothing as modern as attribute morphology.

And trendy instrument types:

• Hyperspectral-anything!
• Synthetic Aperture Radar SAR
• LiDAR

So there you have it… the must use remote sensing techniques for 2008! I’m sure I’ve missed some though.

I’ve just added a new repository to GitHub, called octave-scattered-image, into which I’ve started cleaning up, and pushing my Octave functions for processing scattered image data. Have a look, and feel free to contribute. My current plan is to distribute it as an octave package, but that’s currently untested.

Also, in other news, I’m off to Boston this week, for IGARSS 2008, the first conference of my PhD. The only disadvantage of this jaunt, is that I have to present a paper. I’ll put the sides up when I get back, provided I wasn’t laughed/heckled off the stage! Wish me luck!

Finally, after a nine-month wait, preceded by a year long slog, my paper has made it into print! I am, of course, very pleased :)

Uploaded with plasq’s Skitch!

The IEEE’s copyright policy states that I may host a copy of this paper, provided I state the following:

© 2008 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

So, with that out of the way, the final article may be downloaded here: An Evaluation of Interpolation Methods for Ionospheric TEC Mapping, M. P. Foster and A. N. Evans.

You can find the full abstract, and citation on IEEE Explore.