Recently, I’ve needed to write and use various Makefiles. I was disappointed to find the TM’s syntax highlighting lacking, and just not colourful enough, so I set about adding a few bits and pieces.
You can find the result on GitHub.
I was pleased to find that most of the official TM bundles are now there, so if you’re making small (or massive) tweaks and improvements, it’s should now be easier than ever to get your changes merged into TM proper.
Now, just just leave straight away, but I’m learning perl. I know I’ve only just started waxing lyrical about python, but bear with me, I’ve not got any plans to dump anything else: I just need to learn perl for work. (If you read on in this post, you’ll find some ruby code, too!)
So, armed with some of the ideas from comments on my last post, I’ve tried to build make use of some of the most trendy web resources to help me on my way. Ideally, I’d like to pick up on best practices as I go along.
So far though, I’m not having a huge amount of luck. I’ve worked through learning perl, and I’ve finished reading it feeling like I don’t know anything like the whole story. To be perfectly honest, I’m quite surprised at the complexity of what seems like a small language!
I’ve got a huge pile of books to help though, so I’m going to start digging through programming perl, and setting myself programming goals. My aim is to work out how to write simple modules and unit tests by the end of next week. I’d also like to find some rake/paver equivalent, and work out how to effectively use Devel::REPL (which stackoverflow helped me find.)
Soon I’ll have my automation / test safety net at the ready, so things can proceed a little faster.
Whilst casting around, I found Perl::Critic, a module (and front-end) for checking your code’s adherence to best practices (incidentally, there’s a great book called perl best practices – another one on the pile!).
I decided to write a TextMate command for the perl bundle to run code through perlcritic, since there isn’t one. After a little help from the bundle developers mailing list, I came up with the following:
To run this, you will need a new copy of the TextMate Support directory. I’d suggest using GetBundles. I’ve sent a patch to the perl bundle to the ML, but in the mean time, if you paste the above code into a bundle command, set the scope to source.perl, and trigger to ⇧⌘C, you should get it working fine.
Update: I’ve just started reading programming perl, and I wish I’d started with it! I’m already feeling like I’m learning more useful stuff than was contained in learning perl. I guess I’m not the target audience of that, but I wish I’d known last week!
After seeing lifehacker, I was inspired to add an adverb detector to TextMate’s markdown grammar, since that’s the format I use to write blog posts.
Far from being a couple of hours work, this was dead simple, and involved adding a couple of tiny bits to the grammar, here’s how to do it:
First, fire up TextMate’s Bundle Editor, press ⌃⌥⌘B. Then scroll to markdown, expand it, and find the Markdown Language grammar. This will be marked with a small grey L. Add the following to the repository section:
adverb-detector = {
name = 'invalid.adverb';
match = '(?!\b(supply|apply|family|only|prolly|folly|sully|rally|waverly|reply|early|probably)\b)\b(\w+ly)\b';
};
Now scroll to the section headed inline, and add:
{ include = '#adverb-detector'; },
To the patterns part. Finally, check you have a rule called Invalid in your TextMate theme, with the scope selector invalid. This rule is used to highlight a multitude of sins, so it’s probably already in your theme. Mine looks like this:
Like a lot of mac users, I get excited about MacHeist, and why not, it’s fun and a good bargain.
Since we’re snowed out of work, and a MacHeist Bundle release seems imminent, I decided to hack up a quick script to find out how wide the progress bar thing is, and use that to estimate the release time. Here’s what I’ve been up to for the last half hour or so:
I used BeautifulSoup with urllib2 and a tiny regular expression, and here’s the result:
I’m running it in a simple shell loop:
while true; do sleep 60; python heist_parse.py >> heist_countdown; done
Which is collecting the width every minute (that doesn’t seem too often).
Now that I’ve had this loop running for a while I’ve got 34 data points, so I can do a bit of simple linear regression, with scipy. Here’s a graph:
And here’s the code used to do the regression, and generate the plot.
It’s important to remember that this is based on the assumption that the bar is full up at 500 pixels wide. This may or may not be reasonable.
If it is though, I’d tentatively estimate that the bundle will be release at 3AM (GMT) on the 7th Feb 2009. You read it here first!
Update: Well, it looks like they cheated! Here’s a graph of the bar’s length with time:
As you can probably see, it was rounded up. Ahh well. So much for the estimate!
Abbreviations are ZSH feature I just stumbled across, whilst proselytising about GRML, a cool linux distro.
Here’s a snippet to show you the ultimate power they offer:
OK, so it may not make much sense as it stands, but bear with me. Suppose you want to get some information about the size of the directory tree you’re in. Now, you want to use something like du -sch, traditionally, you might have alias da='du -sch' in a config. file, but what if you want to edit it at the last minute. The abbreviation lets you do just that. If you type da,., the ,. trigger expands the abbreviation to give you du -sch on the command line, and then you can further edit the command, to add that extra x, or whatever, afterwards. This is obviously a somewhat contrived example, but the effort saving, productivity enhancing idea is still there, and it fits in well with the ZSH philosophy, or my version of it at least.
To see what I mean, look at the magic-space command. If you use:
bindkey ' ' magic-space
Then use history, or some other shell expansion, it will be fully expanded when you hit space. Did you miss the colon on the end of that scp command three lines ago? No problem, hit !-3␣, add it, and hit return. It feels similar to the abbreviation mechanism above, and is very powerful too.
It’s worth noting that the original sighting of this trick doesn’t seem to be compatible with the ‘magic-space’, since it rebinds space.
You can find all of this, and more, in my zshkit.
During the course of my PhD, I’ve spent quite a lot of time examining methods of interpolating scattered data. By scattered, I mean something that looks like this:
Or alternatively, like a very, very gappy image. This type of data is very common in geoscience, and so interpolating it so it looks more like this:
Is a common activity. As well as examining common methods of interpolating scattered data, such as cubic, linear and nearest neighbour methods, I’ve looked at natural neighbour interpolation, radial basis function interpolation, everyone’s favourite kriging and a less common methods known as adaptive normalised convolution (ANC).
In order to do meaningful comparisons, I needed working implementations of these, and so ended up writing my own versions of ANC, radial basis function interpolation and kriging. I also wrote a wrapper to Pavel Sakov’s natural neighbour interpolation.
I’ve just released these functions as a toolkit for Matlab, with a BSD license. You can find the download links on the toolkit’s homepage, which also gives the commands you need to clone the repository.
To build the toolkit, you’ll need a working build environment, with make, gcc and python 2.5 as well as a copy of Matlab. I’ve tested the kit on Debian/Ubuntu Linux with Matlab 2007a and Mac OS X (10.5). If you’re using a Mac, you’ll need XCode installed.
You’ll probably need to tell make where your mex binary is (MEX is Matlab’s external interface builder). To do this, run:
make MEX=/path/to/mex
You’ll find mex inside your Matlab installation, under the bin/ directory. On linux, it may be symlinked into /usr/local/bin.
Once make has run, you’ll find a directory called toolkit, which contains the interpolation toolkit. Copy this folder somewhere appropriate, and add it to your Matlab path, by running:
addpath('/path/to/toolkit')
Within Matlab. You can then run help toolkit, for some help and examples of how to test the methods.
I’d recommend using natural neighbour interpolation everywhere you would previously have used matlab’s griddata function. You can find more information on interpolation in my publications, which you can read and download here.
After reading this message on the textmate-users mailing list, I decided to have a go at writing a command to add comment chars to the ends of lines. Here’s what I ended up with:
To use it, create a command in TextMate’s bundle editor, then make a new command, and set the input and output dropdowns to look like this:
Finally, set up scope selections (e.g. source.python, source.shell, …), and a key binding if you want one.
Now that the IPython TextMate bundle has been around for a little while, and has struck a chord with the TextMate wielding, IPython hacking community, I decided that now might be a good time to sum up a few of my ideas for the future.
I’ve really been heartened by the response and enthusiasm of the IPython community, and of particular note is the work by Brain Granger. Brian is currently working on a Twisted based editor server for TextMate (and other editors too), with a block based protocol for robust communications. You can find some initial work in Launchpad, and grab it using bazaar using:
lp:~ellisonbg/ipython/textmate-server
Just be aware that this branch/link might not be around for long.
My top priority for the bundle with regards to this is the creation of a client library, probably also based on Twisted. It would be great to have this core library included in IPython, so it can be called on by other editors.
I think that the core of the bundle is fairly coherent (at least for me!), with the most useful commands appearing at the top of the list. I ideally though, all communications with IPython should be done through the server, so the debugging commands need work in that respect.
My top priority for this area is the implementation of a more user-friendly preferences / config. method. Here’s my initial stab at creating a nib. I now need to work out how best to hook it up with tm_dialog, to store the preferences. Hopefully recent changes to dialogs in TM will make this easier.
A lot of IPython users, myself included, are using IPython for scientific computing and number crunching. For that reason, decent support for scientific python libraries is essential.
Thanks to Ben Racine, we’ve got some decent snippets for scipy/numpy/pylab, and we’re adding more fairly regularly. Feel free to send me your ideas, and I’ll be sure to implement them if I can.
The bundle documentation is fairly complete, and since it’s combined with the README, it’s fairly easy to maintain. This is a good thing.
Demo-wise, so far there’s only the first screencast I made. I’m extremely surprised to see it has over 200 views, so I’m planning on making more short and sweet screencasts, which we can hopefully integrate with the bundle’s help.
Are very, very welcome. Please join the [google group](http://groups.google.com/group/ipython-tmbundle/ “ipython-tmbundle | Google Groups”), or if you’re a gihubber, you can fork the bundle, make your own changes and send me a pull request. Only with help can we make this bundle all it can be.
Sometimes, after seeing a few things, concepts collide (like icebergs, perhaps), and I come up with ideas I think are cool. Here’s one of those:
Refactoring with the MATLAB bundle from Matt Foster on Vimeo.
For those of you who prefer some code, say you have a big long ugly line of code. You know it’s bad, but it works and you don’t want to break it. Here’s an example, very, very loosely based on some Matlab code I saw at work:
The obvious thing to do, is pull out that evil range -pi:0.1:pi. With the new ‘Introduce variable’ command (bound to ⌃⇧C, by default), you can do this be selecting any on of them, and running the command.
You’ll see a dialogue box asking you for a name for you new variable, and after you press OK, you’ll see the following:
Much better. You can also repeat the process to further simplify the line, and the command will insert the new variable before the first instance it finds. Cool hey?!
This command is also general enough that is should work for most languages that use assignments based on =. For more info on this concept, see refactoring.com, which I shall also be scouring for more ideas.
I’m very pleased to announce that the IPython TextMate bundle is in a state where it can actually be used – you can probably consider it to be a pre-beta.
To celebrate this, I’ve made a simple screencast demoing some of the basic features.
IPython TextMate Bundle Demo from Matt Foster on Vimeo.
This screencast can also be downloaded, and I hope to give a quick overview of some of the more advanced features in a follow-up.
The main features of the bundle are:
import ipy_vimserver; ipy_vimserver.setup('session'), and you can connect.
The bundle can be downloaded from GitHub or installed via GetBundles. For installation help, see the README.
We’d love to hear your feedback, so please send comments, suggestions
and feedback to either the ipython-dev list (please prefix the subject
with [TextMate]), or the [ipython-tmbundle google group](http://groups.google.com/group/ipython-tmbundle/ “ipython-tmbundle |
Google Groups”) (just reply to
this email).
Thanks!
I’ve successfully committed Thomas Kjosmoen’s matlab bundle changes into the macromates SVN repository. His work is absolutely stellar, so I recommend you grab the bundle ASAP. I’ve also managed to keep the useful extensions I made to the previous incarnation intact.
As ever I’m eager to hear your feedback and bug reports. Either directly, or via the google group. The code is also available in GitHub for your forking pleasure.
I’ve been doing a little bit more tweaking with chaco, learning how to use groups and containers. The docs are great, if a bit incomplete, so I ended up looking at various examples.
I’ve updated my simple AM demo so that it now shows the frequency content of the signal (calculated on the fly, using an FFT).
The running program looks like this:
I’d still like to add zooming and area selections to the x-axis, and tidy the plots up a bit, but it’s looking good.
The code used to generate this coolness was:
Short and sweet.
This morning I was reading this tutorial on Chaco, a 2-D visualisation toolkit for Python (it’s part of the enthought python distribution), and I wanted a toy project to get a feel for it.
I decided to put together a very quick model of amplitude modulation, the kind of thing elec-eng students cover early in their degrees.
I came up with this:
It’s very heavily based on the code in the tutorial above, but it works well, and it looks and feels pretty cool.
I find this kind of app very useful for helping me get a feel for how something works, and using something like chaco in this way makes it very easy to throw quick models together to play with.
My next challenge to to find something useful to do with it!
Here’s the code:
I’m fairly sure that aesthetics play a large part in how people view your work – ugly correct results are probably viewed as being worse than pretty, but wrong results. Aesthetics must play a fairly big role in how people perceive your work.
I’ve just finished generating a lot of quiver plots for my thesis. A quiver plot is a graph of vectors, illustrating a flow field. My results typically show an image with motion vectors overlaid, all plotted with Matlab, a piece of software which is very heavily used in academia in the UK (and probably all over the world).
Matlab quiver plots are ugly, and all of my plotting got me wondering how other system’s quiver plots look. In particular, I’ve started playing with numpy and scipy, so I decided to do a quick comparison between matlab, and matplotlib, the python close of matlab’s plotting tools. The results are very interesting.
I generated a quick example motion field, and plotted it with quiver. Then I took screenshots. I used the Gimp on my Linux box, for the matlab example) and Skitch on my MacBook for the python example.
First, here’s the matlab example. This was generated on linux using Matlab 2007a:
Here’s the matplotlib python example, generated using the enthought python distribution. I used colour here because I could: matplotlib’s quiver supports it out of the box. Matlab’s doesn’t.
I think the difference between the results is abundantly clear. Matplotlib’s arrows look great, they’re shapely and well scaled. Matlab’s look like jaggedy lines in comparison. However, I think the main difference is down to anti-aliasing. The matplotlib’s result looks better because they’re smoother and look far more modern the 90s throwback that is matlab’s interface (on linux and Mac at least – I can’t speak for Windows).
These results have pretty much convinced me to try my next project using Python and scipy instead of Matlab. Not only is it completely free (EPD is free for academic use, and matplotlib, scipy and numpy are just plan free), the aesthetic is there too. It’s almost like python’s elegant simplicity ideals spilled over into their plotting engines.
Since reading this question on stackoverflow I’ve been intrigued by zsh, and looking at cool things it can do to make my life easier. I decided to jump on the top-n things bandwagon and publish a quick list of zsh bits I find really useful.
Here’s a quick list of my top five tips:
for loops:
for file (prefix*) $file:s/prefix/new_prefix/or if you want more than one command in the body, try:
for file (prefix*) {one; two; three}{0..10}
will give you a range of values, which you can use in a loop (for n ({0..10})), and
<0-10>lets you match ranges of digits in filename globs.
$variable:s/thing/other_thing/gBut alas, there’s no support for regular expressions in these.
**/*.pngwill find all
png files beneath your current directory. You can use this in a loop too:
for f (**/*.txt) {echo $f}Use zsh-lovers to find out more. There’s far too much to zsh for me to be able to list it all here. My first stop is usually this man page, and it’s usually all I need.
Bonus! Google for zshrc config files. Try github and dotfiles first.
