IPython HTML Notebook

Until recently I had never been a fan of IPython but with their HTML notebook they’ve finally won me over. What I like about this tool is that it makes it easy to go back and forth between interactive prototyping and a script. Being able to continuously edit and re-run code in an interactive session is a powerful tool.

The notebook also makes a great tutorial and demo tool. Here’s a PDF of my session developing a Python replacement for IDL’s GAUSSFIT function.

Installation

The IPython notebook requires a few extra packages but if you have a setup like me it’s easy to get everything installed:

brew install zeromq
pip install pyzmq
pip install tornado
pip install ipython

After doing this you may also want to locally install MathJax for JavaScript equation rendering:

from IPython.external.mathjax import install_mathjax
install_mathjax()

To launch the notebook from whatever directory you want to work in:

ipython notebook

This will launch the IPython notebook dashboard in your default browser, from which you can make new notebooks or resume working on existing ones.

See the docs for all you can do with the notebook, and enjoy!

IPython HTML Notebook

IDL’s GAUSSFIT in Python

A colleague recently asked for help getting the functionality of IDL’s GAUSSFIT function working in Python. This was a perfect opportunity to use the handy curve_fit function from SciPy. Here’s the code:

import numpy as np
from scipy.optimize import curve_fit

xdata, ydata = np.loadtxt('focus_output.dat', unpack=True)

def fit_func(x, a0, a1, a2, a3, a4, a5):
    z = (x - a1) / a2
    y = a0 * np.exp(-z**2 / a2) + a3 + a4 * x + a5 * x**2
    return y

parameters, covariance = curve_fit(fit_func, xdata, ydata)

The file focus_output.dat just contains some data in two columns of numbers. For more info on loadtxt see my post on reading text tables. fit_func defines the function we want to fit to the data. In this case it is a Gaussian plus a quadratic, the same as used in GAUSSFIT when NTERMS=6. Now, to plot the results:

import matplotlib.pyplot as plt

fitdata = fit_func(xdata, *parameters)

fig = plt.figure(figsize=(6,4), frameon=False)
ax = fig.add_axes([0, 0, 1, 1], axisbg='k')

ax.plot(xdata, ydata, 'c-', xdata, fitdata, 'm-', linewidth=3)

ax.set_ylim(0.38, 1.02)

fig.savefig('gauss_fit_demo.png')

Data and Fitted Function
Cyan shows the original data, magenta shows the function fit with parameters returned by curve_fit.
IDL’s GAUSSFIT in Python

Reading Text Tables with Python

Reading tables is a pretty common thing to do and there are a number of ways to read tables besides writing a read function yourself. That’s not to say these are magic bullets. Every table is different and can have its own eccentricities. If you find yourself reading the same type of quirky file over and over again it could be worth your effort to write your own reader that does things just the way you like. That said, here are some other options.

Continue reading “Reading Text Tables with Python”

Reading Text Tables with Python

Install Python, NumPy, SciPy, and matplotlib on Mac OS X – Double Click

Update: These instructions are over a year old, though they may still work for you. See the “Install Python” page for the most up-to-date instructions.

I’ve already written a post about installing Python, NumPy, SciPy, and matplotlib on Lion, but it involves a lot of working at the command line, modifying your .bash_profile and dealing with compiler problems. That’s what I’ll call the compile-it-yourself (CIY) method. What I’ll describe below I’ll call the “double click” method.

I personally use the CIY method because it allows me to very easily control what’s installed. With Homebrew and pip I can uninstall and upgrade different things at will, or choose to install bleeding-edge versions. But it’s more hassle than everyone wants and there’s now an easier way using double-click installers.

Until recently the CIY was the only way to get everything working on Lion but now the developers of NumPy, SciPy, and matplotlib have all caught up and it’s possible to just download and double-click on a few DMG files to get a basic scientific Python installation working. Once you get to know Python, though, you will undoubtedly want to install some other packages and when that time comes I suggest you use pip.

Continue reading “Install Python, NumPy, SciPy, and matplotlib on Mac OS X – Double Click”

Install Python, NumPy, SciPy, and matplotlib on Mac OS X – Double Click

Teaching Python at Software Carpentry – Toronto, February 2012

Last week I had the privilege of teaching Python (and generally helping out) at a Software Carpentry bootcamp at the University of Toronto. Mike Fletcher and I worked together to make a lesson plan targeted at students completely new to Python. Overall it was a great experience and I had a lot of fun meeting some other Pythonistas. Here I’d like to talk about some lessons learned from our teaching. (Update: Mike has written up his thoughts as well.)

The overall narrative of our lessons was that they were trying to read a CSV data file. We built up the very basics (data types, if, for) up through functions and modules in a total time of about 5 hours. We tried to alternate between a few minutes of lecturing and a few minutes of exercises that students typed and ran on their own computers using template files we had given them. I think we built a good narrative and I particularly liked how we segued into writing functions, writing modules, and re-using code. However, for the time available and the students we had I think we could make improvements.

Our students were mostly science graduate students. They were largely not experienced programmers and none of them (as far as I know) had experience with Python. We had just a few hours with them, though they will be continuing with online lessons. Our particular failing in this situation was that the complexity of our exercises quickly outpaced the experience of our students, making them confused and slowing everything down. When I do this again I will probably iterate on the same content but update the exercise templates to be much more filled in, requiring the students to fill in only a couple of key lines. This will hopefully allow us to move through more content in less time without sacrificing much in terms of what the students practice.

Teaching in such a short amount of time is a challenge. How much time do you spend selling Python, all the great modules in the standard library, and the great third-party packages that make Python an excellent choice for scientific computing? If you do some really compelling demos maybe the students will be more inspired to continue using Python on their own and it won’t matter that you’ve sacrificed time you could have spent teaching them the basics. How much time do you spend teaching them about installing and managing Python and third-party packages? It’s hard to make a case that you should teach this at all in the first class but it’s bound to come up for everyone when they stumble across cool-package-X.

I definitely feel some initial demos are in order, just enough to say, “Look, you can do all this great stuff with Python!” Then you’ve got to start teaching something and keep it moving as you do.

The Udacity Teaching Model

I’m currently taking CS373 at Udacity (they also offer CS101). They have a pretty cool teaching model with a few minutes of instruction followed by short quizzes so that you get immediate feedback on how you’re doing. The quizzes even involve programming: you’re given a nice text editor (implemented in JavaScript, I believe) that does nice syntax highlighting and automatic indenting. It’s usually pre-populated with some starting code and test variables. You write your code and click “Run” to see the output. When it’s satisfactory you click “Submit” to have your output scored against what is expected. If you like you can write your code in your own editor/environment and copy it back when you’re ready to submit.

The Udacity classes are not live but I can imagine a similar tool being useful in a live classroom setting because it gives the teacher the chance to see what students are writing and what output they’re getting. The code is run on the server side so it can be consistent for everyone. And though it’s not an ideal development environment, it does help for students who might not have Python installed on their computers. One drawback is that I don’t know how it would work when it comes to teaching students to read and write files.

Teaching Python at Software Carpentry – Toronto, February 2012

Install Python, NumPy, SciPy, and matplotlib on Mac OS X

Update: These instructions are over a year old, though they may still work for you. See the “Install Python” page for the most recent instructions.

A bit ago a friend and I both had fresh Mac OS X Lion installs so I helped him set up his computers with a scientific Python setup and did mine at the same time.

These instructions are for Lion but should work on Snow Leopard or Mountain Lion without much trouble. On Snow Leopard you won’t install Xcode via the App Store, you’ll have to download it from Apple.

After I’d helped my friend I found this blog post describing a procedure pretty much the same as below.

Update: If doing all the stuff below doesn’t seem like your cup of tea, it’s also possible to install Python, NumPy, SciPy, and matplotlib using double-click binary installers (resulting in a much less flexible installation), see this post to learn how.

Continue reading “Install Python, NumPy, SciPy, and matplotlib on Mac OS X”

Install Python, NumPy, SciPy, and matplotlib on Mac OS X

Essential Everyday Python Links

This week I’ll be teaching beginning Python at a Software Carpentry bootcamp in Toronto and I’m planning to leave the students with my most frequently visited Python links. This is strictly core Python, no third-party packages.

What are your most visited core Python references?

Updated:

  • Prasanth suggests Doug Hellmann’s  Python Module of the Week. Doug covers most of the modules in the Python standard library with nice descriptions and examples.
Essential Everyday Python Links