Python

Wanted: Testing Frameworks in the IPython Notebook

jiffyclub, , Leave a comment

As a tool when teaching unit testing it would be great to have a way to run nose or pytest in an IPython Notebook. For example, a %nosetests magic would do test collection in the Notebook namespace and do its usual run and reporting. Of course it’s always possible to write test functions and then just call them, but having a report that compiles everything in one place is nice. Plus it could look just like nosetests called from the command line.

Unfortunately for this idea these testing frameworks have for the most part been engineered for doing their test collection using the file system as the starting point. In a couple hours of fiddling I couldn’t figure out how to use either nose or pytest to do test collection in a notebook. I’m sure it could be done with enough hacking.

Just for kicks, though, I threw together a little IPython line magic that does its own limited test collection, running, and reporting. You can see it via nbviewer and grab it on GitHub. This magic only grabs functions that begin with “test” and the reporting doesn’t include tracebacks when there are failures or errors. But you do get the exceptions themselves.

(This whole experiment was inspired by a conversation on Twitter.)

Update: On the advice of @stefanvdwalt I put the %runtests magic into a .py file so it can be installed via %install_ext:

%install_ext https://raw.github.com/gist/4013594/runtests.py

Once that’s done you can run

%load_ext runtests
%runtests

to have it do its limited collection, running, and reporting as in this demo notebook.

Update 2: Someone has made this work with nose: https://github.com/taavi/ipython_nose!

Wanted: Testing Frameworks in the IPython Notebook

Open in nbviewer Firefox Add-on

jiffyclub, , Leave a comment

I’m happy to announce that there is now a Firefox add-on with the same functionality as the Open in nbviewer Chrome extension and bookmarklet. It can be installed via the Mozilla Add-ons directory. It will add an nbviewer icon to Firefox’s toolbar that, when clicked, will attempt to open your current page in nbviewer.

The code is available on GitHub.

Open in nbviewer Firefox Add-on

Open in nbviewer Bookmarklet

jiffyclub, , 6 Comments

Last week I made a Chrome extension that opens your current page in nbviewer. This isn’t ideal, though, since it is limited to people who use Chrome. Today I learned about bookmarklets, which are little bits of JavaScript that live in browser bookmarks (thanks Ethan). The Chrome extension is just a little bit of JS so it was easy to adapt into a bookmarklet. To use this bookmarklet just make a new bookmark (call it whatever you want) and copy this code into the URL field. Once you have the bookmark just click on it while you are on any page that can be loaded by nbviewer.

Open in nbviewer Bookmarklet

Open in nbviewer Chrome Extension

jiffyclub, , 7 Comments

I got tired of pasting URLs into nbviewer so I made a Chrome extension that will try to load your current page via nbviewer.

For example, if you are at https://gist.github.com/3778422 you can click the “Open in nbviewer” extension button and it will load http://nbviewer.ipython.org/3778422/ in a new tab. It also works for URLs ending in .ipynb.

You can download the extension from the Chrome Web Store and see the code on GitHub.

I took the extension icon from the nbviewer favicon, so thanks to them for that! Thanks also for making something as awesome as nbviewer, it’s getting so I couldn’t live without it!

Open in nbviewer Chrome Extension

Teaching with the IPython Notebook

jiffyclub, , 2 Comments

For a few months now I’ve been using the IPython Notebook as my primary teaching tool for Python topics. Within Software Carpentry we’re also switching over to using the Notebook for both in-person bootcamps and our online repository of material. Ethan White and I put together a post on this topic on the Software Carpentry blog and now Titus Brown has blogged with his own thoughts. We’ve put in a PyCon proposal for a panel on this topic in 2013.

The IPython developers have to be given a huge amount of credit for putting together the Notebook and the rest of IPython. The Notebook especially is quite a feat: a top notch research/engineering/teaching tool all in one. And they aren’t resting on their laurels, they have a ton of ideas in mind for the Notebook in the future, including a slide-show mode. I’m definitely looking forward to seeing what they’ve got!

As with many open source projects, the IPython developers struggle to find the time and funding to write their software. If any open source project is helping with your job or your research you can easily help by citing the software in your papers and in public on social media or blogs. This gives the developers more ammunition the next time they’re writing grants, so please make your support known!

Teaching with the IPython Notebook

SnakeViz 0.1 – A Python Profile Viewer

jiffyclub, , , 3 Comments

When profiling code it can be helpful to have a visualization of the profile to really show what areas of your program are taking the most time. The great RunSnakeRun application has been filling this roll for a while now but some of my colleagues have been turned off by the need to install wxPython. The web browsers we use every day are quickly becoming great visualization platforms so I thought it would be a nice project to make a Python profile viewer that works in the browser. Today, co-developer Erik and I are happy to announce the first official release of SnakeViz.

Example SnakeVis visualization.
An example SnakeViz visualization.

Continue reading “SnakeViz 0.1 – A Python Profile Viewer” →

SnakeViz 0.1 – A Python Profile Viewer

What’s In My Stack?

jiffyclub, , 1 Comment

This is a response to a post on the Software Carpentry blog: What’s In Your Stack?

My development setup is pretty simple:

  • Sublime Text 2 with a few plugins
  • git, GitHub, and bitbucket
  • Subversion and Trac (only at work)
  • IPython notebook
  • NumPy, SciPy, and matplotlib (mostly at work)
  • Chrome for debugging web stuff
  • bash and the usual shell utilities

It should go without saying that I’m using all of this to work on Python. And a little  JavaScript comes in for web apps.

What’s in your stack?

What’s In My Stack?

Function Timeout in Python

jiffyclubLeave a comment

In some code I’m working on I want to put a 10 second timeout on a function call. If the function doesn’t return in 10 seconds I just want to terminate the function and return None. Here’s the code I came up with using multiprocessing.Pool:

import multiprocessing as mp

pool = mp.Pool(1, maxtasksperchild=1)
result = pool.apply_async(function, (arg1,))
pool.close()

try:
    s = result.get(10)
except mp.TimeoutError:
    pool.terminate()
    return None

This seems to work but I’m curious if there’s a standard Python pattern for this sort of thing?

Function Timeout in Python

Testing Python with the mock library

jiffyclubLeave a comment

A bit ago I was writing some unit tests for a complicated code at work and when I looked at all of the crazy things I’d need to instantiate just to run this one little unit I despaired. Looking at the code it seemed like the usage of the input objects was pretty simple though, maybe, I thought, I could just write fake little stub objects to stand in for them and use those…

But then it occurred to me, that’s what mock objects are for! Mock objects are useful in unit testing as stand-ins for other functions or objects. You might use mock objects when, for example, the real thing is expensive to create, requires online resources that might sometimes be offline, or when you just want to do really fine-grained testing. With mock objects you can easily control exactly what they do and then test whether your code-under-test used the mock object correctly.

I picked the mock library as it seemed fairly straightforward and the documentation is copious, though dense. I also found the examples to not be very helpful. There are other options. All-in-all I’d recommend mock because once I got the hang of the usage it was really simple to use.

I am apparently the first person in my group to ever try to use mock objects so today I gave a lesson both on the concept of mocks and the usage of the mock library. My notebook is available:

Testing Python with the mock library