Announcing ipythonblocks.org

Way back…

About a year ago, inspired by Greg Wilson, I wrote ipythonblocks as a fun way for students (and anyone else!) to practice writing Python with immediate, step-by-step, visual feedback about what their code is doing. When I’ve taught using ipythonblocks it has always been a hit—people love making things they can see. And after making things people love to share them.

Sometime last year Tracy Teal suggested I make a site where students could post their work from ipythonblocks, share it, and even grab the work of others to remix. Today I’m happy to announce that that site is live: ipythonblocks.org.

How it works

With the latest release of ipythonblocks students can use post_to_web and from_web methods to interact with ipythonblocks.org. post_to_web can include code cells from the notebook so the creation process can be shared, not just the final result. from_web can pull a grid from ipythonblocks.org for a student to remix locally. See this notebook for a demonstration.

Thank you

There are many people to thank for helping to make ipythonblocks.org possible. Thanks to Tracy Teal for the original idea, thanks to Rackspace and Jesse Noller for providing hosting, and thanks to Kyle Kelley for helping with ops and deployment.  Most of all, thanks to my family for putting up with me working at a startup and taking on projects.

Announcing ipythonblocks.org

Broadcasting IPython Notebooks

A useful feature of the IPython Notebook is that you can set the server to broadcast so that others on your local network can see the server and your notebooks. This is especially nice as a teacher so that students can load your notebooks as you work, copy text out of them, and see them in their entirety instead of just what you have on screen. Here’s the outline of what to do, with detailed instructions below:

  1. Create an IPython profile with a password for the Notebook server.
  2. Figure out your IP address on the local network.
  3. Launch IPython in broadcast + read-only mode using your new profile.
  4. Have your students navigate to your Notebook server.

Continue reading “Broadcasting IPython Notebooks”

Broadcasting IPython Notebooks

Install Scientific Python on Mac OS X

These instructions detail how I install the scientific Python stack on my Mac. You can always check the Install Python page for other installation options.

I’m running the latest OS X Mountain Lion (10.8) but I think these instructions should work back to Snow Leopard (10.6). These instructions differ from my previous set primarily in that I now use Homebrew to install NumPy, SciPy, and matplotlib. I do this because Homebrew makes it easier to compile these with non-standard options that work around an issue with SciPy on OS X.

I’ll show how I install Python and the basic scientific Python stack:

If you need other libraries they can most likely be installed via pip and any dependencies can probably be installed via Homebrew.

Command Line Tools

The first order of business is to install the Apple command line tools. These include important things like development headers, gcc, and git. Head over to developer.apple.com/downloads, register for a free account, and download (then install) the latest “Command Line Tools for Xcode” for your version of OS X.

If you’ve already installed Xcode on Lion or Mountain Lion then you can install the command line tools from the preferences. If you’ve installed Xcode on Snow Leopard then you already have the command line tools.

Homebrew

Homebrew is my favorite package manager for OS X. It builds packages from source, intelligently re-uses libraries that are already part of OS X, and encourages best practices like installing Python packages with pip.

To install Homebrew paste the following in a terminal:

ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"

The brew command and any executables it installs will go in the directory /usr/bin/local so you want to make sure that goes at the front of your system’s PATH. As long as you’re at it, you can also add the directory where Python scripts get installed. Add the following line to your .profile, .bash_profile, or .bashrc file:

export PATH=/usr/local/bin:/usr/local/share/python:$PATH

At this point you should close your terminal and open a new one so that this PATH setting is in effect for the rest of the installation.

Python

Now you can use brew to install Python:

brew install python

Afterwards you should be able to run the commands

which python
which pip

and see

/usr/local/bin/python
/usr/local/bin/pip

for each, respectively. (It’s also possible to install Python 3 using Homebrew: brew install python3.)

NumPy

It is possible to use pip to install NumPy, but I use a Homebrew recipe so I avoid some problems with SciPy. The recipe isn’t included in stock Homebrew though, it requires “tapping” two other sources of Homebrew formula:

brew tap homebrew/science
brew tap samueljohn/python

You can learn more about these at their respective repositories:

With those repos tapped you can almost install NumPy, but first you’ll have
to use pip to install nose:

pip install nose

I compile NumPy against OpenBLAS to avoid a SciPy issue. Compiling OpenBLAS requires gfortran, which you can get via Homebrew:

brew install gfortran
brew install numpy --with-openblas

SciPy

And then you’re ready for SciPy:

brew install scipy --with-openblas

matplotlib

matplotlib generally installs just fine via pip but the custom Homebrew formula takes care of installing optional dependencies too:

brew install matplotlib

IPython

You’ll want Notebook support with IPython and that requires some extra dependencies, including ZeroMQ via brew:

brew install zeromq
pip install jinja2
pip install tornado
pip install pyzmq
pip install ipython

pandas

Pandas should install via pip:

pip install pandas

Testing It Out

The most basic test you can do to make sure everything worked is open up an IPython session and type in the following:

import numpy
import scipy
import matplotlib
import pandas

If there are no errors then you’re ready to get started! Congratulations and enjoy!

Install Scientific Python on Mac OS X

ipythonblocks – A Visual Tool for Practicing Python

Learning to program and learning the basics of control flow can be tricky business for novices. I wanted to make something that provided immediate, visual feedback to students as they practice things like for loops and if statements so they can see precisely what their code is (or isn’t) doing. So I wrote ipythonblocks.

The IPython Notebook makes it possible to display rich representations of Python objects using HTML (among other things). That allowed me to make a Python object whose representation in the Notebook is a colored table. Students can index into the table to change the color properties of individual table cells and then immediately display their changes.

With ipythonblocks instructors can give coding problems like ‘turn every block in the third column red’ or ‘turn every blue block green’ and by displaying their blocks students can see right away whether their code is having the desired effect.

Check out the demo notebook to see ipythonblocks in action.

ipythonblocks – A Visual Tool for Practicing Python

Wanted: Testing Frameworks in the IPython Notebook

As a tool when teaching unit testing it would be great to have a way to run nose or pytest in an IPython Notebook. For example, a %nosetests magic would do test collection in the Notebook namespace and do its usual run and reporting. Of course it’s always possible to write test functions and then just call them, but having a report that compiles everything in one place is nice. Plus it could look just like nosetests called from the command line.

Unfortunately for this idea these testing frameworks have for the most part been engineered for doing their test collection using the file system as the starting point. In a couple hours of fiddling I couldn’t figure out how to use either nose or pytest to do test collection in a notebook. I’m sure it could be done with enough hacking.

Just for kicks, though, I threw together a little IPython line magic that does its own limited test collection, running, and reporting. You can see it via nbviewer and grab it on GitHub. This magic only grabs functions that begin with “test” and the reporting doesn’t include tracebacks when there are failures or errors. But you do get the exceptions themselves.

(This whole experiment was inspired by a conversation on Twitter.)

Update: On the advice of @stefanvdwalt I put the %runtests magic into a .py file so it can be installed via %install_ext:

%install_ext https://raw.github.com/gist/4013594/runtests.py

Once that’s done you can run

%load_ext runtests
%runtests

to have it do its limited collection, running, and reporting as in this demo notebook.

Update 2: Someone has made this work with nose: https://github.com/taavi/ipython_nose!

Wanted: Testing Frameworks in the IPython Notebook

Open in nbviewer Firefox Add-on

I’m happy to announce that there is now a Firefox add-on with the same functionality as the Open in nbviewer Chrome extension and bookmarklet. It can be installed via the Mozilla Add-ons directory. It will add an nbviewer icon to Firefox’s toolbar that, when clicked, will attempt to open your current page in nbviewer.

The code is available on GitHub.

Open in nbviewer Firefox Add-on

Open in nbviewer Bookmarklet

Last week I made a Chrome extension that opens your current page in nbviewer. This isn’t ideal, though, since it is limited to people who use Chrome. Today I learned about bookmarklets, which are little bits of JavaScript that live in browser bookmarks (thanks Ethan). The Chrome extension is just a little bit of JS so it was easy to adapt into a bookmarklet. To use this bookmarklet just make a new bookmark (call it whatever you want) and copy this code into the URL field. Once you have the bookmark just click on it while you are on any page that can be loaded by nbviewer.

Open in nbviewer Bookmarklet

Open in nbviewer Chrome Extension

I got tired of pasting URLs into nbviewer so I made a Chrome extension that will try to load your current page via nbviewer.

For example, if you are at https://gist.github.com/3778422 you can click the “Open in nbviewer” extension button and it will load http://nbviewer.ipython.org/3778422/ in a new tab. It also works for URLs ending in .ipynb.

You can download the extension from the Chrome Web Store and see the code on GitHub.

I took the extension icon from the nbviewer favicon, so thanks to them for that! Thanks also for making something as awesome as nbviewer, it’s getting so I couldn’t live without it!

Open in nbviewer Chrome Extension

IPython Notebook Viewer

The nice folks over at IPython have put together a nice viewer for IPython notebooks stored as GitHub gists: nbviewer.ipython.org/. Just enter a gist URL or number. It doesn’t actually store notebooks, just renders notebooks stored as gists.

I’ve had some notebooks as gists for a while and now I can make pretty links to them:

IPython Notebook Viewer

Profiling Python

Profiling a program is a way to take a detailed look at the execution time of individual pieces of the program. When you want to optimize code it’s important to use a profiler first so that you know where your optimization effort would be most beneficial.

I recently demonstrated profiling Python code using the built-in tools, line_profiler, and IPython’s “magic” hooks into those tools. RunSnakeRun also made an appearance.

As usual I used the IPython notebook and you can get the .ipynb file and the PDF, or see it directly.

More info on using the built-in profiler and pstats modules is available via the Python Module of the Week: http://www.doughellmann.com/PyMOTW/profile/index.html.

Profiling Python