An astronomer friend of mine once compared his experience programming to this clip from Fawlty Towers:
I know a lot of scientists, even programmers, who feel the same way, and I’ve felt like that too. But think about it: if programming always caused everyone that level of frustration no one would ever write software. But clearly people are. I code for a good chunk of just about every day. Software developers don’t have super powers that help them survive the mental strain of programming. Rather, they’ve adopted techniques and tools to make things manageable.
Worried about deleting or overwriting important code? Put it in version control on a hosting service and have access to the entire history of your project.
Don’t want to break working code while you add a new feature? Put it in version control and work in a branch.
Working with a big team on a code or paper? Put it in version control and let the version control system keep track of who has change what.
Can’t keep track of the bugs in your code you can’t fix right now? Use an issue tracker like the ones built into most code hosting sites.
Don’t know if your code’s results are correct? Afraid your changes will break code but you don’t know how to tell? Use unit and regression testing to ensure correctness after every change.
Copying and pasting code into many places and tired of keeping track of it all? Learn to write short, modularized code chunks and reuse them again and again from just one copy of the code.
Sick of putting millions of print statements into your code to track down bugs? Learn to use a symbolic debugger so you can inspect the state of your programs as they run.
Sounds great, you say, but how do you learn to do all this stuff? There are people who want to help. Check out the list of upcoming Software Carpentry boot camps for one in your area. If there isn’t one try to get a group of people from your department or lab who would like a local boot camp and get in touch asking for something to be organized at your institution. Software Carpentry has a large network of instructors and does boot camps all over the world.
I can personally testify to how much worry and stress I avoid by testing my code and putting it in version control. No more manually tracking a hundred versions and wondering which is most recent or most correct. No more updating code and seeing the results change and wondering whether that’s good or bad. No more worrying that I’ll accidentally type rm * and destroy weeks or months of work.
There’s a better way.
Pen and Pants 
I was that astronomer about a year ago! I love that video. Python’s community and github’s ease has made my coding-life much better — and I am using most of your suggestions above.
My frustrations, and the reasons I thought that video captured them, stem from my lack of knowledge in the art of programming (e.g., data structures and algorithms) — how to best implement what I need quickly. Like many astronomers, I’ve never taken a programming course but I code almost everyday.
So, what do you suggest after we’ve checked off that list?
Gaining expertise in programming is definitely another aspect of this, and something harder to teach. It’s really no different from developing expertise in any subject, though. Just as you learn which math and which physics to apply in certain situations, you start to learn the computational methods that are best suited to different problems.
One thing I think it’s important to stress, though, is that there’s nothing wrong with doing something the simplest way you know how. Slow, correct code is better than fast, wrong code. There’s an axiom I learned from Greg Wilson that I like to spout in workshops: “make it right, then make it fast”.
To grow as a programmer I think it’s important to try to be an active learner, try new things, and keep your eyes open. I’ve picked things up at conferences like SciPy, by reading others’ code, in books, and even by wandering aimlessly through online documentation (this is actually surprisingly effective). One really useful thing some of us do here at STSCI is try to meet every other week and present or review code to try to get some knowledge transfer. Exposure is critical, you can really only learn from examples. (The IPython Notebook is, I think, really starting to help the proliferation of online Python examples. I see a new one almost every day.)
[…] ← Previous […]