Code Completion Considered Harmful

At least, for me. Here’s why:

I’m working on a project to interact with a complex COM-based Windows DLL. There is some decent (not great) documentation of the deeply-nested object model used by this DLL. So, to get started, I decided to prototype in VB.Net. I figured that the IntelliSense in the Visual Studio IDE would help me get the nested properties right.

A few hours into it, I realize I’m not thinking anymore. I’m not even trying. Instead, I’m typing something that might be close, then scrolling through the list of properties. I pick one, type a dot (.), then repeat. It’s “programming by poking around.” And it’s wasting my time. (Sigh.)

Here’s the truth: I don’t really know what I’m doing with this DLL’s object model (yet). I have a vague idea, but rather than read the documentation, I jumped into it. “Git ‘er done!” After all, “it’s just a prototype,” right?

Wrong. It’s a learning experience. And the best way to learn to ride a bike is to throw away the training wheels.

Four years ago, I was forced to throw away my statically-typed security blanket and code-completion training wheels. I kicked and screamed like a baby. But, I started to mature as a programmer. I became better, by using tools that were worse*. It might stand to reason that I’m better now in VB.Net after learning Python…if I didn’t have to constantly fight the warm, cozy lull of the IDE as it tries to put my brain to sleep.

Finally, a question, esp. for Managers. Which would you rather have: better tools or better programmers? (Hint: the latter will save you a lot of money in the long run.)

*Incidentally, this works with music, too. Get good on a piece-of-junk guitar, then trade up for a Strat.

Upgrading to Jenkins on Ubuntu Natty

I just upgraded Ubuntu to Natty Narwhal 11.04 (from Maverick Meerkat 10.10). Yes, I know I’m behind a release or two, but that seems a bit safer. Esp. when I’ve got deadlines to meet!

I had installed Hudson earlier. Thankfully, Ubuntu put Jenkins into the official repositories, so I decided to go with that. (Besides, Jenkins seems to have most of the developer support these days.)

Unfortunately, Jenkins failed to start. When I looked at the log file in /var/log/jenkins/jenkins.log, here’s what I saw:

jenkins start-stop-daemon: unable to stat /usr/lib/jvm/default-java/bin/java (No such file or directory)

Strange. Well, I saw somewhere that Natty switched completely to OpenJDK, away from any Sun-related JDK. (Blame it on Oracle’s takeover of Sun.*)

To fix it, I simply created a symlink from the existing OpenJDK to this “default-java”, like so:

$ cd /usr/lib/jvm/
$ sudo ln -s java-6-openjdk default-java

Now it works fine.

*Natty also switched from OpenOffice.org to LibreOffice. No warning, just “switcheroo!” I understand it, but still an unexpected surprise. And I don’t generally like unexpected changes of this magnitude.

Stackoverflow Snooping?

For a while now, I’ve had trouble with the technical answer site stackoverflow.com. This same trouble has been documented here:

http://meta.stackoverflow.com/questions/74796/failed-to-load-source-for-http-stackoverflow-com-posts-1234567-ivc-3c10

The answer that Jeff Atwood (founder of StackOverflow?) gives is:

That link returns a 204 no content by design, so I suspect your work firewall is configured incorrectly.

The obvious question is, “Why is that by design?” Or, “What is the design goal here?”

Tim Stone comments,

Note: You’re supposed to be getting a 204 No Content response, it’s just a dummy request for the view counter mechanism.

Should this bother me? Probably not. I understand that StackOverflow rates its questions by how many views are made of them. This “view counter mechanism” is an integral part of StackOverflow.

And yet…it does bother me. I’m not concerned about privacy, since I don’t browse the bad parts of the web and I don’t go looking for trouble. But this feels like a “web beacon” from the long-lost days of the Mozilla Web Browser. I don’t want to be counted, which is one reason I haven’t signed up for a StackOverflow account.

I’ve found a way around this, but I don’t want to post it online. They guys at StackOverflow really like their counting mechanism, so let’s let them keep it. If you feel bothered by the counter, though, let me know by posting a comment and I’ll share what I’ve found out with you, via email. Just don’t go posting it on the Internet, OK?

Environment variables in Python, Django, Windows and Hudson

I was having trouble with one of my Hudson builds. It was failing on Windows, but working fine on Ubuntu.

First, some background:

This build is a “free-style software project” written in Python, using the Django web framework. Python isn’t a compiled language, so there isn’t any real compilation going on. I just have Hudson doing a clean check-out (from Mercurial) and then running a prepared Python script.

I chose to use Hudson for this, so that I could verify that it worked on Ubuntu and Windows. Also, Hudson shows the history of builds, along with a nice trend graph that shows successes and failures. It’s handy to know that my “optimizations” actually do improve running time. (Like the latest, which cut the overall process down from 2.5 hours to 25 minutes.) Also, Hudson keeps the stdout from my Python script for each run, so I can do investigative log-searching after the fact, if I need to.

But, in short, Hudson gives me “fire and forget” for this long-running process. That means I can “hg push” and then go eat dinner, without babysitting the process.

The problem:

I made a change, then fired it off to the repository. Then, my Windows build started breaking. The one running on Ubuntu kept working just fine.

It’s a bit strange to explain, but here’s what was happening:

C:\path\to\hudson\jobs\thisproject\>python manage.py shell
>>> import settings
>>> settings.DATABASES['default']['NAME']
'/path/to/correct/db.sqlite3'
>>> from django.conf import settings as conf_settings
>>> conf_settings.DATABASES['default']['NAME']
'/path/to/wrong/oldversion/db.sqlite3'

I couldn’t figure out what was happening for a while. It seemed that “from django.conf import settings” was importing the wrong settings.py file!

Long story short, it turned out that my Windows PYTHONPATH environment variable was getting in the way. I had set it globally, using the “My Computer > Properties” dialog, so that every Command Prompt had that set. It was set to another version of the same project, at a different path.

So, I deleted that global PYTHONPATH setting. Then–and very important–I had to start a new Command Prompt. After that, it worked as expected.

Fixing Hudson

So, I kicked off a manual build via the Hudson Web interface. Surprise! It failed in exactly the same way as before. What gives?

Well, I was running Hudson via a Command Prompt invocation. (See it, now?) Yep, that Command Prompt shell had the old PYTHONPATH environment variable value. And every process that ran inside it also got that value passed to it. So, Hudson got it, then passed it off to Python.

I shut down Hudson, then restarted it. This time it worked.

Moral of the Story

Environment variables are very important. If you change them, you need to restart processes that are affected by them. This includes anything Python-related whenever you change the PYTHONPATH.

Learn Python the Hard Way

Slashdot highlighted this recently. It looks excellent. http://learnpythonthehardway.org/ (Oh, and you can get it free online, if you don’t want to buy the book. But there’s a nice hard-back for those who enjoy that, too.)

Being Open

First, a definition:

“Being Open” is the conscious commitment to giving away source code, for the benefit of others, to promote collaboration, and to spur oneself on toward best practices.

Being Open is not the answer for everything. There are some questions that cannot be answered (or may not even be asked) in the wide-open global community. There are other questions that are simply not addressed by technology, such as “Teacher, what good thing must I do to get eternal life?“

But what about these questions:

• “What projects are you working on?” (Asked by supporters and other partners.)
• “How can I get a copy of that?” (Asked by others with similar project needs.)
• And maybe even, “Where could I serve?” (Asked by potential volunteers and future full-timers.)

Being Open does have a cost. We must structure our software projects so that they do not contain sensitive data or security account information. We must document it somewhat*, so that others can use it. Managers must support Being Open, too.

*(Documentation, honestly, isn’t nearly as important as actual code. Documentation can be added later. The important thing is to get the source code into the open.)

But Being Closed also has a cost. Being Closed forces others to reinvent the wheel (again and again). It forestalls the contributions of remote volunteers. It allows us to write bad code, because no one else will ever see it. And it makes us close in on ourselves, not seek out collaboration partners, and eventually become paranoid. And who wants that?

Let’s commit to Being Open.

Microsoft Support Silliness

This is a counter-example for openness. Please do not follow Microsoft’s example here.

Microsoft Hides Information

I needed to find information related to VB6, for a project. Google led me to this page:

http://support.microsoft.com/kb/187234

Here’s what I got, using Firefox on Ubuntu:

Non-friendly message for Linux browsers

What?! Microsoft is hiding information from me, because I’m using linux? Yes, Dorothy, you’re not in Kansas anymore. (You’re in Redmond!)

How can they do this?

Now, I know what you’re thinking: How can they do this? Does Microsoft really know what OS I’m using? Is there a legitimate reason for hiding information? (I’ll answer that last one at the end.)

Well, Microsoft doesn’t really know what OS you’re using. However, your browser does tell every server (a) what browser it is and (b) what type of system it’s running on. This information is encoded in the “User-Agent” HTTP Heades that is sent with every web request.

To see what your browser is sending, you need to use a tool that will tell you. Fortunately, Firefox has a neat add-on called “Live HTTP Headers.” When you run this, it records subsequent headers that your browser sends (and receives). This is how I found out what headers are sent on Linux and Windows XP:

On Linux:

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.17) Gecko/20110422 Ubuntu/10.04 (lucid) Firefox/3.6.17

On Windows XP:

Mozilla/5.0 (Windows NT 5.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1

Notice the bit about the Operating System in each one? Well, Microsoft can use that to determine whether they’ll show you all their stuff or not.

Singing it their way

So, let’s tell Microsoft what they want to hear. To do this, grab the “Modify Headers” add-on for Firefox. Then, simply setup a new value for the “User-Agent” value. Give it the Windows XP value, and voilà, you can see all the content!

How should Microsoft use this information?

There is nothing wrong with the technique Microsoft is using to figure out which OS you’re on. This is actually a fairly common technique called “browser sniffing”. However, there is something wrong with what they’re doing with that information.

The Web is an open standard. It is intended to make it easy to share information. Browser sniffing is used most commonly to (a) warn people if your site is arcane and won’t render well in modern browsers or (b) to present a browser-specific format that will render well in cranky browsers (like Internet Exploder Explorer).

This is the first instance I’ve seen of censorship based on operating system. (Now, I’ve personally built sites that didn’t render in Firefox…back when I loved IE and worked in an all-MS shop with complete control over our users. But that was browser-based censorship, sort-of. Not OS-based censorship.) But let’s give Microsoft the benefit of the doubt. (Bear with me, here.) Assuming it was well-intended, it is very poorly executed. A better approach would be to use color or some other means to indicate content that is not specific to my OS. Simply hiding this info is annoying, at best.

And now you know how to view it, regardless.

Why we love Django

We currently use (and love) Django, for these reasons:

• Very low learning curve.
• Good separation of concerns, using the Model-View-Controller (or, Model-Template-View) pattern
• Very clean templating language that keeps code out of the template
• Built-in ORM that supports PostgreSQL, MySQL and SQLite (with others available through 3rd parties)
• It builds your database table structure based on your python models
• Built-in Admin site that rivals phpMySQL (but works for all of the above databases)
• Almost never need to touch SQL
• Built-in unit tests of the core
• Easy to code unit and functional tests of your own stuff
• Built-in user permissions that are easily extensible
• Built-in user authentication
• Develop on your local box, deploy to a different OS or Database (if you’re careful)
• Free and open-source
• Great documentation and tutorials
• Works on Windows, Linux, Mac, etc.
• Uses python (“the world’s best programming language,” says Steve), which keeps it easy to read and understand
• amazing URLs (predictable, easy, and meaningful)
• Paul, who has experience with ASP, says the views are so much shorter than ASP code-behind

And that’s what we came up with in about 5 minutes.

Best thing you could do is download it and try the tutorial at:
http://www.djangoproject.com/

It would be the best two-weekend software project you’ll do for a long time. (Or your money back!)

gerrit and mercurial (hg) online code review

Background

I know a developer team working in missions that is considering whether to go with git or mercurial as the source code management back-end. They said that git users are really excited about gerrit, an online code-review site for git. If there were a comparable thing for mercurial, it would level the playing field a bit.

Here are a few options to consider.

Through the looking glass

I routinely use remote desktop technology to view systems I need to work on. Today, I’m doing three “hops” to get the job done:

• From my Ubuntu 10.04 box, I use Remmina to access the TightVNC server on a Windows XP box on my local network
• which accesses a Windows 2003(?) server via RDP over a VPN
• which uses TightVNC viewer to access a Windows XP machine on the remote network

(Why so many hops? Suffice it to say: it’s complicated, but necessary, due to strange firewall and DNS issues.)

How many “hops” do you use, in your day-to-day work?