Inspiring a New Generation of Engineers

I used to have an old Motorola e810, a standard non-smart phone with a camera, circa 2005. Being the cheap bastard that I am, this phone doubled as my alarm clock, boasting multiple different alarms. I stumbled across an interesting quirk: you couldn't set more than one alarm for a time; that is, if you had an alarm for 6:00pm that said "Laundry," you couldn't have one for 6:00pm that said "Order Piza". Honestly, it wasn't a big deal. It didn't matter which alarm went off, as long as one did, and that was probably a design decision made when they created the phone. But it still bothered me.

The only thing I could think of that made sense was that each alarm was some sort of key/value pair where the time was the key, and duplicates weren't allowed. I can sort of see the logic for this. If it was a hash table, for instance, then at each minute you just perform a look up to see if there was an alarm set. This makes slightly more sense then iterating over all of the possible alarms at each new minute (although since most people only have one or two alarms, probably not a huge savings).

While it was a fun exercise to try and determine why the phone wasn't working as I expected, it was sort of mental masturbation. Without the source code to the phone, I have no way of knowing if my educated guess is even in the ballpark. We're in the black box stage of our information age: we don't have to know why things work, we just know that they work. Is the voodoo too complex? Maybe. Because we don't care to know? Another likely possibility. But why with electronics versus other complicated devices?

The Man Code states that you should have a certain amount of automobile expertise. Since the dawn of cars, it's been accepted, if not expected, for man to have prowess with the car. In the beginning they were simple enough that this was a feasible task: there's the carburetor, there's the engine, there's the fuel pump -- ok, I've got this. However, a 2005 Honda Civic is not a 1949 Chrysler. The Civic is exponentially more complex, with computers, fuel saving devices, emissions lowering devices, etc -- but yet you can still change the oil, replacing the starter is not out of the question, and even replacing the brake pads is a manageable task. Heck, if you don't want to pay someone to replace the rotors or the the alternator, you can do it yourself. If you're a real gear head, you can even tackle an engine swap.

Let's go back to the cell phone real quick. I consider myself the computer equivalent of a gear head, but I was unable to do anything about this problem. I couldn't even confirm why it behaved this way because the software for the phone was unavailable. Not that it was too complex for me to figure out, but that I didn't even get the chance to try. I'm sure that everyone has run into that one glitch, that one feature of a program that doesn't work quite right -- the one that makes you think to yourself, "this is so close to working, I ought to be able to tweak it." Well, if it was a car, you could. You could tweak the air to fuel ratio in your carburetor. It's not like the hood of you car was sealed shut by Honda when they sent it to you, preventing you from peeking inside to see how everything works.

In the past, complex devices like tractors, lawnmowers, furnaces, cameras, clocks, etc have been possible to tinker with, if you're willing to put forth the time to get past the learning curve. Cellular phones, Windows, the iPod, GPSes, and other electronic gadgetry aren't accessible in the same way. And don't give me that bullshit about decompiling it to assembly code and tweaking it there; a human readable form (source code) exists that software gear heads just can't get to because the company safe guards it as proprietary information.

This is why open source is important. If you're curious how MySQL or Firefox works, you simply pop the hood and look. How many inventions that we depend on have been created by creative tinkerers? It's fine that most people don't care how their web browser works -- but for that .001% of people who do care enough to want to see makes it tick, why in the world would we prevent them from that?

This is one of the reasons I am such an advocate of the Google iPhone (aka the G1 aka the Android). Linux, Mozilla, MySQL, and a slew of other software has long been freely available to tinker with, but those are limited to computers. One of the other most important devices that we all use on a daily basis, the phone, has just joined the ranks. This is going to open up the phone to unexpected innovations that we have seen with computer, automobiles, and everything else with which tinkers are free to play with. Society has everything to gain from this.

Ultimately, I'm hoping that the open source movement will inspire a new generation of engineers. Not everyone will be interested, but there's no reason to prevent the people who will be. People who learn how things work because they want the satisfaction of knowing how something works. People who want to tweak something so that it works better for them. People who see something good and want to make it great.

Why I Program

On the bus this morning, I read part of "The World is Flat" that deals with left-brain versus right-brain activities; to Friedman, jobs that can be replaced by technology and outsourcing (left-brain) and those that require an element of creativity (right-brain). Friedman made a statement:

"This weekend there will be accountants painting watercolors in their garages. There will be lawyers writing screen plays. But I guarantee you that you won't find any sculptors who on weekends will be doing other people's taxes for fun."

Friedman is trying to convey that some things are done out of passion, and some out of necessity. People do taxes out of necessity, people paint watercolors out of passion. It's all very black and white, and the jobs of necessity are the ones that are more easily sent overseas to India.

Can't there be be passion for traditional "jobs of necessity"? You can outsource your financial management to an established group or mutual fund -- but what about the excitement of finding good deal and making a sound investment? The satisfaction of having researched it and adding your own judgment to what you find? You need a table to eat dinner on, but can't creating that table be a creative outlet? When categorizing these jobs, I don't think there are clear cut categories; it's up to the individual to say what he is passionate about.

Whenever I go home or spend time with my parents, a lot of my free time is devoted to programming, and they can't figure it out. "You do this all day for your job [or for school]. Can't you put down the laptop?" While it's true that I sit in front of a computer for my day job, I'm working on problems of necessity. Database development isn't particularly exciting for me; what I've been working doesn't require a lot of creativity and seems mechanical. So, in my free time, I need to exercise my creative muscle, and I choose programming for that task.

I've often said that good code is beautiful (no, I'm not talking about the Perl Camel). A well designed component or architecture has elegance and sophistication; simplicity yet robustness. You step back, look at the code, say "Damn, that's gorgeous", and you know when you've created beauty instead of just hacking out a quick fix. This element of beauty is something that I've found lacking in a lot of code and with a lot of programmers. Too often programming is treated like a boring job of necessity than the passionate job of creativity that it can be.

This is why I enjoy coding in my free time. Sure, it's all ones and zeros when it comes right down to it, but the important thing isn't what the end product does; how you got there is so much more important. Programming gives you a chance to come up with creative solutions. And they don't even have to be real problems!

I spent an entire day obsessed with the idea of programming language quines, which are programs that print out their own source code. Sounds easy, right? "Just keep a copy of the source code in the code, and print it out. Oh crap. How do I recursively store the code in the code..." Quines are, as my office mate likes to say, mental masturbation: they produce no value and in that sense are worthless, but so incredibly amazing at the same time.

I look at quines and other beautiful code with the same curiosity and admiration that I look at pieces of artwork in a museum; I appreciate them not because of their utility, but because of their elegance. And I code as a hobby outside of my day job not as a task of necessity, but the desire to create beauty.

Asynchronous PHP Gotchas

As part of a PixCede re-write (read: correcting damage from a rabbit I chased to far down a hole -- more on that in another post), I decided to modularize the scripts for handling new messages. Previously, procmail was sending the email to a single PHP script that handled extracting attachments, storing the image, and sending the confirmation message. As the script started to get unwieldy, I decided to break it into separate scripts:

• newMailDaemon.php - main script that handles writing the attachment to disk and asynchronously calling additional tasks. This is purposely kept minimal, so that there is less chance of something going wrong (e.g. compilation error). Worst case scenario, the email is written to disk; if the other tasks fail, the unit of work can be replayed from the message dump


• processMail.php - handles extracting the image, creating the shortname, inserting it into the database, and sending the email confirmation. This will be split up eventually

After newMailDaemon.php writes the mail file to disk, it calls exec(..) to asynchronously kick off processMail.php. I was about to pull my hair out until I figured out a couple of things:

• exec(..) does not get called as though through a shell. As a result of that, you need to use absolute paths. That means that `php ./processMail.php` becomes `/usr/bin/php /var/pixcede/xxxx/processMail.php`


• As a corollary to the previous point, absolute paths need to be specified in exec'ed scripts as well; e.g. the database file specified to sqlite3


• Permissions matter! I was using a logger to follow the actions once an email was resolved, and I couldn't figure out why nothing from processMail.php was getting logged. I was logging the command that got exec'ed, and running it manually -- and it always worked. procmail calling newMailDaemon.php calling processMail.php runs at the permissions of the user account that procmail is acting on behalf of, so the script needs to permissions to run from that user. Whatever is running the main script needs permission to exec additional scripts

This design is far from perfect. The different actions need separated more; ideally, having newMailDaemon load and parse an external workflow file wouldbe nice -- then I could lock down newMailDaemon and not have to change it to update the workflow (less chance of it failing). To handle and track the different steps, I would like to keep a table of actions to process, have entries put into that, and have another script act on that table until all tasks are set to "complete"; that sounds more scalable and manageable than exec'ing scripts for each action that needs to be done. Having newMailDaemon schedule tasks and processTasks execute those tasks sounds cleaner.

At any rate, PixCede works again now!