The Bug

One of my former students asked about the trickiest bug anyone had run into -- and that got me reminiscing about The Bug. I have some tricky code that I need to write, but I don't want to do it, so I'll stall by rambling about The Bug for a few minutes.

Back in the late 1980s, when I had a brand-spanking-new BS degree in Computer Science, I went to work for a startup founded by some of my college friends. We were putting small microprocessor-powered control units onto oil pump jacks, which were connected by radio to a base station PC (running MS-DOS!) back at the offices. The controllers (Texas Instruments Remote Telemetry Units, or RTUs) would turn the pump jacks on and off to maximize production, reduce the chances of equipment failure, and so on.

We had solar panels to charge batteries, which then supplied the power to the control units and radios. There's not generally a handy plug-in strip in the middle of an oil field in Wyoming.

Everything was working fine until around October. Then, on some mornings, we'd come into the office, and find the base station PC had rebooted. All the data from the night before was lost. Your friendly MS-DOS PC just staring at you, with no clue at all what had happened or why.

For months, we'd sit by the PC, late into the night, waiting for the machine to reboot. And it wouldn't. Unless we got tired of waiting, decided it wasn't going to happen, and then went to bed. After everyone was sleeping, sometimes it would reboot. Did I mention that this was in Wyoming? Not even in a nice part of Wyoming; Casper the Friendly Ghost Town was a bustling metropolis compared to where we were. We were in One-Horse-Oilfield-Shack, in the middle of Winter. Yeah, good times.

This was The Bug. And it kicked our butts for quite a while, until Greg figured it out. I remember when that happened. Greg was in the office next to mine, and I heard a short blast of high volume profanity (trust me, oil field swearing can be pretty intense), followed by a coffee mug smashing against a wall. Greg was a little bit pissed off. But he found The Bug.

Here's the sequence of events for The Bug.....

On a cold and cloudy day, the solar panels wouldn't gather much energy. If that happens for a few days in a row, the batteries can run low. When the batteries run low, the radio will still turn on to transmit from the control unit, but it only sends zeroes (seriously?!?). If the number of bytes sent was odd, it would pass the simple RTU checksum verification (thanks a lot, Texas Instruments RTU protocol developers), and be viewed as a valid message. The base station would then take this message, and update the entry (that's entry zero) in the database. Entry zero was not a valid spot in the database, though, and it turned into a write to the operating system memory space.

Y'know one of the advantages of Window over MS-DOS? That would be memory protection. On a DOS box, you could change pretty much any chunk of memory at any time. Wham, bam, blue screen of death 'mam.

Long story short, I don't trust batteries, and I'm not particularly fond of Wyoming. If we had coded everything in Rust, we could have avoided The Bug entirely.

Now that the story is done, I should jump onto the tricky code that I need to write. If I get the frames of reference exactly right, it'll be tight, clean, elegant, work immediately, and be awesome. Or it could be a few days of weeding through debug output to figure out what I screwed up. Potentially followed by a blast of profanity, but I don't usually throw my coffee mug. And I'm not in Wyoming, so that's cool.

Hmmm. I should make a cup of coffee first, before I jump into the tricky code. Yeah, that's a good idea. No, I'm not stalling. A cup of coffee will help.