Advogato

  • “If there’s anyone to blame for the current open-source mess (both in the ideological sense and in the practical sense), it’s RMS. He’s turned the whole purpose of hacking upside down. Now sharing code, instead of being a by-product of creating a good program, has become an end in itself” (tk, as quoted by chalst).
  • On advogato, chalst, graydon and raph all regularly write interesting material. Definitely worth spending a few minutes digging through their older stuff.

Moving on …

After three months, I’m going to be leaving my telecommuting adventures with Ergnosis and returning to working in Edinburgh. I have enjoyed my time there and they’re great people, but in the end living 400 miles away from the office meant more travelling than I can handle. I’m just not built for getting up at 6am and hanging around in airports, regardless of how good the job is. I’m glad I tried it though, and I plan to do a little writeup of my telecommuting experiences sometime soon.

Would the engineer please stand up?

I occasionally play with molten metal .. just to fill the time, y’know. I’m about to design and build an electric furnace, because I can use it indoors (and living in Scotland during winter months, this is important).

But, you say, this is meant to be a software engineering blog! Ah yes, I’m going to try to throw some evidence into the “is software engineering really engineering?” debate. I don’t personally hold a strong position on this matter (there’s plenty of more interesting things to talk about) but I think it is useful to draw parallels between traditional engineering and software engineering to see what can be learned.

My electric furnace is going to involve mains electricity, so I’m being really quite careful with it. It’s not a complicated project by any means, and I’ve built a non-electric furnace before, but since this is my first big electric project, I am being very cautious and doing a lot of careful reading and thinking before I start. Gosh, sounds like how you’d approach a software project!

After writing down a rough design, I decided that it would be prudent to think of all the things which might go wrong with the furnace, on the basis that I’d rather not be surprised by any of them occuring later! I brainstormed everything which could possibly go wrong (including some highly unlikely ones like “total structural collapse of the furnace”) and wrote them all down. At this point, I realised that I was really just doing a formal Hazards Analysis. I wasn’t doing it in a perfunctory way, or because someone told me I had to do it. I was doing it because it seemed like a pretty sensible idea, and I didn’t want to electrocute myself later! Later, it turned out that even the unlikely “total structural failure” hazard led me to change the design slightly (by adding an extra power switch at a safe distance from the furnace).

Now, if your company follows certified processes (such as ISO9001) you may be familiar with doing hazards analysis for software. Software often has the potential to cause harm to humans, and this is reflected in the ACM code of ethics with it’s Asimov-like “avoid harm to others” imperative.

But traditional engineer goes much further than this kind of “codifying common sense” which is familiar to their software counterparts. For example, when I am choosing which wire I should use in the furnace, once I have established the maximum current it needs to carry, I can look up a standard table to find out which wire I should use. Well, it doesn’t tell you exactly which product you should use. It tells you which “British Standard” the wire should meet, and you can go to the shops and ask which wire meets that particular standard. So, it’s a bit like interfaces and implementations in software.

The guys who wrote the “British Standard” have done a lot of thinking about what properties the wire should have. For example, if you’re looking to wire up a fire alarm system, there’s probably a BS1234 standard which ensures that the wire isn’t going to burn up in a typical house fire.

An electric engineer would never just pick up a random bit of wire and say “this’ll probably do”. An engineer would appraise the situation, look up the appropriate standard and choose their material based on that information. Later on, they could justify their decision by refering to their appraisal and the standards.

Now consider what a typical “software engineer” does. “Uhh, we need a bit of code which does a linked list. Hmm, here’s one here. Seems to work. Okay, let’s move on”. This doesn’t bear much similarities to other kinds of engineering. “Seems to work” is what is keeping the software industry at it’s current stage of buggy programs and unpredictable delivery times. “Seems to work” just isn’t good enough.

Now it’s much harder for software bunnies, or so the argument goes, because we produce new and innovative program. We’re never just “building another bridge” so it’s hard to distill relevant experiences down into tables or whatever. Partly, I agree with this. But I can also see that software writers often waltz by opportunities where a more strict engineering approach could be usefully applied.

Engineering standards and tables of engineering data are essentially “distilled experience”. There are two different examples of “distilled experience” common in software: design pattens and libraries.

Design Patterns can guide you to choose appropriate design elements for your software, and highlight possible difficulties with related approaches. They certainly are an example of “distilled experience”. But currently, they’re a process which you apply to structure your source code. Often once you’ve used a design pattern, it disappears away into your codebase – possibly you leave a comment saying “I’ve used the blah pattern here” to note it’s passing. We should demand languages where the patterns are one of the building blocks we use to compose our programs. If you use a pattern, you should be able to point at it in your codebase, and it should stand out distinctly as “the pattern” rather than being blended in with whatever code is surrounding it.

The other example of “distilled experience” is libraries. If you want, say, a linked list you’d never consider writing one from scratch since there are many high-quality implementations already written. We should be standing on the shoulders of giants as much as possible. But, then again, with our “engineers hat” on we should ask “what does high-quality mean?”. An engineer would check all of the relevant properties of a particular kind of wire before using it – the resistance, the density, how the resistance changes with temperature, maximum operating temperature etc. How many properties would you check for a third-party library? The “it seems to work” property? Does it provide O(n) or O(lg n) access to data? How much memory will it use? Is it thread-safe? If so, what effect does the locking policy have on performance? Does it safely interact with DLLs? All of these questions are ones which have bit me in the past. What do you know about the people who wrote the library – how much testing has the library had? To what standards was it written? Did they do code reviews? Are the results of the reviews available? Did they track any code quality metrics? Did they provide a list of “common misuses of this library”?

To the software engineer, this might seem like overkill. You just gotta trust that the libraries do what they claim to do, yeah? Well, no, that’s not good enough. Engineers don’t trust. They measure, they quantify, they document what they did and why they did it. There is traceability and there is responsibility.

Having worked on a big software project for several years, I’ve learned to respect the required level of “process” and accountability which you must have above the “coding layer” in order to capture and learn from experience (community experience and your own). That level of formality never seems to get carried on down to the level actual coding, except possibly if you are writing code for the Space Shuttle.

I guess this goes full-circle back to my interest in languages and tools. We can improve the situation by critically examining our tools and practises, and improving them to capture our knowledge and experience. Tools, particularly, shouldn’t be viewed as immutable – they should be flexible and adaptable. Too often, I see examples of primitive tools and adhoc rules being used in building software, and they make me wince.

There’s a quote I once read by, I think, Marvin Minsky which went along the lines of “computing is still such a young field – you have to remember that 99% of what you have learned will turn out to be total rubbish”. I feel that way about our current crop of tools.

(Incidentally, I keep starting to write blog entries, and then give up on them once they reach about 1000 words and have become out-of-control essays, sprawling out across multiple topics in an unstructured way. I find that writing a blog has required me to reexamine my own writing skills, and how I think through things. Which is all good for me, but I’m no longer convinced that I can easily produce blog-sized entries without a lot of effort. Oh well, I originally started this blog as a means of keeping track of my computer-related interests .. since otherwise I get to the end of the year and think “what did I do this year?”. It still fulfills that purpose at least).

Danger, danger

Climbing is a high-risk sport, yeah?

Last night at Alien Rock some kids threw a firework rocket into the main climbing area. I was part-way up a climb when I heard this loud screeching sound. I’d just started to think “That sounds just like a …” when I turned around to see what looked like a sparkler weaving violently across the room, heading straight towards me. I turned back to face the wall and make sure I had a good hold (inbuilt sense of self-preservation kicking in) when the rocket exploded just behind me, and immediately on front of DaveM who was belaying for me. Fortunately, it was just made a big bang at the end rather than sending out a shower of sparks. But there was little bits of rocket left on the floor, and I think Dave got hit by something hot on his eyebrow.

So, yeah, next time you go climbing, double-check your rope and harness and watch out for incoming explosives!

Back to Basics

I keep getting dragged down to the roots of computer science and other disciplines. If you set out to write a Better Development Environment, you’ll pretty quickly slide away from code-related problems into general problems such as “information visualization” and “cognitive studies”. Playing with programming languages, it’s amazing how quickly everything interesting reduces to theorem proving. In a way, it’s disappointing because it means there’s less virgin territory out there than you’d expect. It means that you’re often better off heading for a library and spending a few hours reading rather than hacking “something new”. Sometimes I feel that I spend more time “reading” than “doing”, because so much of what I could write has already been done before in a research project somewhere.