Yahoo is going to shut down this group unless I send a message every
now and then so here's an unoffical blog entry. ;-) Good thing XP
values communication and feedback. Don't all jump in at once!

Speaking of blogs, we added blogging support to bOP. I find it
interesting when people raise an eyebrow to this. We added a Wiki,
and got the same reaction. Reuse is hard, and reusing random code
without tests is very hard.

bOP contains lots of tests so it makes it easier to reuse its parts
than some random blog system. The bOP blog language is the same as
our wiki language, which is comprised of XHTML tags written using a
SCRIBE-like syntax (SCRIBE is the predecessor to LaTeX) and CSS files
in each wiki/blog directory to control the look and feel of the XHTML.
The difference between blogs and wikis is naming: Blog entries are
implicitly named and wiki pages have explicit names.

The hard part to get right with blogs is everything but the blog,
e.g. webserver integration, forum/user name space management, UI
widgets, file up/download, authentication, authorization, and search.
The Blog code turned out to be 185 NCLOC and 150 lines of unit tests.
Not much code when you reuse all the hard stuff.

I was away when the other programmers at bivio added the blog modules,
and I wrote wiki modules, including the formatter. It was unfortunate
timing. However, they got it done, and in record time.

When I got back, I helped implement the blog publish/draft feature for
one of our customers. It is an important feature, because they are
using it for corporate communications, not personal diaries. Blog
entries need to be reviewed by several people before they are
published.

However, my "vacation" wasn't one. I wasn't refreshed. I came back
agitated. My mind was already in a fearful state, which is absolutely
the wrong way to look at new code. I should also add that I had been
having a frustrating time adding search to bOP. The libraries out
there are not easily reusable, and I know that search engine
technology is quite complicated so creating our own is on the wrong
side of the risk/reward equation, but I digress. Back to stiffling
innovation...

I paired on the blog publish/draft problem for a while, and got
frustrated. There were no unit tests. There were acceptance tests,
but they were overly specified so they required lots of change due to
the interface changes required. We had to change too much at once to
get it working.

Not. I was afraid, and my thinking was muddled. I was under time
pressure due to the fact I am behind on search. What I needed to do
was take a breath of fresh air, and dig in a small change at a time.
Instead, I vented stale, putrid air. It was the worse kind of
immature dissing you can imagine. I muttered loud enough so people
could hear but didn't give them an opportunity to discuss the
problems.

I'm ashamed of myself and have since apologized. Unfortunately,
negative feelings have a much longer half-life than positive feelings.
The wounds will heal, but we all were less efficient due to my lack of
self-awareness.

Mistakes are essential to innovate. If there's one good thing I can
say about Bill Gates is that he understands the value of mistakes.
Here's an excerpt from a recent Economist about Billanthropy:

http://www.economist.com/displaystory.cfm?story_id=7112702

Ready to make mistakes

To its credit, the Gates Foundation has built performance
measurements into all its projects?and importantly is prepared to
axe those projects that do not come up to scratch. One of the
things that Mr Buffett says he admired about the Gateses is that
they are ?more prepared to make mistakes than me?. An example is
the foundation's funding of the Global Alliance for Vaccines and
Immunisation. National programmes last for five years, and are
scrutinised by an outside auditor during year two or three. Those
ahead of target are rewarded with an extra $20 per vaccinated
child. Those on target continue to be funded as planned. Those
below target have the tap turned off, at least until the problems
are resolved.

Mr Gates talks obsessively about the need to be willing to fail,
and to learn from mistakes. Mrs Gates notes that over time the
foundation has taken a broader approach to the issues it is
addressing. For instance, in a big project to combat HIV/AIDS in
India, the foundation became aware of the need to help prostitutes
find alternative sources of income, which is leading it into
microlending to poor people.

It may soon become clear whether the foundation is as ready to
deal with failure as Mr Gates suggests.

There's the rub. Dealing with mistakes is hard, especially my own.

While it is easy to sit here calmly and accept that mistakes are
natural and necessary to innovate, it's tough when I don't have the
right frame of mind. It's really tough, especially given the
environment in my family-of-origin. Mistakes weren't allowed when I
was growing up. Spilled milk was a huge affair.

The lack of unit tests in the blog modules was a simple mistake. My
co-workers were under tremendous pressure to figure out the wiki code,
and add the blog modules. We work in a bullpen for a good reason, and
that's because programming-in-the-large while staying small
(aka. reuse) is very hard.

My experience is not unusual; history repeats itself. Which brings me
to the new blogs: podcasts. Podcasts are their name thanks to the
fact that Sony is a big company just like IBM. Remember the IBM PC?
IBM didn't want the PC to be a success so it gave Microsoft the keys
to its future by outsourcing the PC's operating system. Well, Sony
made the same mistake imiho.

Remember the walkman? Or, if you were at UCSD in the late 80s, the
Sony Walkbird? The walkman and the discman were truly innovative
devices. And, well, the minidisc player isn't. Sony failed to
innovate, and as we all know, Apple did. I'm pretty sure someone at
Sony was afraid of bastardizing its own "mainframe" business: CD
sales.

Apple didn't invent the MP3 player. Apple didn't invent online music
stores. Apple coupled the two with iTunes, and spent a lot of money
marketing that combination. They now command 70% of the market of
portable music players. I don't know if we'll all have Mac Minis in
our living rooms in a couple of years, but I do know that Apple's
iTunes isn't going away, and it's clear that Microsoft is afraid.

And what about Sony? In 2000, Sony was still thinking of the minidisc
player as a way of selling more music through its massive physical
distribution network at exhorbitant prices.

Media is big business. Podcasts, GarageBand, iTunes, Skype, and their
ilk are a big pain in the ass to the Media Moguls. Consider Adam Dorn
(aka Mocean Worker), who remixes jazz with new grooves. It's just him
and his computer(s), and he cranks out (great imiho) album after album
which are sold through and iTunes (some iTunes only EPs). You can get
the Mowo instantly, and groove now. Why wait?

My guess is that the IBM/Sony stories are rooted in fear. You can't
innovate if you are afraid of making a "big" mistake, like destroying
an existing revenue stream, in the case of IBM in the 80s and Sony in
the 90s. And, Kodak this decade.

One last point about fear. It's pervasive. It's why I am writing
this email. I'm afraid to keep making the little mistakes required to
figure out how Xapian (search engine) works so I am writing this email
to relieve stress. It's something I have complete control over. It
feels very safe. It's like playing chess in a way. I think about
each move (word) and sequences of moves (sentences). Unlike chess,
mistakes are easy to correct.

To give myself some credit, this message (unlike the vast majority of
chess games) will remain in the ether forever. I have no control of
the content -- an important distinction between message boards and
self-published blogs, btw. And, I'm proud of myself that I have the
courage to hit the send key.

I hope you all find the courage to give me some feedback.

Thanks,
Rob