Experience

Especially with a code base that is mature, assumptions correctly made years ago can be terribly difficult to deal with later when the current and new assumptions reign. Writing code that lasts even 5 years and "is robust" is what everyone wants to do for sure, but the recipe for doing just that is not easy to learn and even harder to actually do. For example, you might know what changes need to be made to conform to even the most obvious object oriented design principles, but the business requirements and time to market needs dictate leaving once again old crusty code alone and racking up yet another round of technical debt. Over time, this technical debt will demand payment and the effects on your ability to hire, employee morale, design changes possible, speed of delivery, testing burden, marketing message etc. become very real and very painful. I wonder, can these concepts can be fully learned without actually experiencing pain? Could you even attempt to learn these concepts experientially in college? My experience so far makes me think that one may know something intellectually, without really knowing it. It seems like for so many, one may talk about design patterns or abstraction or low coupling, but until you actually try to build something the other way, the painful way, you just don't appreciate what you are avoiding. What's worse, junior developers who, for no fault of their own lack experience with "the hard way" have a difficult time understanding why one must "go the long way around" to do what seems like such a direct solution. Passing on the stories of the past and their consequences and lessons seems to be an unending challenge.

From my economics book comes this fictional story which I think illustrates the point:

"One Pepsi plant is managed by an economics major with an MBA and has a labor force with an average of 10 years of experience. This plant produces a larger output than does an otherwise identical plant that is managed by someone with no business training or experience and that has a young labor force that is new to bottling."

I definitely fall into the "young labor force that is new to bottling" camp. In the technology industry there is a tendency to glorify the young and bright rather than those who have the experience and have paid the price to learn the lessons that matter in the long run. Whenever I talk to some developer and we talk about where they learned some valuable lesson about software development, rarely do they refer to a book. Almost invariably, they say, "I worked with Joe on this project and he showed me x, y and z. I learned so much working with him." Smart developers, experienced developers who know how to teach and share important lessons to junior programmers seem like a key to the experience problem. Sadly, junior developers who have the presence of mind to ask, listen and apply what they hear are hard to find, and senior developers who are both willing and able to teach effectively are even more rare.

Perhaps the answer to all this is simple. Perhaps it's just he or she who writes the most code, wins. What I mean by this is simply when one writes a lot of code that increases the probability that he or she will make more of the key mistakes needed to learn how to write software with longevity. Just getting more exposure to "how bad things can get" helps bring a sober reality to each line of code written thereafter.

What's remarkable about all this, is that failures in teaching and learning are what keep us "discovering" new ideas that are 40 years old. Truly, "there is no new thing under the sun."

Metacognitive Miscalibration

A few tweets ago, (I always feel weird referring to Twitter in the past tense) I posted: Why are the unintelligent or uninformed so arrogantly confident while the intelligent and well informed so often unsure and apprehensive? There is something very human to thinking you know more than you really do about a subject or issue. While, this problem can be seen in many areas generally it's particularly acute in software development. For example...

AppleScript in Office X

Back when Mac OS X was just about to go 10.0.0 we were busy working on getting Mac Office working on the new platform. There was a lot of pain involved with the transition. Many APIs had been removed and alternatives needed to be provisioned (and tested), the new Aqua interface guidelines had to be applied to the whole of Office and a host of other issues needed to be addressed. I was at the time on the team that wrote the tools for automated testing and I was pushing hard to get Office wide AppleScript support on the list of features we'd commit to doing for Office X. I wanted this not only for our customers, but to augment our testing efforts. With an API to drive the applications we could automate many "smoke tests" on a daily basis as well as set the stage for long term AppleScript based test suites. Automation testing benefits are often not fully realized, especially static automation, until the version after you setup the automation, so I was especially anxious that we get our API in Office X, so we could reap the return on investment for Office 2004.

I remember talking with Jim Murphy about getting AppleScript support like it was yesterday. We were in building 44 and his window office was near the 2nd story walkway that connected our building to the cafeteria. In my naivete I asked why we couldn't just build a some kind of layer to call into the engine code of Office and presto, AppleScript dictionaries for Office would be complete. I pointed to some improvements Apple had made to make AppleScript a better inter-application communication protocol and ask, "How hard could it be?" Jim turned to me and said, "Hard? There's nothing easy about it. It's all pain. All pain." Then as was typical, turned back to his work, leaving me to think.

In this case, I was the unintelligent and uninformed, but very enthusiastic novice. Jim turned out to be right. For Office X, we did try to do the AppleScript work, but it turned out to be much more difficult a problem to solve. After months of work and many dead ends, we pulled the feature from the Office X feature list. For Office 2004, we tried again. In this case, Jim himself, one of our best developers, ended up spending the better part of a year getting AppleScript to work with Office, which is probably worth another post in and of itself. I thought I knew more than I did, I thought the problem was simpler than it was. My confidence and thinking was miscalibrated. The root cause was my lack of understanding and experience.

Wicked Problems

Sometimes, even the best developers underestimate the scope, breadth and depth of a problem. In a drive for simplicity, I observed in myself and others another kind of design time problem where thinking was miscalibrated. The cycle I have observed looks like this:

1. Developer looks at a problem A, and thinks he or she fully understands the problem.

2. Developer designs a simple solution to problem A.

3. Developer codes the simple solution.

4. Developer or tester or marketing or customer use simple solution for problem A.

5. Bugs trickle in and solution A is modified, bit by bit, bug by bug, until it is patched in a thousand ways and no longer looks or acts like the simple solution. The solution is complex.

Some will say, "Hey! Why didn't that lame developer take the time to really understand the problem? Then he could have taken the time to fully comprehend the complexities of the problem, and then design a simple, elegant solution!" The problem with this is that many subtleties to the problem do not manifest themselves until very late in the development process which make re-architecting the solution for such a small issue un-reasonable. But this is a "death by a thousand paper cuts" issue. What's worse, many architectural issues can only be comprehended after years in a problem space, and as promotions or attrition happen, the so called simple solutions, with their complex instantiations proliferate in code.

Adam Richardson spoke to the challenges of wicked problems when he said:

Wicked problems are very difficult to understand by staring straight into them and looking for clear detail, however. They need to be approached from the edges, sort of like doing a jigsaw puzzle where you find the edge pieces first. Having peripheral vision that is trained to be sensitive to the edges is a key capability (this applies both to product teams and to business units - wherever wicked problems occur).

Normally this causes the developer to tack on simple solution to solutions with patch upon patch applied with the Hippocratic Oath echoing in their ears to "do no harm", but deceptively the bandage is not big enough for he wound, and no one knows.

The Desire to Learn

Another reason why some can feel "informed" when in fact they are not is that they have lost the hunger to learn. They've lost the desire to learn and grow. René Descartes said it this way:

Good sense is the most equitably distributed of all things because no matter how much or little a person has, everyone feels so abundantly provided with good sense that he feels no desire for more than he already possesses.

There is a reason for diversity of opinion, it provides the natural tension that keeps us from thinking we've got the problem solved. When I left Microsoft to go back to school and finish my degree, many asked, "Why are you doing that? What are they going to be able to teach you?" I'll admit that this partially resonated with me. I wanted to be the sage of wisdom, but as I've spent time with teachers and students, many of whom do have less "industry experience", I have learned an enormous amount! To put it bluntly, I have been humbled. I've learned things technically, intellectually, physically and even spiritually. I feel so blessed. I'm beginning to think that what any given situation can teach you depends largely on the person experiencing the situation and very little on the experience itself. Put another way, there's a great difference between 50 years of experience and 1 years worth of experience repeated 50 times.

Personal Pride - the Anti-change Agent

Closely related to the lack of desire to learn is the desire to avoid being wrong. So much in school is focused on being right, knowing the right answer to a test or the correct proof or solution. In sports, no one likes to lose. Everyone likes a winner! But this desire can work into our minds in a limiting way. Leo Tolstoy wrote:

I know that most men, including those at ease with problems of the highest complexity, can seldom accept even the simplest and most obvious truth if it be such as would oblige them to admit the falsity of conclusions which they have delighted in explaining to colleagues, which they have proudly taught to others, and which they have woven, thread by thread, into the fabric of their lives.

At work I would say, "We don't fail half as much as we need to." I still think this way, but now I've found a new reason: Failing keeps our mental muscles and joints from stiffing with pride and forming into the arthritis of the mind.

The Well Intended Deception

Another reason for metacognitive miscalibration I think stems from the very basis of good object oriented abstraction and encapsulation. When I write some simple app with Cocoa's AppKit and Foundation libraries, I am, as they say, standing on the shoulders of giants. I've wondered how many actual lines of code are called when I double click a simple app like TextEdit. Who wrote these lines of code? When did they actually get written? What were the discussions behind their design and implementation? Even Apple couldn't fully answer these questions. It is all this code that even the simplest of Cocoa applications depends upon. But still we continue to have demos around how "few lines of code" were needed to accomplish something. This is a well intended deception, because really, more code is being executed and written, but only now how this code works is opaque. It's well intended because who wants to write, maintain and debug more code? Why not offload this to Apple? I expect these demos to continue, because less code for a developer to write is the canonical example of efficiency, but there's something wrong that comes from this. It's the feeling that you actually really understand what is going on to make your application tick. These assumptions and abstractions can be benign, but they can also make one feel and think that they know more than they do and can do more than easily possible.

In the process of detail management that is software development, making things simpler and more tractable is an excellent goal, but there's a limit to how "simple and easy" things can get and still be valuable. Recently I overheard two students talking about building an iPhone application. The one said to the other, "Dude, they showed this awesome demo of Spore, where in only 2 weeks, 2 weeks! they got this awesome iPhone game running! It's so easy. We can totally build an awesome iPhone app!" First, I love the enthusiasm and I hope they do in fact build an awesome iPhone application. Apple was trying to demo how easy it was to build iPhone applications and I think we've never seen a mobile platform that's better for developers, however:

Taking a senior developer who as spent the last several years of his life building games and developing Spore, bringing him into Apple with full and uninterrupted access to Apple's staff of iPhone engineers that have been developing the iPhone SDK and iPhone applications hardly equates to a college student being successful at all the complexity involved with boot-strapping a company and building a successful iPhone application from scratch. Building great software is still hard, hard work. It's tedious, often un-glamorous and takes someone who doesn't mind getting down and dirty with the details to make something great. Most will simply give up, and even those with the passion and stamina to continue are hardly assured of success. I suppose I'm arguing simply that a healthy dose of Steve Job's famed "Reality Distortion Field" do not a great developer, company or software make.

There are probably other reasons we tend to think we know more than we actually do. But the lesson for me is this: I need to take some time to ponder and reflect regularly. Am I in the "unintelligent or uninformed and arrogantly confident" camp? If so why and how can I get humble? Am I part of the "intelligent and well informed but unsure and apprehensive" group? If so why and what can I do increase my tolerance for risk and decrease my fear of being wrong? A little meta I know, but the title should have warned you. :-)

CodWarrior

Here's a blast from the past. As Xcode continues to improve as the default IDE on the Mac and for the iPhone, the majority of Apple's current developers don't even how CodeWarrior saved Apple. With the release of the iPhone SDK, I think it's not far fetched to imagine Apple's WWDC attendance tripling. Someone sent me this graphic from an old Metrowerks t-shirt. If you have this shirt, you fully qualify as an old timer.

Heuristically Thinking

You don't have to understand calculus to appreciate this entertaining story:

A teacher, trying to explain what a theory is, asked this question: “If you take a letter half the distance to a mailbox and stop, then start over going half the remaining distance and stop, then repeat the process over and over, theoretically will you ever really get to the mailbox?” One bright student said, “No, but you’ll get close enough to mail the letter.”

There are many definitions for what a heuristic is, but the one I like best is illustrated in this story. A heuristic is "close enough." I am beginning to believe that more and more what matters in software isn't so much building the perfect algorithms, that match and mimic real life in every way and hold up in every edge case. What matters most is getting a model that comes close enough. What matters are heuristics.

My favorite chemistry textbook, has this to say on the topic of "The Kinetic Molecular Theory of Gases" (KMT):

However, although laws summarize observed behavior, they do not tell us why nature behaves in the observed fashion. This is the central question for scientists. To try to answer this question, we construct theories (build models). The models in chemistry consist of speculation about what the individual atoms or molecules (microscopic particles) might be doing to cause the observed behavior of the macroscopic systems (collections of very large numbers of atoms and molecules).

A model is considered successful if it explains the observed behavior in question and predicts correctly the results of future experiments. It is important to understand that a model can never be proved absolutely true. In fact, any model is an approximation by its very nature and is bound to fail at some point. Models range from the simple to the extraordinarily complex. We use simple models to predict approximate behavior and more complicated models to account very precisely for observed quantitative behavior. In this text we will stress simple models that provide and approximate picture of what might be happening and that fit the most important experimental results.

The textbook then goes on to explain postulates for this model of thinking, many of which by themselves are absolutely false, but taken together and used properly, they create a system of thinking that works for a wide range of situations. This collection of half-truths produce a half-truth baked solution, absolutely, but one that is indeed close enough.

For example, some "half-truths" or "simplifications" if you will, involve assuming things like: each molecule of gas is perfectly spherical in shape and any collision is perfectly elastic in result. Or worse, the volume of all these individual molecules is assumed to be zero! Individually, each of these 3 statements is categorically false, but they were the right bits to "design away" as they defined the problem space so that the model could be simplified, made useful and the problem of dealing with billions of particles be made into something tractable.

To me, this is more than simple object oriented encapsulation and abstraction. This is thinking about the whole problem differently. It's about looking at individual behaviors from a very small sample and knowing, by some spark of genius, which attributes are the important ones to the ultimate outcome of the system as a whole, and which are not. This is writing software that is able to predict things, for example, test software that is able to predict when "something good" has happened and also when "something bad" has occurred. This is about designing software by building software models that categorically do not reflect the real complexities of the system, but that taken together, get "close enough" to do real work in the real world. Ultimately your models will fail when pushed to the limits, but even just understanding those limits will help you better understand the problem you are tasked with solving! It even begs the question: What kind of programming language best allows for the definition and use of heuristic models?

I don't know where I first heard this, but someone once said something like, "Where Microsoft codes if statements, Google codes in Bayesian probabilities." There are more data and variation in that data than there ever was before. If you are going to write or use programs (very likely) that deal with large amounts of data (also very likely) it might be a good idea to get used to thinking about things in heuristic terms. It may not be exactly perfect in every case, but it will be close enough.

Finishers Wanted

When I was a little boy my Mom had me memorize this little poem:

Stick to your task ’til it sticks to you;
Beginners are many, but enders are few.
Honor, power, place and praise
Will always come to the one who stays.

Stick to your task ’til it sticks to you;
Bend at it, sweat at it, smile at it, too;
For out of the bend and the sweat and the smile
Will come life’s victories after a while.
—Author Unknown

I think my Mom had me memorize this poem because she knew I would need it. She understood better than I the old adage that "Life does not reward us for effort expended." Finishing is required.

For me, it is exciting to find a problem and imagine a way to solve it. The creative exhilaration in coming up with a solution that will work within all the constraints involved is almost intoxicating. I have a remarkable tolerance for ambiguity and when the major "problems" as I see them, have been solved, filling in all the details seems so much less important. The hard design work has been done. There's perhaps little glory in all the simple, small and detailed work needed to connect the dots and make the grand vision a reality. However, software is ultimately just simple 1s and 0s and if you don't fill in all the details, then all you are left with is a dream. You've got to have both the vision and the finishing of all those tiny details.

"This is all your app is: a collection of tiny details." - Wil Shipley

Missing a few details can drastically reduce the value of the whole idea. I guess that's why I love Mac software so much: There's the constant demand from both the users and my peers for my concerted effort across the entire spectrum of "pie in the sky" ideal to actual, practical details in implementation. To make it work in software, you need to consistently execute well across the whole spectrum of work. Let me underscore the words consistently execute again, they are very important.

Matt Ball recently wrote a nice post about some up and coming Mac developers that worked so hard on their first release, but since then have produced relatively little. They haven't created new apps, updated their 1.0 apps, even posted to their blogs. Some still have ideas in picture form posted in all their high fidelity glory, but with no application to show or sell to the customers that have been waiting to see the finished product. These developers seem to be struggling with consistently executing against their plans.

Matt goes on to explain that he thinks this is related to their young age. Most of these developers are young (19 years old) and he thinks suffer from some kind of "shiny ball syndrome" where they are easily distracted from one project to the next. I don't know the developers, and they could be easily distracted or they could have absolutely justifiable reasons for their delays, but the result is the same: Doubt builds as to their ability to consistently finish their ideas. Their credibility and reputation weakens.

Many of life's failures are people who did not realize how close they were to success when they gave up. - Thomas Edison

I don't think it's age. I think that's far too simple an answer. I know developers in their 50s who struggle from this exact same problem. It's not size or lack of resources either. Look at Microsoft. Here's a company where a large part of their problems revolve around consistently producing, not a lack of money or great people or innovative ideas.

The idea is not the thing!

One part of the problem comes from patent law. There is a remarkable and universally held assumption that ideas are worth a great deal. I will not say ideas are worthless, but they are worth far less than most of us realize. Even the most simple idea takes remarkable effort, and follow through to get designed, built, packaged, and ultimately used by others. This is why I felt My Dream App was destined for difficulty. They had enthroned ideas as the product, when in reality it's all the grunt work after the idea that make the product. It's all those pesky little details and the consistent effort required to follow up and deal with each of them that matters.

Finishing is the thing!

When the iPhone was released, there was a collective groan world wide from designers who had years before envisioned the ideas that Apple had now so beautifully produced in a finished product. Kim Lenox, Senior Interaction Designer of Adaptive Path explains:

With the launch of the iPhone, I’ve been hearing many grumblings from interaction designers who’ve worked for various, well known consumer electronics companies. We can all see in the iPhone aspects of our concepts from years past that were brushed aside or died prematurely. Our concepts are suffocating under the pile of NDA verbiage, never to see the light of day. What sets our mere concepts apart from this final product however, is a company with leadership who has the fortitude to take the risk, find the budget, and push the technology for the single cause of designing compelling user experiences. Apple got it right.

Amazing isn't it? Once again, it's all about execution and finishing, not just the ideas. Leadership is important for sure, but finishing the job in a company is so much more than Steve Jobs simply saying, "We're going to build an iPhone and it will have a compelling user experience." It's thousands of decisions made by hundreds of employees at Apple and elsewhere. It's dealing with setback after setback and still pushing forward. It's taking the right calculated risk (EDGE and AT&T) and saying no to other things (10.5 on time, a Dev SDK) in order to finish. That is the task of finishing and at Apple, it seems to be part of their DNA.

Tranquil and Steady Dedication of a Lifetime

One of the most important attributes of a software company I would like to work for, comes from an idea the late Adlai Stevenson a U.S. Democratic politician explained when referring to patriotism:

What do we mean by patriotism in the context of our times? I venture to suggest that what we mean is a sense of national responsibility ... a patriotism which is not short, frenzied outbursts of emotion, but the tranquil and steady dedication of a lifetime.

In a great software company, there wouldn't be "short, frenzied outbursts of emotion" but a consistent focus on finishing in a steady and sustainable manner. My guess is that those prone to "putting on a big, glitzy show" and those that don't effectively resist the "constant bombardment of new and exciting things to try out" will have set themselves up as an unsustainable business, and ultimately end up disappointed.

Saying No: a feeling of strength in reserve.

One of the biggest challenges is just saying no to things. What's hard about this is often you need to judge between what is "good", what is "better" and what is "best". In order to do that which is "best", you will, you must say no to many, many things that are "good" and "better". This is heart wrenching work, but choosing what you do now to remain focused and finishing, this is your competitive advantage. When asked what work he was most proud of from among his work at Apple, Steve Jobs famously said, "All the products we didn't ship." Many people and businesses talk about focus and priorities, but very, very few actually finish the idea and follow through with the well executed decision making and focus required.

“You must always work not just within but below your means. If you can handle three elements, handle only two. If you can handle ten, then handle five. In that way the ones you do handle, you handle with more ease, more mastery and you create a feeling of strength in reserve.” - Pablo Picasso

Keep Moving Forward!

There is a scene in Disney's animated movie Meet the Robinsons that I love. The story is of a young boy inventor who is learning. During this particular scene he is trying hard to fix a peanut butter and jelly gun, used to automate sandwich building. Everyone is watching him and it looks like he's going to succeed, finally the time comes to try his fix. The whole thing explodes sending peanut butter and jelly everywhere and onto everyone in the room. He is devastated, but immediately he hears cheers and people start to comment on what a great failure that was! "You Failed!" "And it was awesome!" "Exceptional!" "Outstanding!" "Uh, I've seen better." "From failing you learn, from success, not so much." They congratulate him like he succeeded. They ultimately propose a toast to his brilliant failure. He is stunned. The motto of this family is: Keep moving forward!

This is a good motto for anyone working with software. The challenges are so great and the problems so complicated and frequent, you simply must have the determination to keep moving forward, to and through the finish. Start small and build momentum and keep finishing small things, just to keep in the habit of it.

This is not always easy. Sometimes I would come home from work so frustrated with how slow things were going and how little progress was being made, I'd tell my wife I just needed some time alone to cool down. I'd go into my room, open my laptop and write a blog post. I'd post it and point to it while saying to my wife, "There, I did it. I produced something today! It may not be much, but at least I produced something tangible!" You've got to keep in the habit of producing or finishing. You can't let those muscles atrophy.

One of the best development techniques I've seen over the years is test driven development. The pattern is to build a small test that represents an improvement you want to make to your program. Once the test is built, run it and watch it fail. Then write just enough code to make the failing test pass, then run the code and watch the test pass. Repeat. This tends to lead to low coupling and good cohesion and a reasonable test bed. The real hidden value is regular focus on tangible completion in a consistent way, over time. Just keep moving forward step by step and then after some time you'll be impressed as you look back on the mountain of work you've accomplished by such simple means with constant effort.

There are bugs to be fixed, old code to re-examine and refactor, performance problems to analyze and improve, build automation, test automation and website improvements, help docs to write, blogs to read, posts to write, ideas to explore, customers to contact, emails to read and write. Your job is to choose which of all of these you will do now, and then keep moving forward. Those who master the art of consistently and sustainably producing value, are setup for success. Be one of them. Don't quit. Don't stop. Focus on consistently finishing something of value, no matter how small. That's what the world will pay you for, and since so few seem to stick to it, there's plenty money in play for those who pay the price to consistently finish the job.

iPhone Applications

When Apple announced that the "sweet" iPhone Software Development Kit was going to be AJAX and "Web 2.0 stuff", most developers scoffed. A few went to work, and now we are starting to see some web applications built specifically for the iPhone.

First up is a very nicely designed shopping list application named OneTrip. It was the first iPhone application released as far as I can tell. It's also my personal favorite.

Then 37Signals Ta Da List web app simply senses if you are hitting their website from an iPhone and adjusts to look like this:

OmniGroup is working on a version of their getting-things-done application OmniFocus for the iPhone and they have some screenshots online here.

Notice how these iPhone web apps follow or try to follow very closely the iPhone look and feel. This is a testament to how these developers really get the "build the experience" ethos. That said, even with all this focus on looking "iPhone-like" without the smooth transitions, it's really quite a different feel.

An attempt at a full list of all iPhone web apps can be found at iPhone Application List, but be prepared for some ugly looking iPhone apps. I think the delay in releaseing a "real" iPhone SDK may actually help cement the iPhone experience. With a solid customer expectation of what it means to be an "iPhone app" this could actually lead to an better user experiece with all 3rd party iPhone applications long term. Let's hope!

Apple's official documentation and guidelines for developing for the iPhone are located here: http://developer.apple.com/iphone/

The Cool Kids

Everyone has their own favorite language. Steve Rowe just pointed me to this funny my-language-is-better-than-yours graph. I found it really funny. I think Objective-C is just above C++ but below C. Where are you? ;-)

On Infrastructure

The following quote is from a long article on Toyota in the New York Times:

Improving efficiency in the factory, though, doesn’t necessarily lead to greater profits. Savings on the assembly line can mean a nicer dashboard without making the customer pay more for it. “If you’re efficient in the things the customer doesn’t see, then you can put it into the things the customer does see,” Ron Harbour, a consultant whose company rates the efficiency of auto plants, told me. A result is a car more popular with customers. Success on the assembly line, in this way, begets success in the showroom.

For me, this best describes the business case for choosing Cocoa as your application framework for Mac applications. This is not to say that you can't write terrible Cocoa applications. You can, to be sure, but in general the APIs, the design patterns, even the style of the code subtly try to keep you from rebuilding things that have already been built. Many of the programming idioms try to get out of your way and "do the boring stuff" so that you can spend time adding value that the customer actually sees.

If you are always spending time working on the guts of your application, especially the parts that the user doesn't directly interact with, there's often little perceived value in the work you are doing. If you are all of the time "platform building", it's easy to loose sight for what the platform is being built to support. Don't get me wrong, I'm a big fan of infrastructure work, I've spent most of my professional career doing it. However, it's absolutely critical to see the connection from the work you do to the customer value. This is why a good user interface is so important. This is why a good application programming interface is so important.

On the other hand, if you are working with people that watch out for the schedule and bottom line and you propose an infrastructure improvement, immediately you will, or should be, accosted with pointed questions about cost, time and customer value. All of these are very good things to discuss, but when you do discuss them, you must not forget the long term impact of the work you do. If you're only going to be there for one version of the software, then maybe the near term results are all you care about. There are as many shortsighted programmers as clueless business men, but I think the real answer is that working on improving the guts, the engine, the non-visible, un-photoshopable parts of your application are the long term the "critical path features" that will allow for money and time to be spent on the high customer impact features of version n+1.

Toyota’s executives recognized early on that improving the process by which cars are designed and built is just as important as improving the vehicles themselves.

You've got to have both, but if you only ever focus on what the user sees (and this is easy to do!) eventually, your application will collapse under its own weight. What I'm trying to discribe might be the best business case for the model-view-controller design pattern. How does this apply to Cocoa? Sure, you can build an app using the model-view-controller design pattern without Cocoa, it's just that with Cocoa, you just fall into doing the right thing long term, even if you don't quite know why.

Coding Blogs

This is part 3 in my "Blogs I Read" series. I hope you find it useful.

ADC Headlines - Apple - Feed

I find it really useful to keep on top of the new documentation that is added to Apple's Developer Connection website. I'll often find something new I didn't know, but mostly just seeing a "how to" or bit of sample code is enough to jog my memory when I come up against a similar problem.

Big Nerd Ranch Weblog - Big Nerd Ranch Folk - Feed

The Big Nerd Ranch started as a week long immersive training experience in Atlanta, Georgia. You focus on code intensely and remotely. The one fee covers food, lodging, transportation to and from the airport. It looks like they now have a "ranch" setup in Rome, Italy. I subscribe mostly because, well, I'd like to attend there the Cocoa training some day. Also, I think it's a great idea for helping people focus. I've heard that out on The Ranch, there's no internet connection and poor cell phone reception. It's amazing what cutting out distractions can do for a learner's capacity to gain understanding. I've been to a lot of training, but nothing like this, so I guess it really appeals to me. To top it all off, Aaron Hillegass author of arguably two of the best Mac OS X programming books, is one of the Cocoa instructors there.

Cocoa Dev Central - Scott Stevenson - Feed

This is my favorite learn-about-Cocoa site. Scott Stevenson has a clean, clear style and knows how to remove the unnecessary details making his Cocoa tutorials content-dense and easy to read. The site is well designed and just looks great and I admit, for me, that counts. It is really a one stop shop for learning about Cocoa for the beginner and advanced developer alike.

Mac OS Forge - Apple - Feed

What can I say here? I'm interested to see if Apple can make their second try at open source community work. I hope they can figure it out.

Mac OS X Internals - Amit Singh - Feed

Amit has been publishing extremely interesting and low level details about Mac OS X since at least 2003. I've always enjoyed his writing and recently he published a book named Mac OS X Internals. He doesn't post very often, but when he does, it's always worth the read. One of the main reasons I value Amit's writing is the way he contrasts and compares the Mac OS with other operating systems. If OS level stuff interests you, you'll love this blog.

Martin Fowler - Martin Fowler - Feed

Martin Fowler has been around for a long time, so his seasoned opinion holds a lot of weight for me. Seems like I am continually discovering "new" ideas and programming or computer science concepts that were discovered 20 years ago! He talks about software architecture, object-oriented analysis and design, refactoring, Unified Modeling Language, software patterns, and agile software development methodologies.

TextMate Blog - Allan Odgaard - Feed

And speaking of old ideas revisited, Allan has taken ideas from the venerable emacs command line editor and put and UI on them and then added some great new ideas of his own. Just when it seemed like BBEdit had the whole text editor market tied up, along comes TextMate and really ups the innovation ante. I'm a fan because: 1) I'm learning about the editor and the blog often shows me how to do stuff I didn't understand before and 2) I love rooting for the underdog. :-) If you spend a lot of time editing text on a Mac, you've absolutely got to check out TextMate.

WWDC Sessions on iTunes

I just got this email from Apple: What can I say? Apple, you just made my day. (And pushed me that much closer to a Video iPod...) It looks like there's more content on the way. Did I mention since I attended the conference, all of this is "free"? It will be interesting to see if ADC members who didn't attend will be able to purchase this content. I sure hope so. Update: For a direct link to the iTunes area, click here

C4

Jonathan Rentzsch is putting on a small developer conference called C4. For various reasons I'm not able to go, but oh how I want to. I'm SO glad it's going to be re-broadcast like Evening at Adler was. Any how, I just found out that one of our Office developers, namely Olof Hellman, is making the trek all the way from Redmond to Chicago for the conference! I'm so jealous. The good news is that he'll be blogging about it on our team blog, Mac Mojo, so if you haven't subscribed to the Official MacBU blog and are interested in C4, I'd subscribe to the blog just for Olof's guest posts while at the conference.

Using Scrum in MacBU

Today marks the official beginning of sprint number 2 for the Automation Team. Last month was our first attempt at a modified Scrum. I mention "modified Scrum" simply because of the cruel fact that I don't know everything there is to know about the Scrum Methodology. We just kind of picked out what made immediate sense and did it. It's a good change and we are learning. While our team is the first team in MacBU to be using this Agile process, hundreds of teams at Microsoft have had great success with it. We've had daily standup meetings for a long time, but this was the first time we actually did the product backlog and sprint backlog so I thought I'd record some of my personal reactions to the experience. There's a lot to like about what we've experienced so far, and as we figure out better how to apply this "Agile" stuff, I'm hopeful things will get even better. Here are some of my first impressions about the process: The backlog provides an awesome communication vehicle. Before the beginning of September the four of us on my team got together and generated a big long list of all the things we would like to do. We put all of this in OmniPlan, and then selected a subset that we would tackle in September. This did several very good things:

1. It got us all excited about the ways our team could contribute to the overall success of the software products we produce.
2. Helped us "get on the same page" with respect to the meaning of the individual items. You'd be surprised how different people can interpret even the shortest sentence!
3. Allowed everyone on the team to see exactly what everyone else had to do.
4. Allowed me, as a lead, to post the sprint backlog in the Lab for everyone to see. If folks wanted to see what we were working on, or how much progress we were making, it became very trivial. Perhaps low tech, but effective.

Developers are less randomized and more focused. Part of working on an internal team that builds software for folks sitting in the office next to you is that you get a lot of requests and lots of feedback. This is good, but the flip side is it can also lead to lots of interruptions and cause your efforts to be spread so thin that your effectiveness suffers. With our focus set in the Sprint backlog, new requests simply wait until the next sprint. All the devs get to remain heads down getting stuff done. Cross Team Collaboration Improves. With the sprint backlog in place, when someone or some team comes to us to ask us to "Do X" we can immediately respond, "That's a great idea, does it need to happen this sprint?" Almost always, it can wait, and this does two great things:

1. It allows time for the customer to really think about the request. By the time the next sprint rolls around what was life-and-death-urgent is now better thought out and prioritized more realistically.
2. It gets our "customers" in sync with our rhythm of delivery. The theme becomes, "Get your ideas in the next sprint's backlog, and you'll see some action on it in a month." All the lobbying for changes and design discussions about what goes in next will happen with the Product Backlog owner (me in this case) while the rest of the devs are uninterrupted.

High quality things get done! This seems maybe a bit silly, but at the end of the sprint, even if you way over estimated your production capability, (which we did) you have something to show for your efforts. Not just working software, but reasonable easy to understand metrics (like units of work per day) that everyone can understand when you need to explain why it will indeed take 3 weeks to deliver a high quality solution next sprint. Things get better now. One of the most interesting things about this whole process is how it elevates and exposes problems in the stuff you are doing. As you look at the backlog and compare your velocity of production to what you thought you could do, immediately you and everyone else begin to consider, "Why does this take so long?" On the last day of the sprint, (last Friday for us) we got together in the Cafeteria and took some time just to reflect on how things went, what slowed us down, and what can we do better. What's great about this is you remember what you were doing and then we add to the next Sprint items that will increase our velocity. As Henry Petroski has said in To Engineer Is Human: The Role of Failure in Successful Design, we learn more from our failures than our successes. But only if we pay attention to the failures and figure out what to do right the next time. This is so much better than waiting until we ship Office to have our "Post Mortem" to discuss how the release went. Problems are fresh, and fixing them right away have a good chance of paying off in the current product cycle. All in all, I'm very happy with how Scrum is working for our Team. For October, the Tools Team and Lab Team are joining us in testing out the Scrum process. I hope it works out for them, as well as it did for us. P.S. If this kind of software development stuff interests you, there's a great overview talk I recently found by Ken Schwaber which was produced as part of the Google Tech Talk series. I really enjoyed it. Lastly, if any of you have any sage advice for a Scrum Master in Training, I'd love to hear it.

Mixing Carbon and Cocoa

Oh now this is so, SO refreshing and from a "Cocoa" developer no less! I don't know how many times I've heard the comment that we've got to re-write all of Office in Cocoa as if that were some magic pill that turns all Mac applications into "pure" Mac applications.

Wake up people! It’s 2006. It doesn’t matter what you program in, it’s how you get the job done. Arguing about whether to use Carbon or Cocoa is like arguing about whether to use a net or a hook to catch a fish. You use whatever the circumstances call for. If you don’t, you die. (This, from the vegetarian, city-dwelling Mac programmer).

Once again, it's all about execution. Now I love Cocoa (We use it internally to build all our GUI based tools) I love the design patterns and consistency, and I love CoreData and bindings, but very often if you want to do something cool, you almost always have to drop down to the C API and well, more often than not, you're using Carbon. Remember what Steve Job's said when introducing Carbon? "We named the API Carbon, because that's what all intelligent life forms are made of!" I'm personally looking forward to the end of the whole Cocoa vs. Carbon debate. Sometime in the future, applications will be judged by what they enable you to do rather than the API under the hood. If you want to get that concerned about the nuts and bolts, consider this the next time you fly: Every part that makes up every plane, came from the lowest bid supplier. Life's too short, find the right tool for your job and enjoy the ride!

Web Development for My Dad

This is kind of a test, since I don't know what kind of expertise exists with you my faithful reader, but I need some advice, or rather, I need some advice for my Dad. He's 55+ and looking at getting back into the workforce and one option that has piqued his interest is web development. The question is this: What should he use as his web platform and why? Options that I've looked into are: Ruby on Rails on Mac OS or Linux WebObjects on Mac OS ASP.NET on Windows PHP on Mac OS or Linux Witango on Mac OS or Windows Note: I'm sure I've missed some others, but I think these are the big players. Keep in mind that he's used the Mac OS all his life as the basis for several of the businesses he's started over the years, so he'd like to stay developing on the Mac if at all possible. But it's not the end of the world if he's got to switch platforms. This has got to be a question others have asked, so if there's some web page that answers this directly, I'm sorry I haven't found it yet, but please leave the URL in the comments. Am I the only one that finds making a rational comparison in web development platforms difficult?

David Anderson: Software Development and the Theory of Constraints

I just watched a video on Channel 9 where Robert Scoble interviews David Anderson about the work he's doing at Microsoft. I thought it was super interesting and thought you might enjoy it as well. The movie can be downloaded here. The following are some of my favorite quotes from the video:

My work focuses on applying the teachings of two management science gurus, one is Eliyahu Goldratt and other one is W. Edwards Deming. Goldratt has something he calls the Theory of Constraints which basically says if you can identify the bottle neck in your process your should focus all your management attention, all your investment dollars in alleviating that bottle neck in what ever fashion is appropriate. And he has some guidance on how to do that. Initially that started in manufacturing, and then he had a solution for project management, there was one for distribution channels and so on. I took all this theory and figured out how to apply it to software engineering. Some people said, "Well, that can't possibly work!" and, um, it does!

I think if you can reduce the team size down to something small that makes a big difference. If you can put four guys in one room and they can talk to each other all the time in really high fidelity that makes a big difference. But there's a certain scale of things where that doesn't work.

Speaking of domain specific languages as applied to the quality process:

It's a really cool use of domain specific languages in the Team Architect product and it's a fantastic way to actually have quality insurance in your quality insurance group!

Productivity goes up when you focus people on their production rate and don't force them on this conformance to plan concept. And If that's one thing you could change to make a big difference is stop estimating and start measuring velocity.

If I walk through the building at 7:30 at night and the half a dozen people there all have the debugger open and they've been debugging for the last 8 hours, that's a fairly good indication to me that there's a problem.

Good management, good organizational engineering makes for happy well balanced people that have a life and that's what I think we're all looking for optimally.

He has a blog at Agile Management Blog and his book is named Agile Management for Software Engineering: Applying the Theory of Constraints for Business Results. I just added it to my list of books to read. P.S. If you want to view this on the Mac, download the free Windows Media QuickTime plugin Flip4Mac.

Veni. Vidi. Codi.

I just got my WWDC 2006 T-shirt and it has just these three words on it:

Veni. Vidi. Codi. Does anyone know what they mean? Is this some Latin thing I should understand? It seems like it should mean: I came, I saw, I coded, but I'm not sure. I thought I'd run it through the translation widget, but no Latin, so I tried Italian and all I got was: Any ideas? Why would Apple code this in another language? Update: This cool latin phrase was totally lost on me, but oh well. Here's the info, and the translation was "I came, I saw, I conquered". May you all be the wiser.

Software Factories

Now this is interesting. Jack Greenfield and Keith Short take on the future of software development:

Total global demand for software will grow by an order of magnitude over the next decade, driven by new forces in the global economy like the growing role of software in social infrastructure, by new application types like business integration and medical informatics, and by new platform technologies like web services, mobile devices and smart appliances. Without comparable increases in productivity, total software development capacity seems destined to fall far short of total demand by the end of the decade. What will change to provide the massive increase in capacity required to meet demand? It is not likely to come from adding developers. Instead, software development methods and practices will have to change dramatically to make developers much more productive.

Read the whole article here.

Martin Fowler has a whole essay on this topic: Language Workbenches: The Killer-App for Domain Specific Languages?

Most new ideas in software developments are really new variations on old ideas. This article describes one of these, the growing idea of a class of tools that I call Language Workbenches - examples of which include Intentional Software, JetBrains's Meta Programming System, and Microsoft's Software Factories. These tools take an old style of development - which I call language oriented programming and use IDE tooling in a bid to make language oriented programming a viable approach. Although I'm not enough of a prognosticator to say whether they will succeed in their ambition, I do think that these tools are some of the most interesting things on the horizon of software development. Interesting enough to write this essay to try to explain, at least in outline, how they work and the main issues around their future usefulness.

And to round out the discussion Neil Davidson responds to Steve Cook with this insightful comment:

As much as I find the technical side interesting, the thing which really fascinates me is how the way people write software will change in the future. ... I've been thinking about it a bit, and I think that although the analogy with the changes in industrial manufacturing (from craftsman to mass production to mass customization) is interesting, I'm not sure it really holds true.

I think one of the key things you mentioned was the analogy to the supply chain, and how this chain will lengthen. The way I see it, software will always need a craftsman at one end. Software is intrinsically hard to do, and requires people, or teams of people, to think very carefully and deeply about what they're doing. The tools, processes and components they use will have to change though - it is at this point in the supply chain that I can see mass production happening. I think the analogy is that you're always going to need craftsmen like carpenters and bricklayers to build a house, but the tools, techniques and materials they use will be mass produced. At the moment we're at the stage where the bricklayer makes his own bricks, and the carpenter cuts his own trees down. I don't think the craftsman will be replaced, but the tools he uses will be provided by companies who provide (or are) software factories. That's the point I was trying to make about a few companies dominating the market - somebody will discover that they can produce an e-commerce software factory and sell hundreds of thousands of the things at $1,000 a piece rather than two or three a year at $100,000 a go (because it's no longer a problem constrained by people's time). Presumably Microsoft believes it will be them and that's possibly part of the reason why they're entering the CRM, accounting and business markets.

Does Apple see any of this? Do they think it all too distant to consider currently? Either way, what an interesting time to be developing software!

Composition Arts

In a kind of déjà vu moment, two of my favorite blogs posted about the same thing, on the same day! The topic: How writing code is similar to writing prose. Christina Wodtke of Boxes and Arrows takes E.B. White's “List of Reminders," from The Elements of Style and compares them to web design. Matt Linderman of 37 Signals compares the stages of the writing process to that of software development. I think there's a lot to be learned with these comparisons. Without a full understanding of how construction or manufacturing actually occur, it's easy to compare software projects with construction projects. I think there's probably more similarity between writing software and writing prose than manufacturing or construction.

ExCodeWarrior Schwag

I've written before about our ExCodeWarrior dev shirts we got as reward for successfully moving our code from CodeWarrior to Xcode. Well, now you too, can pay your respects to CodeWarrior, in mournful black, as well as honor the great work the Xcode team has done, and all this in time for WWDC 2006! I'll have my ExCodeWarrior shirt on, will you? ;-)

My WWDC Wishlist

After reading John Siracusa's hilarious post about WWDC Buzz Word Bingo, I got to thinking about what I'd most like to see at WWDC. Here's my list:

1. Make it brain dead simple to write multi-threaded applications.

How simple? If my application runs fast on a dual core machine, I want it to run twice as fast on a quad-core, no recompile. Too difficult? I'd settle for automatically routing all AppKit method calls to the main thread only, so anything the user sees with their eyes is on the main thread, but everything else happens on other threads, no extra coding by me.

2. Widescreen iPod phone

First and foremost, make it a great phone. Second an iPod. Third, and a distant third at that, a platform I can build upon.

3. Video Airport Express

Encode TV signal to MPEG 4 and send to my Mac's disk drive in real time while I watch another channel streaming from my Mac.

That's it. I'd be happy with that.

P.S. If you are going to be at WWDC this year, drop me a line. I'll be there, and I'd love to chat!