Metacognitive Miscalibration
A few tweets ago, (I always feel weird referring to Twitter in the past tense) I posted: Why are the unintelligent or uninformed so arrogantly confident while the intelligent and well informed so often unsure and apprehensive? There is something very human to thinking you know more than you really do about a subject or issue. While, this problem can be seen in many areas generally it's particularly acute in software development. For example...
AppleScript in Office X
Back when Mac OS X was just about to go 10.0.0 we were busy working on getting Mac Office working on the new platform. There was a lot of pain involved with the transition. Many APIs had been removed and alternatives needed to be provisioned (and tested), the new Aqua interface guidelines had to be applied to the whole of Office and a host of other issues needed to be addressed. I was at the time on the team that wrote the tools for automated testing and I was pushing hard to get Office wide AppleScript support on the list of features we'd commit to doing for Office X. I wanted this not only for our customers, but to augment our testing efforts. With an API to drive the applications we could automate many "smoke tests" on a daily basis as well as set the stage for long term AppleScript based test suites. Automation testing benefits are often not fully realized, especially static automation, until the version after you setup the automation, so I was especially anxious that we get our API in Office X, so we could reap the return on investment for Office 2004.
I remember talking with Jim Murphy about getting AppleScript support like it was yesterday. We were in building 44 and his window office was near the 2nd story walkway that connected our building to the cafeteria. In my naivete I asked why we couldn't just build a some kind of layer to call into the engine code of Office and presto, AppleScript dictionaries for Office would be complete. I pointed to some improvements Apple had made to make AppleScript a better inter-application communication protocol and ask, "How hard could it be?" Jim turned to me and said, "Hard? There's nothing easy about it. It's all pain. All pain." Then as was typical, turned back to his work, leaving me to think.
In this case, I was the unintelligent and uninformed, but very enthusiastic novice. Jim turned out to be right. For Office X, we did try to do the AppleScript work, but it turned out to be much more difficult a problem to solve. After months of work and many dead ends, we pulled the feature from the Office X feature list. For Office 2004, we tried again. In this case, Jim himself, one of our best developers, ended up spending the better part of a year getting AppleScript to work with Office, which is probably worth another post in and of itself. I thought I knew more than I did, I thought the problem was simpler than it was. My confidence and thinking was miscalibrated. The root cause was my lack of understanding and experience.
Wicked Problems
Sometimes, even the best developers underestimate the scope, breadth and depth of a problem. In a drive for simplicity, I observed in myself and others another kind of design time problem where thinking was miscalibrated. The cycle I have observed looks like this:
- Developer looks at a problem A, and thinks he or she fully understands the problem.
- Developer designs a simple solution to problem A.
- Developer codes the simple solution.
- Developer or tester or marketing or customer use simple solution for problem A.
- Bugs trickle in and solution A is modified, bit by bit, bug by bug, until it is patched in a thousand ways and no longer looks or acts like the simple solution. The solution is complex.
Some will say, "Hey! Why didn't that lame developer take the time to really understand the problem? Then he could have taken the time to fully comprehend the complexities of the problem, and then design a simple, elegant solution!" The problem with this is that many subtleties to the problem do not manifest themselves until very late in the development process which make re-architecting the solution for such a small issue un-reasonable. But this is a "death by a thousand paper cuts" issue. What's worse, many architectural issues can only be comprehended after years in a problem space, and as promotions or attrition happen, the so called simple solutions, with their complex instantiations proliferate in code.
Adam Richardson spoke to the challenges of wicked problems when he said:
Wicked problems are very difficult to understand by staring straight into them and looking for clear detail, however. They need to be approached from the edges, sort of like doing a jigsaw puzzle where you find the edge pieces first. Having peripheral vision that is trained to be sensitive to the edges is a key capability (this applies both to product teams and to business units - wherever wicked problems occur).
Normally this causes the developer to tack on simple solution to solutions with patch upon patch applied with the Hippocratic Oath echoing in their ears to "do no harm", but deceptively the bandage is not big enough for he wound, and no one knows.
The Desire to Learn
Another reason why some can feel "informed" when in fact they are not is that they have lost the hunger to learn. They've lost the desire to learn and grow. René Descartes said it this way:
Good sense is the most equitably distributed of all things because no matter how much or little a person has, everyone feels so abundantly provided with good sense that he feels no desire for more than he already possesses.
There is a reason for diversity of opinion, it provides the natural tension that keeps us from thinking we've got the problem solved. When I left Microsoft to go back to school and finish my degree, many asked, "Why are you doing that? What are they going to be able to teach you?" I'll admit that this partially resonated with me. I wanted to be the sage of wisdom, but as I've spent time with teachers and students, many of whom do have less "industry experience", I have learned an enormous amount! To put it bluntly, I have been humbled. I've learned things technically, intellectually, physically and even spiritually. I feel so blessed. I'm beginning to think that what any given situation can teach you depends largely on the person experiencing the situation and very little on the experience itself. Put another way, there's a great difference between 50 years of experience and 1 years worth of experience repeated 50 times.
Personal Pride - the Anti-change Agent
Closely related to the lack of desire to learn is the desire to avoid being wrong. So much in school is focused on being right, knowing the right answer to a test or the correct proof or solution. In sports, no one likes to lose. Everyone likes a winner! But this desire can work into our minds in a limiting way. Leo Tolstoy wrote:
I know that most men, including those at ease with problems of the highest complexity, can seldom accept even the simplest and most obvious truth if it be such as would oblige them to admit the falsity of conclusions which they have delighted in explaining to colleagues, which they have proudly taught to others, and which they have woven, thread by thread, into the fabric of their lives.
At work I would say, "We don't fail half as much as we need to." I still think this way, but now I've found a new reason: Failing keeps our mental muscles and joints from stiffing with pride and forming into the arthritis of the mind.
The Well Intended Deception
Another reason for metacognitive miscalibration I think stems from the very basis of good object oriented abstraction and encapsulation. When I write some simple app with Cocoa's AppKit and Foundation libraries, I am, as they say, standing on the shoulders of giants. I've wondered how many actual lines of code are called when I double click a simple app like TextEdit. Who wrote these lines of code? When did they actually get written? What were the discussions behind their design and implementation? Even Apple couldn't fully answer these questions. It is all this code that even the simplest of Cocoa applications depends upon. But still we continue to have demos around how "few lines of code" were needed to accomplish something. This is a well intended deception, because really, more code is being executed and written, but only now how this code works is opaque. It's well intended because who wants to write, maintain and debug more code? Why not offload this to Apple? I expect these demos to continue, because less code for a developer to write is the canonical example of efficiency, but there's something wrong that comes from this. It's the feeling that you actually really understand what is going on to make your application tick. These assumptions and abstractions can be benign, but they can also make one feel and think that they know more than they do and can do more than easily possible.
In the process of detail management that is software development, making things simpler and more tractable is an excellent goal, but there's a limit to how "simple and easy" things can get and still be valuable. Recently I overheard two students talking about building an iPhone application. The one said to the other, "Dude, they showed this awesome demo of Spore, where in only 2 weeks, 2 weeks! they got this awesome iPhone game running! It's so easy. We can totally build an awesome iPhone app!" First, I love the enthusiasm and I hope they do in fact build an awesome iPhone application. Apple was trying to demo how easy it was to build iPhone applications and I think we've never seen a mobile platform that's better for developers, however:
Taking a senior developer who as spent the last several years of his life building games and developing Spore, bringing him into Apple with full and uninterrupted access to Apple's staff of iPhone engineers that have been developing the iPhone SDK and iPhone applications hardly equates to a college student being successful at all the complexity involved with boot-strapping a company and building a successful iPhone application from scratch. Building great software is still hard, hard work. It's tedious, often un-glamorous and takes someone who doesn't mind getting down and dirty with the details to make something great. Most will simply give up, and even those with the passion and stamina to continue are hardly assured of success. I suppose I'm arguing simply that a healthy dose of Steve Job's famed "Reality Distortion Field" do not a great developer, company or software make.
There are probably other reasons we tend to think we know more than we actually do. But the lesson for me is this: I need to take some time to ponder and reflect regularly. Am I in the "unintelligent or uninformed and arrogantly confident" camp? If so why and how can I get humble? Am I part of the "intelligent and well informed but unsure and apprehensive" group? If so why and what can I do increase my tolerance for risk and decrease my fear of being wrong? A little meta I know, but the title should have warned you. :-)
16 comments:
There is actually research in social psychology suggesting that this is not true only of programmers. As a quite general principle, incompetent people do not realize they are incompetent. A relevant article can be accessed on the web at http://www.ministryoftruth.me.uk/wp-content/uploads/2006/05/DunJohnEhr&Krug.pdf
There's the old quote: to become educated is to move from cocksure ignorance to thoughtful uncertainty.
This reminds me of an interesting study reported in the NYT (http://query.nytimes.com/gst/fullpage.html?sec=health&res=9E03EFD61E3AF93BA25752C0A9669C8B63&fta=y).
The words that stick in my head are "the skills required for competence often are the same skills necessary to recognize competence."
You can probably put yourself into the right camp. ;)
I agree with your observations, but I think that no software would ever be written if programmers weren’t hopelessly optimistic. I started a company and ran it for 9 years because I didn’t know ahead of time how hard it would be. In my current position, I routinely start projects that I doubt (in my heart of hearts) can be successful, and yet they always are (even if success has to be re-defined a few times). The young teams that work for me enthusiastically attack “hard” problems because they don’t know how hard they are.
I am firmly in the category of knowing or at least suspecting how little I know and being cautious, but my customers demand that I spend their money anyway. Along the way, my teams always raise to the challenge, and I learn more, and my experience pays dividends, and the customer ends up so happy that they give me more money and ask for solutions to even harder problems.
Your description of wanting to add Apple script support MS Office with optimistic naiveté’ sounds familiar to me. I suspect that all software projects are started and finished by naïve people.
It's always good to "empty your cup" as Bruce Lee used to say. Good read.
Well said. You have a great voice and it's clear that you've taken the time to think through everything you've written here.
I think we all struggle with periods of listlessness and fighting to maintain the tenacity to learn.
"The more you know the more you don't know."
The fact you wrote this essay is answer enough for your last question.
That being said, I think there's a danger in being too unsure of yourself. Sometimes you need to be able to have faith and confidence in your own abilities. I'm always second-guessing myself, and it certainly pays off, but it also destroys your self-esteem.
The people who write profiles of successful people always say that in practically every case, the person would never have started whatever they did if they knew at the beginning how difficult it was going to be. Maybe ignorance is both a force for good as well as evil.
(I'm a psychologist and do some work in Filemaker.)
Creativity as in problem-solving, is a stressful activity. High creative people are competent, and competent enough to doubt their abilities.
Research has shown that brainstorming with a group of people will actually produce less and poorer ideas than when these people are put apart.
The group however will have a better feeling about the outcome. Thus, although the individual produces better results, he feels a lot worse than the group. So in order to be creative, a person must combine doubt with competence and a low self-esteem. E.g. when he solves the problem, he will possibly feel bad that he did not see it earlier, or will feel that it wasn't all that difficult. On top of that, the more elegant the solution, the more the whole thing looks easy, the less the 'other' or the user…, can evaluate the difficulty. It's like in all performing arts. It seems so easy when you see them doing it…
CB
Brussels
Your first sentence was put so nicely by Yeats:
"The best lack all conviction, while the worst
Are full of passionate intensity."
http://en.wikipedia.org/wiki/The_Second_Coming_(poem)
[quote]
Am I in the "unintelligent or uninformed and arrogantly confident" camp? If so why and how can I get humble? Am I part of the "intelligent and well informed but unsure and apprehensive" group? If so why and what can I do increase my tolerance for risk and decrease my fear of being wrong?
[/quote]
I'm not sure I've spent more time on a comment, in a while, than I have on this one.
If you're in camp A, open yourself to the prospect of learning something new and thank those who have gone before you. Consider your seniors (Senior Developers, Art Directors, CEOs et al.) your mentors, whether they know it or not.
If you're in camp B, learn to disregard that which is not immediately pertinent and focus on that which is immediately pertinent. Live now, and disregard what follows.
Now that I think about it, you probably already know this. Heh.
Chris from Brussels here again.
A pragmatic answer to MARTIN.
If you are in camp A you are incapable to ask this question, you are confident you are in camp B because you just can't see the possibility where you could be wrong.
PS. Please do not use to word arrogant lightly. Here in my language it has almost become the synonym of intelligent.
For those in camp B. You will notice that it is quiet lonely in that camp. You wonder what all the accommodation is for in your camp. Well thats for when the work is done, suddenly they come from everywhere to be part of the rejoice and the credit.
I think "... how can I get humble?" is a great question. I suppose reflecting on past mistakes should be enough, but who wants to do that, who can remember to do it at the right time?
For "... what can I do increase my tolerance for risk and decrease my fear of being wrong?" look up the story of the pottery class from Art and Fear. Help you remember about what you learned even from those past mistakes that you reflected on for humility.
It seems to me that you're ALL :) missing a point: everybody seems to think that it's somewhat "better" to be in camp A than in camp B. But strong leaders or energetic people who are motivating groups etc. don't need to be competent ... Some effective leaders are not even socially competent or specially humble(Larry Ellison and Steve jobs are both notorious A. and egomaniacs)...
Who would dare to swim after being humbled by a course in fluid mechanics? ;)
I was under the impression that competence was used here as creative intelligence, or the ability to solve problems.
It does not mean successful.
Running and winning 100 meters is not really a problem-solving activity. You can be successful in a number of things in which a lot of other abilities play a greater role. Please read the article, link posted by Eliot, for which I thank him.
I suppose another interesting question is, how do we get into those camps and move between them? Why aren't there uninformed people who know they're uninformed? And why aren't there any smart know-it-alls that really know it all?
Perhaps parts of the answer lie in these presumptuous assertions (which may or may not be true).
1. It's impossible to know everything, and everyone knows it. Dumb people think everyone else is lacking but forget themselves; smart people realize they themselves are lacking.
2. If you don't know something, you have a desire to know.
3. If you have no desire to know something, then you think it's not worth your time to know.
4. If you have a desire to know something but don't have time to learn it, then you are already busy learning.
5. Either you are getting smarter, or you are getting dumber.
6. People forget what they don't use.
7. People remember what they use (for a while, anyway).
8. People who love truth always want more.
9. People who are bothered by new information will ignore and make exceptions to what they know until they no longer know it anymore.
Post a Comment