Nielsen: The Myth of the Genius Designer
Jakob Nielsen has recently written to debunk the idea that all you need to do is hire a Genius Designer, and all will be well. Certainly he’s got a point. But the longer I work in an agile development environment, the more caveats I find. (OK—I, a former usability engineer, have officially drunk the Agile Design Kool-Aid.)
Nielsen starts his Alertbox article with, “I often hear the following argument against usability: Just hire a great designer, and you don’t have to worry about that pesky user testing.” He moves on to rebut that argument with some good points. I won’t re-hash all of them here, but some of his statements that need qualification.
“You must run your projects with the team you actually have, not the team you wish you had.” His point is that you may not have the luxury of working with a Design Genius; therefore the designs might have more usability problems. Sure. But “the team you actually have” might not include a dedicated usability person. Doing the best with “the team you actually have” in many cases means doing a solo act. That doesn’t necessarily mean no user research or testing, but it may very well mean another hat for designers to wear, at least some of the time. Since Nielsen has historically been a big proponent of programmers conducting their own usability tests, surely he would agree that designers could also perform this function.
“Design is an inexact science; even if you have a superb designer, not all of his or her ideas will be great.” I guess that’s true. It’s also true that not every usability engineer is created equal, nor are all studies by a given usability professional of equal validity or usefulness. But all in all I guess it’s a valid point; Good Designers can certainly produce Bad Design some of the time.
Nielsen also states that designers need empirical data on their designs to get good at design and that designers produce successful products only if their designs solve the right problems, which would require user research (my paraphrase).
Yes, testing provides some data, and for some kinds of products and development processes, it’s probably the best way to get it. But it’s far from the best data, as anybody who does field research is well aware. Real-world use is not the same as in-lab use. Under the right circumstances, the real world can provide your data instead of lots of pre-release user testing.
Before we begin development, the designers here at the Church meet with the customer, nail down requirements for a reasonably small but meaningful chunk of functionality, show them working prototypes soon thereafter, then build the real thing.
After it’s built, the customer runs through the working software and accepts it or sends it back for refinement. Because many of the products I work on are for internal release to a small number of users, a very high percentage of our user base has already seen the prototypes and given input before it is built. Furthermore, we are able to release new software about every 2 months or so to these internal customers. We start getting feedback on anything we missed pretty quickly, and have the opportunity (at our customer’s discretion) of fixing the problems and releasing the solution in another 8 weeks or less, along with our next batch of new features. You could argue that customer interviews and prototype reviews are really user research after all, but again—it can be the designer wearing that hat, not a full-time usability professional.
“Nobody’s perfect. Even a very good design can be improved when you follow an iterative process of continuous quality improvements.” I agree. But that doesn’t have to mean testing. Our process is highly iterative, meaning we iterate with prototype reviews and frequently released software, constantly delivering value out to our users where it can really be evaluated under conditions that matter.
I realize that it is not always possible to have easy access to such a large portion of your user base, nor to be able to release working software (and fixes) so frequently. Also some applications have high risk to life, limb, property, or reputation; maybe the higher that risk, the more necessary to run more extensive tests before releaseing—even if you will have the opportunity to release improvements again in a few weeks or months.
But when you have a lower risk level, frequent releases to production, and access to most of your users during development—would a robust usability program provide enough value to justify it? I do wish I had the luxury of running more tests, doing some formal contextual inquiry, and so on. But I don’t think it would be a wise use of resources in my case.
And I’m not even a Genius Designer.