The study suggests that research and development departments switch between new forms of experimentation such as computer simulation and a technique called rapid prototyping, a technology that helps researchers to quickly build experimental models that are then used to test designs. Firms may save millions of dollars in product development cost and time by using such new experimental techniques and methods and finding the optimal point of switching between them.
The findings are applicable to research and development in sectors of the computer, pharmaceutical, and chemical industries.
"Such an experimentation strategy can sometimes benefit from 'getting it [the prototype] wrong the first time," writes Dr. Stefan H. Thomke, a management scientist and faculty member at Harvard Business School.
Tests in $100 Billion Integrated Circuit Industry
The author tested his thesis by compiling data on the integrated circuit based systems design industry. He chose the field because the overall integrated circuit (IC) market is over $100 billion a year and because it has benefited greatly from computer simulation and rapid prototyping. Data on mode switching strategies were collected first through a field study of a local Massachusetts high tech firm and then through a questionnaire that garnered responses from hundreds of designers nationwide.
The study found that an approach like the author’s that utilizes many experimental cycles outperformed its rival by 220% in development effort — and that over 43% of that difference can be attributed to differences in experimentation strategies like the one described.
Dr. Thomke believes subsequent research could show similar benefits when applied, for example, to mass screening and combinatorial chemistry, two forms of experimentation used in pharmaceutical drug design.
Experimental Efficiency
With increased competition and shorter product life cycles, accelerating time-to-market and lowering development cost have become increasingly important to product success, the author notes, making speedy testing techniques especially important.
Relatively inexpensive computer simulation is used as a substitute for "real experimentation" in numerous fields. The analysis of an experiment carried out in simulation, although often less accurate than a "real" test, typically allows experimenters to get much more and better data per experimental run. Getting an experiment wrong in simulation can be enlightening and allow subsequent cycles of the experiment to be designed more efficiently.
Rapid prototyping is used by developers to generate a quick, inexpensive, easy-to-modify model that can be tested against the actual use environment and allow "real" experimentation. Rapid prototyping is often more costly than computer simulation. Compared to developing real prototypes, though, it proves to be an inexpensive and fast way to carry out an experimentation cycle while preserving the advantages of real experimentation. These advantages include higher degrees of accuracy and experimentation speed.
Computer simulation is usually more efficient at building a test model, analysis of errors, and design modification. In contrast, full prototypes are more accurate and, depending on the application, can execute test runs much more rapidly than simulation. Since the number of design errors is usually high prior to starting experimentation, most designers find it economical to start with simulation. As a design progresses and the error detection rate of simulation declines, the efficiency of continuing to simulate at some point falls below the efficiency of prototype testing.
Thus, the author maintains, designers will find it economical to switch from simulation to prototype testing at the exact, optimal switching point where their efficiency trajectories intersect.
Ignoring Good Advice
Although sensible and profitable, the author discovered, the practice is often ignored. During his research he discovered that some integrated circuit designers abandon simulation for prototyping too early because their pre-allocated simulation time has expired. He also found biases among members of simulation and prototype teams and competition between them. These interfered with implementing his recommendations for cutting costs and speeding products to market.
The study, "Managing Experimentation in the Design of New Products" was written by Dr. Thomke, Assistant Professor of Technology and Operations Management, Harvard Business School. It appears in the current edition of Management Science, a publication of INFORMS.
The Institute for Operations Research and the Management Sciences (INFORMS) is an international scientific society with 12,000 members, including Nobel Prize laureates, dedicated to applying scientific methods to help improve decision-making, management, and operations. Members of INFORMS work primarily in business, government, and academia. They are represented in fields as diverse as airlines, health care, law enforcement, the military, the stock market, and telecommunications.