The goal of both the Design Signatures app and the Dear Design seminar described on this website are to provide opportunities for designers to reflect deeply on their design process. One possible objective for designers is to move towards design behaviors demonstrated by expert designers. For that reason, we (Cindy Atman and colleagues) conducted a large number of experiments using research methods from cognitive science to understand the design processes of designers with different levels of expertise. Here we present a brief summary of the findings from that research. For more details about our research over the past several decades, please see the papers listed at the end of this text.  

To summarize the design expertise research results, as an individual moves towards more expert design behavior, they:

See Figure 5 below for graphic representations of these findings.

These findings come from a deep exploration of design processes used by people with various levels of expertise.  We gathered verbal protocol data from a large number of people who solved design problems while talking out loud. The goal of this work was to understand how individuals with different levels of expertise engaged in design processes. The differences we could see between novice and expert designers would enable us to develop ways to teach about expert design behaviors and how designers can think deeply about the broad implications of their designs.

Here I will briefly describe the findings from a subset of these data, 69 individuals who designed a playground for a fictitious neighborhood. They each individually solved this paper-and-pencil design problem in a lab-based setting in 2 to 3 hours. We evaluated their solution quality and transcribed what they said verbatim. We then segmented that text into phrases, and assigned a design activity code to each phrase to capture what part of the design process they were engaging in. The design activity codes were a way to describe the actions a designer took at a given point in time. The codes we used were generated by looking at engineering design textbooks. The final codes used in this work can be seen on the left in Figure 1 (note that the first and last codes, “identify a need” and “implementation” are grayed out because they were not used in this design task.) 

We collected data from 26 incoming first-year engineering students, 24 graduating engineering students, and 19 engineering design experts.  Looking at the student data we found that the graduating seniors had higher quality designs and more sophisticated design processes.  As a way to visualize these results, we created timelines from the design data for each participant. A sample timeline is presented on the right in Figure 2. There is one design code for each line in the figure. Time goes from left to right, and a tickmark is put on the appropriate line when a participant says something related to each code. The example shows that a slash was put on the “information gathering” line when this participant said “Hmm, do you have a list of materials.” The timeline representations are helpful displays to identify patterns of time allocation across design activities.

Figure 2 gives a sense of the overall findings as it presents three first-year and three graduating senior timelines (one each for a low, medium and high quality playground design.) Here we focus on the process of two students: the first-year student with the middle quality playground solution and the graduating student with the high quality design. The first year student with the middle quality design did not spend much time at the beginning doing problem scoping (gathering information, generating ideas) and spent the majority of the time in solid blocks of modeling (or prototyping) without much iteration or transitioning to other design activities. 

In contrast, the example graduating senior who created the high quality playground spent a significant chunk of time at the beginning of their process in problem scoping—gathering information and generating ideas as they endeavored to understand the problem they were solving. They continued to gather information throughout the process—a rich set of transitions and iterations throughout their design process demonstrates their agile movement across design activities. This student displayed an overall distribution of activities that goes from the upper left to the bottom right of the timeline—what we are calling a “cascade shape”—as presented in Figure 3.  When presented with this data in class, one mechanical engineering student sketched the shape and called it an “ideal project envelope.”

Figure 4 adds timelines from three experts to the display. Here the data show that there is less variability across the three example timelines. On the whole, the experts spent longer solving the problem than the students did. They also considered more possible solutions to the problem and scoped the problem more effectively by gathering more information and covering more categories of information than the students.

Figure 5 uses the high artifact quality graduating senior's design timeline as a canvas to present the overall findings of this research. Here we highlight specific behaviors that demonstrate moving towards more expert design behavior.

These timeline representations can be a useful tool for design educators. Design processes are typically taught with traditional box-and-arrow diagrams as displayed on the left of Figure 6. These types of diagrams are helpful ways to understand the overall design process. Adding the timeline representation allows readers to see an example of a specific design process, making abstract concepts more visible. Figure 6 displays how the concept of iteration is conveyed through both a traditional design model and a design timeline. Students have found the timelines to be very effective to learn about expert design behaviors and help them plan their future design processes.

The concept of “design signatures” that is the basis of the work on this website grew out of the design timelines. Figures 2 and 4 demonstrate that each individual solving a design problem has a unique tracing of design activities. This tracing, seen as timelines in this work, can also be seen as a “design signature” for that particular instance of design. Thinking about one’s own design signature across design problems can provide opportunities for reflection as well as opportunities to compare one’s own process to expert processes. The Design Signatures app enables designers to track their own process and see the timelines, or signatures, that result. Topic 7 in the Dear Design seminar gives students the opportunity to represent their design processes through the lens of the design expertise research results. Student postcard representations can be seen here.  We hope that you enjoy exploring each of these sections of the website.

So many people commented that the timelines look like musical scores that we sonified them. In Figure 7 you can “play” the timelines and listen to the differences in the processes of a novice first-year student and a more sophisticated process of a graduating senior with more experience.

One of the authors (Cindy Atman) started this research program with funding from the National Science Foundation in 1992. Jennifer Turns joined the team when Cindy moved to the University of Washington in 1998. Over the decades, we have had the privilege to work with amazing collaborators, co-authors, and research team members including Robin Adams, Arif Ahmer, Brad Arneson, Theresa Barker, Maria Buan, Emma Bulojewski, Mary Besterfield-Sacre, Jim Blair, Carie Bodle, Laura Bogusch, Jim Borgford-Parnell, Karen Bursic, Ryan Campbell, Monica Cardella, Soomin Chang, Justin Chimka, Dharma Dailey, Kate Deibel, Zach Goist, Brian Hayes, Melissa Jones, Aaron Joya, Allison Kang, Deborah Kilgore, Kristina Krause, Vipin Kumar, Alex Lew, Terri Lovins, Stefanie Lozito, Janet McDonnell, Kenya Mejia, Annegrete Mølhave, Andrew Morozov, Susan Mosborg, Carie Mullins, Heather Nachtmann, Wai Ho Ng, Will Richey, Eddie Rhone, Axel Roesler, Wendy Roldan, Jason Saleem, Giovanna Scalone, Kathryn Shroyer, Elvia Sierra-Badillo, Shaunte Smith, Roy Sunarso, Steve Tanimoto, Jennifer Turns, Hannah Twigg-Smith, Cheryl Wang, Ken Yasuhara, Mark Zachry, and Eileen Zhang.

This work was supported by National Science Foundation grants 9358516, 9714459, 9872498, 012554, 0227558, and 0354453; the Center for Engineering Learning & Teaching at the University of Washington, the Helmsley Charitable Trust, the Mitchell T. and Lella Blanche Bowie Endowment and the Mark and Carolyn Guidry Foundation.