Design by Tangible Stories:
Enriching Interactive Everyday Products with Ludic Value
Department of Industrial Design, KAIST, Daejeon, Republic of Korea
This paper presents a design method that enhances the ludic value of interactive everyday products. The purpose of this method is to help designers create meaningful products in the digital realm while preserving the benefits of technology. There is a great opportunity for design to enhance the extra experiential value of products and systems in a world with digital technologies. We highlight one major way of approaching this opportunity by infusing items used in domestic settings with ludic value. In this paper, we clarify the meaning and key attributes of the ludic value of products, as well as present a review of existing research in the design of interactive everyday products. Based on this clarification and the supporting literature, we propose a new interaction design method – Design by Tangible Stories. Our method has three key features to support the openness and engagement aspects of the ludic value: (1) the creation of narratives about imaginary creatures, (2) embedded serendipitous functions, and (3) physical interaction. Through several design experiments and user trials with our method, we found that everyday products can be transformed into interactive products that have extra experiential value. The design experiments showed that everyday products can be recreated and used as a mediator which supports ludic activities. The proposed method enables us to understand new interaction and functional opportunities in the design of interactive everyday products.
Keywords – Emotional Value of Technological Products, Aesthetics of Interaction, Design Method, Interactive Product Design, Narrative Based Design, Serendipity, Tangible Interaction.
Relevance to Design Practice – This research helps interactive product and system designers create emotional and meaningful objects in the digital realm. The proposed method will be useful to designing new interactive everyday products with ludic value. The design cases and user studies also provide insight into how the method can be systematically applied as a source of inspiration for new design projects.
Citation: Nam, T., & Kim, C. (2011). Design by tangible stories: Enriching interactive everyday products with ludic value. International Journal of Design, 5(1), 85-98.
Received February 26, 2010; Accepted December 18, 2010; Published April 30, 2011.
Copyright: © 2011 Nam and Kim. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
*Corresponding Author: firstname.lastname@example.org
The introduction of calm technologies and ubiquitous computing (Weiser, 1991) brought forth a growing number of interactive digital products. They even created new categories of digital products including digital audio players, car navigation devices and mobile phones. At the same time, they transformed traditional everyday items, such as chairs, tables and kitchen appliances, into smart, interactive products through embedded microcontrollers, sensors and new electronic displays. These new kinds of everyday products and environments shape our lives and have become major topics of design.
One of the most important roles of designers in this situation is to infuse more emotional value into such technological products, being that emotional value in such products can help people be happier and more creative when using them (Norman, 2005). This supports aesthetic user experience beyond goal oriented qualities such as effectiveness and efficiency. Borgmann (1984) argued that technology is removing an important part of human life, which he clarified through defining two terms, commodity and thing. According to his definitions, commodities are objects that have no other significance apart from their main functions. Things, on the other hand, are not just physical objects, but they also include the emotional feelings and meanings that are connected with the object. He was concerned that new technologies might be creating only commodities. People’s lives are made more convenient, but at the price of losing meaning.
The importance of increased emotional value has been highlighted by many design practitioners and researchers. Chapman (2009) explains the notion of emotional durability and introduced a six-point experiential framework. Papanek (1995) uses the term “spiritual values” in response to “materialistic values” and insists that design should consider spiritual values more significantly. People buy products based on materialistic values which the product provides with functions, finished quality and decoration. If the materialistic values are degraded, the product loses these values. However, if extra experiential values are added to the product, it becomes sustainable and highly valued even if the materialistic values degrade. For example, if a dish has been used by one’s family for several generations, even if it has some cracks, it holds immeasurable emotional value for the owners.
It is rare to see users consider digital products similar to how they see conventional products. This may be because of the nature of new digital technologies. The ease of duplicability and the rapid improvements in performance make it difficult for them to contribute extra emotional value that traditional products might have. This may explain why digital watches rapidly decrease in value while analogue watches retain value for long periods of time. This underlines the necessity for designers to explore new approaches that add intrinsic emotional value to digital objects.
The emotional value of digital products is not only concerned with static design attributes but with interactive, temporal and dynamic attributes as well. In the fields of human computer interaction and interactive product design, there is a growing interest in design for aesthetic interaction (Petersen, Hallnäs, & Jacob, 2008; Hummels & Overbeeke, 2010). This is in line with Borgman’s concerns that technology has too much focus on tasks and goals alone. Hallnäs and Redström (2001) introduced the notion of slow technology, which is a design philosophy for technology aimed at reflection and moments of mental rest rather than efficiency in performance. Because everyday products are supposed to be lived with, this philosophy is particularly pertinent as more technological convenience is not always better.
One of the emotional values appropriate for interactive everyday products, and the focus of the paper, is ludic value. The term ludic means playfulness in Latin. In the book Homo Ludens (1955), Huizinga explained that people are characterized by play as much as they are by thinking or tool use. Ludic value is considered to be the property of a design that can address the need for new pleasures during various everyday activities. In this process of pursuing ones need, enlightenment or a new understanding can increase ludic experience.
Domestic everyday objects and environments are representative design subjects that can be enriched by ludic value. Different from tools and devices in work environments where activities take place to achieve specific tasks or goals, domestic environments are emotional and multi-purpose spaces where people live and share presence. Moreover, cultures of different generations coexist and basic living activities take place privately. Gaver et al. (2004) mentioned that many activities in a domestic environment are ludic. This implies that new design opportunities can be found by focusing on ludic values while applying new technologies to interactive everyday products of the home environments.
The main goal of this paper is to help designers create meaningful things, not commodities, in the digital realm while preserving the benefits of the technology. In particular, we present a new design method, Design by Tangible Stories, to increase the ludic value of interactive everyday products. As a research method, we reviewed existing design studies and clarified the meaning and the attributes of ludic value for design. We then developed a framework for our method that explains key features and its application. We conducted design experiments by developing design cases with working prototypes. Two of the prototypes had a user trial to elicit information on the method and the ludic value of the design outcome in a user context.
Design by Tangible Stories has three key features: the creation of narratives about imaginary creatures, embedded serendipitous functions, and physical interaction. These features are intended to support openness and engagement aspects of the ludic value. In the new narrative created by the designer, a product becomes a world where imaginary creatures live. Serendipitous functions are then embedded in the product. Serendipitous functions are not necessarily visible from the beginning of usage and they may often be discovered by users unexpectedly. Physical interaction is used as the main means of encouraging users to interpret the narrative of the imaginary creatures and supporting basic and serendipitous functions of the product. We developed a number of design cases using this method, three of which will be discussed in more detail: Talkative Cushion, Trembling Hug, and Time Door. Their introduction will be followed by a discussion and conclusion.
Recently, research on how to enrich aesthetic user experience with new digital products is increasingly popular, especially in the fields of design (Hummels & Overbeeke, 2010; see also: http://www.utc.fr/dppi09/) and human computer interaction (Petersen et al., 2008). Some researchers focus on building theoretical frameworks, while others report on methods to add new experience qualities and design experiments. Although there have been studies on new methods (e.g. Gaver, Beaver, & Benford, 2003; Landin, 2005, 2009; Lundgren, 2006), the design discipline is lacking more practical methods and techniques applicable for designing new digital products with emotional value. More design case references are also required to show how such methods can be applied and to reflect on the usefulness of the methods and the design outcome in a real use context. In this section, we discuss works related to understanding the meaning and the attributes of ludic value for product design, experiential qualities of interactive products, and methods for interactive product design.
Understanding Ludic Value
The term ludic covers a wide range of creative and exploratory activities. Huizinga (1950) explains that it does not simply mean playing, but includes mental creation activities like meditation, composing music and writing literature. Also, Schiller (1954/2004) used the term “ludic drive” to explain the power that constantly makes humans pursue fun, which is one of the characteristic human attributes.
A great amount of research and case studies have been conducted on the theories, methods and techniques regarding fun, pleasure and enjoyment in design. Jordan (2000) explained that pleasurable products are living objects that people can relate to. They bring not only functional benefits, but also emotional ones. Blythe and Hassenzal (2003) explained that being different from fun, pleasure is not short-lived or spontaneous. Pleasure is a deeper form of enjoyment achieved when people are devoted to an object or activity. It happens when people try to make sense of themselves, exploring and nourishing their identities.
Attributes of Ludic Value
Ludic activities have been studied in the area of play, user experience design, culture and content theories (Pink, 2005; Tiger, 1992; Verbeek, 2002). Ludic activities cover a wide range of creative, exploratory, self-motivated actions and seek to obtain enjoyment, fun, or refreshment of one’s feelings (Gaver et al., 2007). Although ludic activities can be highly related to playfulness, they are differentiated from game play which tends to follow a set of arbitrary rules and has a sense of competitiveness. The ludic value of a product can trigger fluid and self-motivated activities. Inspired by the concept of ludic activities (Gaver et al., 2007), we identified two aspects which are fundamental to increasing the ludic value of a product: openness and engagement.
Openness is the aspect of the product that enables users’ free interpretation and exploration of an object or situation. It allows self motivated interpretation while interacting with a product. According to play theory, people can be satisfied by the mental activity of discharging energy without a specific purpose. Ambiguity is often considered as a means to support the openness aspect of the ludic value (Gaver et al., 2003). In contrast to the purpose of optimization for functionality, it allows people to act for the sake of an activity itself rather than for a specific goal. It is expected that users find ludic value by reinterpreting the meaning and value of an activity related to the use of a product. It can also stimulate a person’s curiosity through inspiring a user’s imagination about various and exciting possibilities of use.
Engagement is another attribute that is highly related to ludic value of a product. Humans can reach an enjoyment stage in usage by concentrating and participating in specific situations. Engagement is also related to the state of flow (Csikszentmihalyi, 1998), as in the mental state of operation in which the person is fully immersed in what he or she is doing. This is characterized by a feeling of energized focus, full involvement, and success in the process of the activity. For example, humans feel enjoyment from immersion in literary or cinematic story development (Douglas & Hargadon, 2000). Especially from the viewpoint of product design, engagement can be obtained from participating actively in a process through interaction with a product.
Experiential Qualities of Interactive Digital Products
Some studies provide a theoretical basis to understand experiential qualities of emerging digital products. Löwgren and Stolterman (2004) describe different use-oriented qualities of digital artifacts, such as efficiency, transparency, control/autonomy, playability, parafunctionality and seductivity. Petersen, Iversen, Krogh, and Ludvigsen (2004) introduced aesthetic interaction to promote aesthetics of use rather than aesthetics of appearance. There have also been studies and design projects on how certain experiential qualities, such as ambiguity (Gaver et al., 2003), curiosity (Dunne & Raby, 2002), fragility and magical interactions (Landin, 2005), and reflection (Hallnäs & Redström, 2001), can be included into design to enrich people’s thoughts and behaviors in relation to digital products. These studies often use domestic products and environments as a main application context. For example, Dunne (2006) showed a series of prototype objects investigating people’s attitudes to and experiences of electromagnetic fields in the home. The objects were made purposely diagrammatic and vaguely familiar. They showed that the new objects with different purposes were open-ended enough to prompt stories. The objects also elicited both factual and imagined stories about the secret life of electronic things.
Gaver et al. (2003, 2004, 2008) also reported a series of designs and empirical studies in domestic environments showing that digital technologies can be used to provide pleasant experiences to users in addition to their basic functions. Drift Table (Gaver et al., 2004), for example, is an electronic coffee table that displays slowly moving aerial photography controlled by the distribution of weight on its surface. Local Barometer (Gaver et al., 2008) is an object that shows local information in a new way dependent on the outside wind strength. Plane Tracker (Gaver et al., 2008) is a device that imagines journeys based on information picked up from passing aircrafts. Local Barometer and Plane Tracker connect the inside with aircraft outside of the house giving explorative experiences to users. These projects show that technology can support ludic activities, which are motivated by curiosity, exploration and reflection rather than by externally defined tasks.
In another example, Thefuntheory (http://www.thefuntheory.com) shows a number of design projects in which a notion as simple as fun is used as an easy way to change people’s behavior for the better. For example, Bottle Bank Arcade Machine is an interactive game that encourages people to recycle used bottles, changing the player’s attitude about recycling by making it a fun activity. Piano Staircase makes taking the stairs an entertaining and musical option while reducing the use of escalators and elevators. The World’s Deepest Bin is another design that encourages people to discard rubbish appropriately in the bin through the fun experience of hearing a falling sound as if the rubbish has been thrown into a deep hole.
Landin (2005) investigated how fragile and magical properties of computational technology, among many experiential qualities, influence acts of use. She introduced new concepts of computational things with these properties, such as a self-destructive digital music data and a playful (foe) computer. She also introduced methodological exercises that can be used in a creative phase of the design process. Soniture and Information Art (Hallnäs & Redström, 2001) are other design examples that aimed at reflection and moments of mental rest rather than at efficiency in performance.
Methods for Interactive Product Design
Existing design methods for interactive products have focused on assisting step-by-step design activities, how to find user needs with different user studies, and how to evaluate a proposed design. For the early stage of the design process, various design methodologies including HTA (Annett & Dunaca, 1967), GOMS (Card, Moran, & Newell, 1983) and Contextual Design (Beyer & Holtzbaltt, 1998), were developed to assist identifying users’ needs and understanding the context of a design problem and the users. Meanwhile, various evaluation methodologies have also been developed for the final stage of the design process. For example, Usability Engineering (Nielsen, 1994) approaches to design, as well as the DECIDE framework (Preece, Rogers, & Sharp, 2011) are methods that can be used for evaluating interactive products.
For the idea development of interaction design, some have suggested tools and methods focusing on interaction styles. Djajadiningrat, Gaver, and Frens (2000) proposed interaction relabeling which is a technique that maps interactions with a known mechanical device to the functions of an electronic device designed to explore new interaction ideas. Additionally, Hummels, Overbeeke, and Klooster (2007) introduced a design tool for movement based interaction.
There is also a series of design methods and tools that apply human or animal traits in the design process. Janlert and Stolterman (1997) proposed that complex artifacts may become easier to use if they are ascribed a certain character. Norman (2005) introduced anthropomorphism into design, which is the attribution of human motivations, beliefs, and feelings to animals and inanimate things. Lundgren (2006) introduced assigning a personality when designing complex systems to help reason about the project and create aesthetic decisions. With their project digital poet, they showed that it is particularly useful in making all aspects of complex systems merge into a unified set of expressions and behaviors. Animal Expression Transfer (Lundgren, 2006) is a design tool that can be used when focusing on the interplay between expressions and interactions. When creating a novel object, the expressions and behaviors of an animal are mapped onto an artifact. The Iron Horse (Landin, Lundgren, & Prison, 2002) is a bike designed to sound like a horse. By biking at different speeds, it will give off sounds as if walking, trotting or galloping. Sometimes it will snort, and it greets its owner and other iron horses with a neigh.
Ross and Wensveen (2010) suggested a design approach called “Designing for Aesthetic Interaction through Aesthetic Interaction,” referring to the use of aesthetic experience as a design mechanism. They showed the design of intelligent lamps outlining the design techniques in three steps. Bell, Blythe, and Sengers (2005) suggested de-familiarization as a new design approach. They argue that because a context like home is so familiar, it is necessary to make it strange, or de-familiarize it, in order to open its design space.
Summary and Reflections on Literature Review
Ludic value is related to the notion of fun, pleasure, and enjoyment. Our definition of ludic value is highly related to the concept of ludic engagement suggested by Gaver et al. (2007) and the concept of pleasure by Blythe and Hassenzahl (2003). Ludic value of a product can trigger creative, explorative, self-motivated experiences. This is different from the play in games. We argue that this value is achieved by adding features of open interpretation and self motivated engagement in digital products.
Theories for understanding the properties of new interactive products are being developed. As such, it is difficult to see united principles and rules that fit all projects. The material and the technologies are changing, and the design terms are used differently in different disciplines. Recently there has been growing interest in experiential qualities of interactive digital products beyond efficiency in use. Such experiential qualities include curiosity, creativity, emotion, reflection and mental rest. In addition, many associated experimental designs are suggested in a prototype or a conceptual level. Popular examples for these experiments are targeting domestic artifacts and environments.
There are only a few systematic design approaches focusing on ludic experiences (Gaver et al., 2004; Lundgren, 2006). It is also difficult to examine the value of these methods in real practice because it is rare to see commercially successful interactive products in a real use context. There is a lack of general and practical methods to be applied to the design of interactive everyday products. Design methods and tools are emerging, but they are developed for a research setting. Among these, design methods using personification, characterization or anthropomorphism are strongly related to the focus of this paper. The mapping of animal expressions or borrowing the traits of people or pets can be useful developing the design concepts of interactive everyday products. However, these methods are highly dependent on the relations between expressions and interactions, as the expressions are often directly mapped. Although it is a useful tool for a creative phase of a design process, it is difficult to be generally applied in a variety of different design projects of interactive everyday products.
Design by Tangible Stories
Our new design method, Design by Tangible Stories, makes use of this understanding of ludic value in designing interactive everyday products. Although it can be applied to novel products, the main focus is on the design of transforming existing everyday products into interactive ones with ludic value. This method is characterized by three key features: imaginary creature based narratives, serendipitous functions, and physical interaction. The features are meant to support openness and engagement, the key attributes of ludic value.
The first feature of this method integrates an imaginary creature-based narrative into a product. Narrative is a significant means that gives a new value to a product (Chapman, 2009). In the case of digital products, it is difficult to include historical and social stories which are often tied to traditional non-digital products. Therefore, we propose to create a new narrative and to integrate this narrative into a product to enhance its symbolic and spiritual value. The narratives used in this method are particularly concerned with that of imaginary creatures. In this sense, the narrative is not a sequential flow of events but a description of appearances, behaviors, habits, identities and other characteristics of the creatures and the living environment. The interpretation of the narrative is open for users. In this method, it is regarded that everyday products are a playground or a living environment for the creatures. This imagination is associated with imaginary creatures which live within the world. The narratives can be made from knowledge and information about the product, as well as the status and emotion of the users. Imaginary creatures, which are the main characters of the narratives, are a means for interaction. This new perspective enables users to reinterpret interaction or to use the product for ludic purposes.
The second feature of this method is attaching serendipity to the function of a product. Serendipity is the faculty of happily making unexpected discoveries. It is the effect by which one accidentally stumbles upon something fortunate, especially while looking for something entirely unrelated. Serendipitous functions can provide unexpected and pleasant surprises that cannot be expected from traditional non-interactive products. These new functions may not be essential but could be auxiliary or additional. They may not be found easily at first, but they can give serendipitous enjoyment and curiosity by discovery during usage. Through this feature, self-motivated engagement can be enhanced over a longer period. It is also expected that everyday products can be transformed into a means for experiencing ludic activities creating symbolic meaning.
The third feature of this method is the incorporation of physical interaction to be used to support ludic value. Physicality can make the interaction more intuitive, emotional, fun and magical (Ullmer & Ishii, 2000). Specifically to our method, as characteristics of imaginary creatures of the narrative become a physical and touchable media, people using the product can have more emotional responses to it. Users’ interaction with the product can have extra meaning when it is combined with narratives.
Openness, a key attribute of ludic value, is supported by both the narratives and the serendipitous functions. Imaginary creature narratives are not explicit when first used. The detailed behaviors and the expressions can be different depending on users’ interpretation. The indirect relationship between the characteristics of the creatures and the product attributes allow users to be more explorative and creative. The serendipitous aspects foster long term engagement with the products and make them emotionally durable. The physical interaction is not the same as the status of the inner creatures and the serendipitous functions emerge over time. This also enhances the engagement aspect of ludic value on a long term basis.
Design by Tangible Stories is different from existing methods based on anthropomorphism (Landin, 2008; Lundgren, 2006; Norman, 2004), although it shares the idea of the narratives about live creatures being used as an inspirational resource for design. For example, with the previously mentioned Iron Horse, the animal expression is directly mapped to the features of a bike. It is difficult to use that design method for a product without directly transferrable features. With our method, the behavior and expressions of creatures are not directly mapped to the interactive features of a product. The product is presented as an environment through which the creatures interact with the real world. It is not necessary to directly map the characteristics of imaginary creatures and the interactive features of a product. This is useful when the method is generalized for different kinds of interactive products and systems. This indirectness allows comparatively more opportunities for open interpretation for the users and it offers more potential as a general design approach. Moreover, users can have different interpretation of the relations between the narratives and the products. In addition, the integration of serendipitous functionalities and physical interaction are additional features that make this method different from existing ones. It is more focused on the transformation of existing everyday products into interactive ones with ludic value.
Application of Design by Tangible Stories
The products designed with our method have the three features discussed above. These features can be independently developed, but better user experience can be achieved when they are closely related. It is not required to follow a step by step linear design process to create each feature. Instead, a designer can start from any feature that becomes concrete and then develop ideas by considering its relationship with the rest of the features toward finally making an integrated whole. The following is an example of how our method can be applied to the practice of design.
The first step is to consider a product as a living environment of imaginary creatures. The designer should develop these creatures fully, and the creatures should be created to give inspiration to each element of the product. The ideas about the creatures can be developed through observation and investigation into a product. At this stage, the name, shape, behavior and character of the creatures are created. Sources from literature or fantasy stories (Ulric, 1996) can be used.
The second step is to create an integrated narrative with the imaginary creatures created at the first stage as main characters. Characteristics and behaviors of the creatures are mapped to the serendipitous functions and physical interactions of the product. As such, the characteristics of the creatures provide the basis for physical interaction and serendipitous functions. Also, by allowing users to interact physically with the creatures, ideas of interactivity can be developed further.
The third step is to include serendipitous functions hidden into the product. These provide enjoyment while users, who at first don’t recognize the special functions, unexpectedly discover them when the product reacts differently. The serendipitous functions can be obtained from the stories developed in the first stage, the features of imaginary creatures, or directly from physical interaction. The function can also give inspiration to other elements of the product.
The final step is to make the idea concrete so that it can be used in physical interaction. In this step, physical interaction is used to express the movement or property of action which the artificial creature has. It can also be used as a main interaction for serendipitous functions developed in the third stage.
It is difficult to validate new design methods in relation to experiential values using the traditional laboratory based experiments or user studies that are often conducted in the field of human computer interaction. We adopted case-based and research through design (Zimmerman, Forlizzi, & Evenson, 2007) approaches from several research methods in the design research field. Through experiments using our method, more than ten design cases, from concept exploration to building working prototypes, were developed with graduate student designers. After initial introduction to the method, the design project took about four weeks. We also observed how some working prototypes were used in a real user context to reflect on the ludic value of a design outcome. Table 1 shows the outcomes that were successfully created from the design experiment. Our method made it possible to transform a variety of everyday products into new interactive products with ludic value. What follows is a presentation of three specific design experiments.
Talkative Cushion is a cushion with the additional function of being an audio recorder (Figure 1). A narrative of having imaginary audio bugs that eat, digest and throw out ordinary sounds was applied to the design of the cushion. The cushion uses recorded sounds as a playful communication medium between family members.
The cushion was designed with the central concepts of our method. The cushion presents an environment where imaginary bugs eat ordinary audio sound. The story about the bugs transforms the cushion into a new interactive product with a narrative. It is also a tangible interactive object that supports asynchronous communication between people, in particular family members at home. Users physically interact with the cushion to record or to play back recordings. As an audio recorder, it is a tool for asynchronous and indirect communication. The audio function with the narrative of sound-eating bugs is a serendipitous functionality that is hidden and provides playfulness when discovered and used.
The Talkative Cushion design began by developing a narrative about sound. The narrative includes the name, appearance, behavior, habit and other characteristics of the creature living in the product. An everyday product was sought to include the narrative and the imaginary sound eating creatures. The idea of a recording cushion was developed with the feature of sound pitch transformation during playback. The physical interaction of pressing the cushion was added to change the pitch and the speed of the playback sound. Design development details about the elements included in the Talkative Cushion are as follows.
Narrative & Imaginary Creature
The narrative of Talkative Cushion is based on the idea that a sound bug is an evolving living creature. We developed a story where there are cells in the world which attach to any material. When activated or stimulated, however, they start to drift in the air. These cells are transformed into sound bugs by reacting with oxygen in the air. These bugs are invisible but exist around us whenever sounds are created. Sound bubble monsters are imaginary living creatures whose main food is sound. When they digest and expel out the food, it creates and transforms the sound. Their states also change depending on the interaction with the external world. When there are no external stimuli, they behave as thinking bugs. When pressure is applied, these bugs turn into speaking ones. When a stronger pressure is applied, the bugs transform into larger more noisy ones (see Figure 2).
A hidden function of the cushion is an audio recording and playback function that is related to the behavior of the imaginary creatures and physical interaction applied to activate the bugs. In addition to simple recording and playback, sound pitch modulation features differentiate it from existing audio devices. The modulation happens when a user records and presses the cushion. For the bugs inside the cushion, that process is the act of feeding and stimulating the bugs so that they eat, digest and throw out their main food. The unexpected audio feature encourages more indirect communication, and supports playful interaction.
The shape of the cushion was inspired from a talk bubble, as seen in comic strips. The shape affords natural physical interaction, such as hugging, leaning and pressing. The tail of the talk bubble serves as a handle to activate the recording function of the cushion. The playback and sound pitch modulation are activated when the surface of the cushion is pressed.
Prototype Implementation of Talkative Cushion
In a finished product of the Talkative Cushion, the hardware system should be a standalone system with a microcontroller, sensor, an audio recording device and a flexible organic display unit. It also requires an embedded software system to drive the hardware and manage recorded sounds. In the prototype, we implemented the desired functions in an experience prototype level with an Arduino microcontroller (http://www.arduino.cc.), flex sensor, electronic sheet, wireless microphone and speaker device (Figure 3). The microcontroller and the audio device were wirelessly connected to a computer to process all recording and playback functions. The sound processing software was developed using Max/MSP (http://www.cycling74.com).
In the audio recording mode, the microphone is activated by holding the tail switch of the cushion. The recorded sound is saved in a memory buffer of Max/MSP software. When the cushion is pressed on the surface, the flex sensor detects the pressure level and changes the speed and the pitch of the playback sound. The electronic sheet was cut and placed in layers to show the animation of the status change of the bugs. According to the sound modulation status and the pressure applied to the cushion, the display of the electronic sheet was changed interactively.
The user trial was conducted in two groups in domestic environments with two or three participants (Table 2). We asked the users to use Talkative Cushion in their everyday living environment for three days. We didn’t give a special task; instead, we just explained the simple operating method. After the field trial, we conducted an in-depth interview to understand how the participants interacted with the prototype and perceived the values of the design elements applied.
The user trials showed new user experiences while Talkative Cushion gave a new meaning to an everyday product through hidden functions. Participants thought that the cushion was a new kind of product between a traditional commodity and an interactive thing. Participants answered the interview question of how they would explain Talkative Cushion to others in different ways. One said that “it is a cushion that becomes a companion to talk with.” Another asserted that the cushion “is a parrot that repeats after me.” It was also reported that the cushion “is a subordinate that repeats after me.” They perceived that the cushion was a new symbolic object rather than a special function-oriented device.
In regard to the interview questions about the purpose of the use, participants answered that they used the cushion for recording playful conversation and for unexpectedly recording the voice of other people to create a fun situation. One participant mentioned that the experience of recording was similar to the experience of drawing scribbles on the wall when he was young. He also mentioned that this cushion made him to say ridiculous and unusual words. These responses reveal that participants used Talkative Cushion continuously and playfully without need for a specific purpose.
The narrative-enriched cushion also inspired a creative and symbolic interpretation of the product and its interaction. Participants suggested that this cushion could be used as a singing companion, or as a means to keep a diary. We also found that users add special meaning to the product by stating using it gave a feeling like “hugging the world warmly.”
Trembling Hug is a haptic cushion that physically reacts to sound. The design is based on a narrative about imaginary vibrating creatures that react to sound. The haptic feedback is hidden as a serendipitous functionality.
The Trembling Hug cushion was designed using the key features of our new method. The story about its imaginary creatures is based on their response to an audio frequency in the real world. Multimodal feedback on ambient sound or connected media such as a movie or music player is added as a serendipitous functionality. Physical interaction such as hugging the cushion, along with its vibrating response, allows the user to interact with the imaginary creatures.
Narrative & Imaginary Creatures
The narrative about Trembling Hug is based on scientific knowledge about audio frequencies, where a material has a unique frequency and creates resonance by external sound. Imaginary trembling creatures were created in the narrative to explain the unique frequency for different materials. In this narrative, the creatures absorb sound frequency and create a resonance effect. The cushion with the creatures vibrates when a creature generates resonance by absorbing the frequency of external sounds.
Three kinds of creatures were created in the narrative. They are different in appearance and in how they react to an external frequency (Figure 4). The physical proportion of their feelers, legs, arms, eyes, and body is different depending on the frequency level that determines the kind of creature it is. The vibration creates living energy for the creature, so they don’t eat. Instead, they need to vibrate regularly to stay alive. This vibration is applied to the material in which the creatures exist.
The haptic feedback is the main physical interaction for the product. The vibration is the representation of how the creature deals with the frequency of their choice. The lights in the surface of the cushion show the status of the eyes of the bugs. When vibration occurs, the eyes turn red indicating a bloodshot status.
The transformation of the modality from sound to vibration gives an unexpected but pleasant satisfaction to the users. The multimodal feedback that reacts to ambient sound is not expected in the cushion. This haptic sound feedback also supports extra immersive user experience when it is used while watching a movie or listening to music.
Prototype Implementation of Trembling Hug
The prototype consists of hardware and software components (Figure 5). The hardware device is divided into two parts. The first part is where vibration is created and the status of the creature is displayed. A vibrating speaker and an array of LED lights are used. The second part is for the control of sound signals and LEDs. It consists of an amplifier for external sound input, an Arduino microcontroller to control the LED lights, and a power supply. The embedded software uses Max/MSP which processes the level of the sound input from the microphone. Depending on the sound level, the LED lights are controlled via the Arduino software. No control is required for the vibration of the sound because it is directly applied to the vibrating speaker.
Three participants used Trembling Hug for about three days in their everyday living environments (Table 3). All three participants are quick to adopt new products and enjoy digital entertainment.
The Trembling Hug let the participants develop new ideas about, as well as engage in new activities with, a familiar living environment being that ordinary cushions are considered as a supplementary object and not absolutely necessary for their daily life. They offered that new questions could easily arise from the functions that the cushion supported. One user explained that the cushion absorbs physical impact or protects them, which perhaps contrasts the vibration function of the prototype. The participants also suggested a new function with the cushion. One participant liked to use the prototype for an alarm clock in the morning. Another participant pointed out that a smaller portable size would create possibilities to enhance the enjoyment of digital contents in other situations. It was found that the participants’ perceptions about how to interact with the cushion changed over the trial.
The prototype was used by the participants continuously throughout the trial period using the Trembling Hug as they watched movies or listened to music. In addition, they created new ways of using the cushion. For example, they felt the vibration of their own voice through the cushion while speaking. These behaviors indicate the possibility of active, playful and consistent engagement with the cushion.
Time Door is an interactive automatic door in which imaginary time spiders live. Through the narrative of the time spiders and associated functions, people using the time door can virtually interact with other people who already passed through the door, as well as with the door itself. The door displays a digitally projected spiral web as a time graph. It also projects traces of people who walked through it in the past. When someone passes through, the identity and the time is captured and updated in the web. Our design method transforms an ordinary door into a playful, informative and social object with extra experiential value.
Time Door has a story about a new imaginary creature that lives in the door. A spider-like character was created for this narrative through appearance, behavior, and movement. Functions of time stamping and an ambient display indicating people who passed the door were added as unexpected but pleasurable functions. The physical movement of the door, peoples’ movements in relation to the door, and the projected augmented display were used as the means for physical interaction with the narrative and the serendipitous functions.
Narrative and Creature
The narrative of the Time Door is a story of an imaginary time spider that lives in the Time Door and subsists by eating people’s traces as they pass through. Time spiders make a time web in the door (Figure 6) which captures a trace whenever people pass the door. Traces become the main food for the time spider. The door also shows the time information. The time web expands hourly and the time spider starts to make a new web from the center. Opening the door when someone approaches it is interpreted in the narrative as the time spider pulls the door. Multiple time spiders can live together in the door. When the time spider pulls the door so the door is open, the position of traces and spiders make music feedback.
Various ambient data, including information about time and the people’s usage of the door, were added as unexpected but pleasant functions of the door. The projected information about the traces and the time web creates a new aesthetic for the door. With this information people intuitively understand how many people have used the door in the past. The trace information is presented though music as well as in a visual form. Time is presented to those people who pass the door. This indirect and ambient information adds new social and informative functions to the door.
The main interaction with the door is the physical movement of people and the door. Because the door has existing physical interactions, it was used as the main means for people interacting with the narrative and the functions. Additionally, sound feedback and a projected display were added to represent different information embedded in the door.
Prototype Implementation of Time Door
The prototype of Time Door consists of sensing kits, a screen, a projector, a speaker and a computer (Figure 7). The sensing kits consist of one infra-red distance sensor and an Arduino microcontroller board. The infrared sensors are triggered by movement of the door. The door’s physical movement is sent to the computer through a serial connection and MIDAS (Nam, 2008). Flash software sends and visual information to the speaker and the projector.
The key challenge of the hardware implementation was to accurately sense the movement of the door. The non-linear analogue values from the infrared distance sensor were recalculated for a linear mapping of the distance of the door from the sensor. The position of the projected visual image was synchronized to the movement of the door so that the projection area followed the movement of the door.
The design experiments with concept generation, prototype implementation and user trials demonstrated that our method can be used to transform existing products into new kinds of value-added interactive everyday products. The stories of imaginary creatures could be applied to many types of products, including kitchen appliances, furniture, musical instruments, electronic audio devices, and business products. The diversity of design cases shown in the experiments indicates that the method can be a general design method for designing interactive everyday products.
The fact that our method uses traits of imaginary creatures can be compared with the design approaches using personality (Lundgren, 2006) or anthropomorphism (Norman, 2004; Landin et al., 2002) for designing interactive products. For example, Iron Horse designed by the animal expression transfer method (Landin et al., 2002) is a bike showing horse traits. The PillO’-Mate (McGee & Harup, 2003) is a cushion designed using contact expressions of people and animals. It activates itself when its proximity-sensor detects something is nearing it. When touched it responds with sound, heat and vibrations. In these existing design approaches, the functional and expressional characteristics of a product tend to be directly mapped to an animal or a living creature. Therefore, it is not appropriate for designing or transforming a product which has little associated expressions of people or animals. The Design by Tangible Stories method, however, can take an indirect mapping between the narratives and the product features because the product is considered as an environment for imaginary creatures. The narratives can have multiple creatures in a product. The story can be independent from the product’s core features. In Talkative Cushion, there are three types of audio bugs. The appearance and the behavior of the bugs are independent from the appearance and the physical interaction of the cushion.
The two user trials showed that the openness and engagement aspects of ludic value can be infused in the design. Users could be more explorative due to the embedded stories and the interactions. Their interpretation of the interactive features of the products could offer room for reflection. In addition, new usage ideas were created. The observation and the feedback also indicate that engagement with the design increased.
From the design experiments, we also found some interesting design patterns. For most design cases, it was possible to add extra ludic value while keeping original functions. In addition, such ludic value could be combined with other design intensions such as education, supporting sustainable energy consumption or communication between people. For example, Posture-Watcher has an educational benefit as it can help school children sit with better posture. MuseScope can make piano practice more pleasurable. Electro-bug sockets support sustainable energy consumption behavior (Table 1).
The design experiments also revealed the scope of application for our method. The application of narratives is popular in designing products for children, such as toy design. Our design cases, however, show that this approach can be applied to general products for all ages of users. The selection of a product category was open in the design exercises, with two of the ten design cases being products for children. Moreover, the narratives, added functions and physical interaction are not directly visible and apparent, which means the methods can be applied to many different types of products for all people.
Another pattern emerged in regard to the physical structure of the products. Most products kept their original functions when features of ludic value were augmented. For some products, the augmented feature required a significant change of the product structure, or additional parts. Most commonly, the parts required for this method were representational devices. Small display devices such as LCD screens were most commonly used. Six prototypes out of ten were implemented with 7-inch LCD screens. The size and the structure of the prototypes were limited to the size and type of current display technologies. This means that this method is also related to the development of display technologies.
It was also possible to create an additional device to support the idea of narratives and interactions. For example, Airium is a new device showing the imaginary creatures in a glass window. Music Scope is also a means to show the environment inside of a piano. Additional ludic values are delivered through these additional devices. Being combined with the related products, it becomes a system that provides original functions and additional ludic value. This modularized approach would be useful when evolutionary design approaches are necessary.
In the meantime, the introduction of this method raises some research issues in design practice and education. It is necessary to have a more structured approach to develop narratives with imaginary creatures. The narrative of our method is different from a conventional narrative with a sequence of events. By investigating the patterns of narratives used in games or literature, it would be possible to build a framework that is optimized for the development of narratives with imaginary creatures. In addition, associations between product attributes and the attributes of creatures in the narrative can be made to help development. This kind of narrative can be explored by structuring categories and attributes of possible imaginary creatures.
From the design experiments, we found that this method requires a variety of skills from a designer or a multi-disciplinary design approach. Although design cases were successfully completed within four weeks, designers had many challenges. The participants of the design experiments were graduate level industrial design students. Some had little knowledge about physical interaction prototyping and developing narratives. For industrial designers, developing narratives and visualization of imaginary creatures were new challenges, and both were required to deal with the temporal and interactive aspects of the digital and physical materials.
To be more successful with this method, a designer needs a variety of new skills, such as being able to visualize the imaginary creatures. It is also necessary to realize physical interaction for the behavior of the creatures. For a successful outcome, the designer needs to have a good sense of visualization and skills for interactive programming. Because different design fields, such as graphic and product design, are merged due to the new digital media with both hardware and software properties, it is difficult for a designer to master all the skills required to prototype the design concepts of this kind. Moreover, this method needs additional skills of storytelling. An interesting future research opportunity would be a comparison of teams of integrated designers and teams that combine their different expertise into a complete set.
One of the impacts of this method is the creation of meaning for things in the digital realm, while preserving the benefits of the technology. Chapman (2009) suggested that users feel a strong emotional connection to a product due to the service it produces, the information it contains, and the meaning it conveys. Our method can be useful to create this attachment aspect when the narratives and serendipitous functionalities become known to the users. However, after special features are exposed to users, it can be difficult to recreate the initial experience. To keep a sustainable emotional connection, our method can be further developed to be transformative and evolutionary.
This paper presents a design method, Design by Tangible Stories, to enhance the ludic value of interactive everyday products. This method can effectively transform existing everyday products into interactive ones with ludic value. It has three key features: creation of narratives about imaginary creatures, embedded serendipitous functions, and physical interaction. We also presented design experiments and user trials to show the value of this method. The contribution of the paper is summarized as follows.
In this paper, we argued that there are design research opportunities to enhance the emotional value of the products and systems in a world with digital technologies. In particular, we highlighted that one way to respond is to enhance ludic value of interactive everyday products in domestic environments. We clarified the meaning and attributes of ludic value and presented an overview of existing research work exploring different experiential values and interaction design methods. Based on this clarification and the review, we proposed our new interaction design method.
We have shown that by using our method, everyday products can be transformed into interactive products that have extra ludic value. The design cases showed that everyday products can be recreated and used as a mediator which supports ludic activities. The proposed method enables us to understand new interaction and functional opportunities of everyday products, and provides a new perspective on traditional products. In the process of making imaginary creatures and narratives related to a product, non-related information, fragmentary knowledge, story and emotion become a source of inspiration. They have the potential to become interesting and exciting stories to be included in products. Also, the method gives a chance to generate ideas on new interaction by interpreting the behavior of imaginary creatures in response to the interaction with users. This implies that our method has the possibility to be used as a design ideation technique.
The design experiments showed that everyday products can be mediators which create new ludic value. Product features that offer unexpected enjoyment and ambiguous purpose lead users to explore a new activity. Users gain new interests through exploration and engagement. We also discussed design patterns and challenges when we adopt this method.
For interactive product and system designers, designing meaningful things while keeping the benefits that technologies provide in a technological world is a significant research challenge. The method and cases presented in this paper are initial attempts. We focused on ludic value among a number of emotional values that constitute meaningful things, rather than commodities, as described by Borgmann (1984). In future work, we plan to explore other experiential values for meaningful things. At the same time, to further develop our method, a theoretical framework can be further developed by adding deep discussion on each element of the method. For example, the theoretical framework of narratives for the method can be explored further by analyzing the structure and effective representation. Theoretical basis on serendipitous functions should also be further investigated to develop design principles. Further investigation is required for effective prototyping technologies used for this kind of tangible interaction. For example, emerging transitive materials, electronic display and sensor technologies can make an impact on the representation of the narratives and the tangible interaction.
This work was supported by WCU (World Class University) program through the National Research Foundation of Korea funded by the Ministry of Education, Science and Technology (R33-2008-000-10033-0). We would like to thank Sangsu Lee, Younkyung Kim, Jinha Sung, Howon Sohn, Seungah Ham, Dami Choi, Jihun Lee and Sine Kim for their help in the design experiment, and Munhwan Lee, Yongki Lee and Mark Whiting for their feedback. Special thanks go to the reviewers for their valuable feedback and comments on the manuscript.
- Annett, J., & Duncan, K. D. (1967). Task analysis and training design. Occupational Psychology, 41(4), 211-221.
- Bell, G., Blythe, M., & Sengers, P. (2005) Making by making strange: Defamiliarization and the design of domestic technologies. ACM Transactions on Computer-Human Interaction, 12(2), 149-173
- Beyer, H., & Holtzblatt, K. (1998). Contextual design: Defining customer-centered systems. San Diego: Academic Press.
- Blythe, M., & Hassenzahl, M. (2003) The semantics of fun: Differentiating enjoyable experiences. In M, Blythe, K. Overbeeke, A. F. Monk, & P. C. Wright, (Eds.). Funology: From usability to enjoyment (pp. 91-100). Dordrecht, the Netherlands: Kluwer.
- Borgmann, A. (1984). Technology and the character of contemporary life. Chicago: University of Chicago Press.
- Card, S. K., Moran, T. P., & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.
- Chapman, J. (2009). Design for emotional durability. Design Issues, 25(4), 29-35.
- Csikszentmihalyi, M. (1998). Finding flow: The psychology of engagement with everyday life. New York: Basic Books.
- Djajadiningrat, J. P., Gaver, W. W., & Frens, J. W. (2000). Interaction relabeling and extreme characters: Methods for exploring aesthetic interactions. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (pp. 66-71). New York: ACM Press.
- Douglas, Y., & Hargadon, A. (2000). The pleasure principle: Immersion, engagement, flow. In Proceedings of the 11th ACM on Hypertext and Hypermedia (pp. 153-160). New York: ACM Press.
- Dunne, A., & Raby, F. (2002). The placebo project. In Proceedings of the 4th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (pp. 11-14). New York: ACM Press.
- Dunne, A. (2006). Hertizan tales: Electronic products, aesthetic experience, and critical design. Cambridge, MA: The MIT Press.
- Gaver, W. W., Beaver, J., & Benford, S., (2003). Ambiguity as a resource for design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 233-240). New York: ACM Press.
- Gaver, W., Bowers, J., Boucher, A., Law, A., Pennington, S., & Walker, B. (2007). Electronic furniture for the curious home: Assessing ludic designs in the field. International Journal of Human-Computer Interaction, 22(1-2), 119-152
- Gaver, W., Boucher, A., Law, A., Pennington, S., Bowers, J., Beaver, J., Humble, J., Kerridge, T., Villar, V., & Wilkie, A. (2008). Threshold devices: Looking out from the home. In Proceedings of the 26th SIGCHI Conference on Human Factors in Computing Systems (pp. 1429-1438). New York: ACM Press.
- Gaver, W., Bowers, J., Boucher, A., Gellerson, H., Pennington, S., Schmidt, A., Steed, A., Villars, N., & Walker, B. (2004). The drift table: Designing for ludic engagement. In Extended Abstracts of the 22nd SIGCHI Conference on Human Factors in Computing Systems (pp. 885-900). New York: ACM Press.
- Hallnäs, L., & Redström, J. (2001) Slow technology: Designing for reflection. Journal of Personal and Ubiquitous Computing, 5(3), 201-212.
- Huizinga, J. (1950). Homo Ludens: A study of the play element in culture. Boston: Beacon.
- Hummels, C., & Overbeeke, K. (2010). Special issue editorial: Aesthetics of interaction. International Journal of Design, 4(2), 1-2.
- Hummels, C., Overbeeke, K., & Klooster, S. (2007). Move to get moved: A search for methods, tools and knowledge to design for expressive and rich movement-based interaction. Personal and Ubiquitous Computing, 11(8). 677-690.
- Janlert, L. E., & Stolterman, E. (1997). The character of things. Design Studies, 18(3), 279-314.
- Jordan, P. (2000). Designing pleasurable products. London: Taylor & Francis.
- Landin, H. (2005) Fragile and magical: Materiality of computational technology as design material. In Proceedings of the 4th Decennial Conference on Critical Computing (pp. 117-120). New York: ACM Press.
- Landin, H. (2009). Anxiety and trust and other expressions of interaction [Doctoral dissertation]. Gothenburg, Sweden: Chalmers University of Technology.
- Landin, H., Lundgren, S., & Prison, J. (2002). The iron horse: A sound ride. In Proceedings of the 2nd Nordic Conference on Human-Computer Interaction (pp. 303-306). New York: ACM Press.
- Löwgren, J., & Stolterman, E. (2004). Thoughtful interaction design: A design perspective on information technology. Cambridge, MA: The MIT Press.
- Lundgren, S. (2006). Facets of fun: On the design of computer augmented entertainment artifacts. Master thesis, Chalmers University of Technology, Gothenburg, Sweden.
- Lundgren, S. (2008). Treating and teaching aesthetics as personality. In Proceedings of the 5th Nordic Conference on Human-Computer Interaction (pp. 511-514). New York: ACM Press.
- McGee, K., & Harup, A. (2003, September). Contact expressions for touching technologies. Paper presented at the 3rd International Conference on Computational Semiotics for Games and New Media, Middlesbrough, UK.
- Nam, T. J. (2008). Interactive tangible and augmented prototyping with MIDAS. In M. Ladly & P. Beesley (Eds.), Mobile nation: Creating methodologies for mobile platforms (pp. 229-234). Waterloo, ON: Riverside Architectural Press.
- Nielsen, J. (1994). Usability engineering (1st ed.). San Francisco: Morgan Kaufmann.
- Norman, D. A. (2004). Emotional design: Why we love (or hate) everyday things. New York: Basic Books.
- Papanek, V. (1995). The green imperative: Natural design for the real world. New York: Thames and Hudson.
- Petersen, M. G., Hallnäs, L., & Jacob, R. J. K. (2008). Introduction to special issue on the aesthetics of interaction. ACM Transactions on Computer-Human Interaction, 15(3), 1-4.
- Petersen, M., Iversen, O., Krogh, P., & Ludvigsen, M. (2004). Aesthetic interaction: A pragmatist’s aesthetics of interactive systems. In Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (pp. 493-502). New York: ACM Press.
- Pink, D. (2005). A whole new mind. New York: Riverhead Books.
- Preece, J., Rogers, Y., & Sharp, H. (2011). Interaction design: Beyond human-computer interaction (3rd Ed.). New York: John Wiley&Sons.
- Ross, P. R., & Wensveen, S. A. G. (2010). Designing aesthetics of behavior in interaction: Using aesthetic experience as a mechanism for design. International Journal of Design, 4(2), 3-13
- Schiller, F. (2004). On the aesthetic education of man (R. Snell, Trans.). New York: Dover Publications. (Original work published 1954)
- Tiger, L. (1992). The pursuit of pleasure. Boston: Little, Brown.
- Ullmer, B., & Ishii, H. (2000). Emerging frameworks for tangible user interface. IBM Systems Journal, 39(3-4), 915-931.
- Ulric, K. (1996). Mythical animals: A reference pathfinder. Collection Building, 15(3), 32-35.
- Verbeek, P. P. (2002). Devices of engagement: On Borgmann’s philosophy of information and technology. Techné , 6(1), 69-92.
- Weiser, M. (1991). The computer for the 21st century. Scientific American, 265(3), 94-104.
- Zimmerman, J., Forlizzi, J., & Evenson, S. (2007). Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 493-502). New York: ACM Press.