Designing for Unexpected Encounters with Digital Products: Case Studies of Serendipity as Felt Experience
Rung-Huei Liang
Graduate School of Design, National Taiwan University of Science and Technology, Taipei, Taiwan
Random mechanisms are often employed to enrich the user experience of digital products: We can see this, for example, in the shuffle listening mechanism of music players. Based on the improvisational characteristics of digital materials, this paper presents serendipity as an experiential quality in everyday life. As a process of meaning-making, serendipity refers to the phenomenon of spontaneously understanding unexpected things, including time, space, people, and contents. Following a research-through-design approach while examining emerging issues such as social connectedness, reminiscence, coincidence, and implicit interactions, the concept of serendipity will be unfolded here through a discourse on three mobile applications: Social Radio, Social Clock, and Sound Capsule. These prototypes, examined through long-term usage and qualitative inquiry, allow us to present implications for designing serendipitous interactions and to articulate serendipity as the act of creating meaningful unexpectedness through interactions that emerge in our everyday practices. We argue that regarding serendipity as felt experience in interaction design would encourage designers to design digital artifacts that place interaction in a state of emergence as well as create improvisational encounters. This research intends to locate serendipity at the core of a novel interactive experience and to contribute knowledge that will enrich the body of interaction design research by merging design practice, criticism, and discourse.
Keywords – Serendipity, Interaction Design, Research through Design, Experiential Quality, Implicit Interaction, Randomness.
Relevance to Design Practice – This paper revisits serendipity as an experiential quality evoked in the use of interactive artifacts. It examines the nature of digital materials so as to help designers create meaningful and serendipitous interactions for people in their everyday lives. The design cases presented also provide knowledge and resources for addressing the improvisational nature of interaction.
Citation: Liang, R. -H. (2012). Designing for unexpected encounters with digital products: Case studies of serendipity as felt experience. International Journal of Design, 6(1), 41-58.
Received Sep. 5, 2011; Accepted Feb. 14, 2012; Published Apr. 30, 2012.
Copyright: © 2012 Liang. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
Corresponding Author: liang@mail.ntust.edu.tw.
Dr. Rung-Huei Liang received his PhD in 1997 from the Department of Computer Science and Information Engineering at National Taiwan University, Taipei, Taiwan, R.O.C. He is now an Assistant Professor in the Department of Industrial and Commercial Design, National Taiwan University of Science and Technology. His research interests include interaction design, computer graphics and multimedia. He leads the Spatial Media Research Group, which focuses on interaction design, virtual reality, aesthetic computing, and tangible interfaces. He also is involved in leading students to create new forms by thinking of technology as a material.
Introduction
The last decade has seen a large number of interactive products and a growing body of research on interaction design. With the rapid growth in interactive technology, including the development of such items as sensors, actuators, and microprocessors, and the great potential this technology has shown to transfer almost any object into an interactive product, there has been an increasing interest in interaction design in both the HCI community and the design disciplines. The HCI community has contributed to the body of knowledge about human-computer interaction by drawing on scientific methodologies and well-developed experimental instruments. Until recently, however, a great deal of thought in the HCI community had gone into addressing functionality and usability, with relatively little attention paid to the articulated experiential qualities of interactive products (Jordan, 2000, pp. 43-44; Löwgren, 2007). As various types of everyday artifacts using digital technology have emerged in the world around us, interaction design research has gradually shifted to highlighting different qualities of usage, such as fluency (Löwgren, 2007), efficiency, transparency, playability, seductivity (Löwgren & Stolterman, 2004), slowness (Hallnäs & Redström, 2001), reflection (Kolko, 2011), poetics (Lin, Chang, & Liang, 2011), and ludic value (Nam & Kim, 2011). Examining computation from more material perspectives, researchers from design disciplines have identified “the material turn” (Wiberg & Robles, 2010) as well as “the material strategy” (Vallgårda & Sokoler, 2010) in interaction design. In addition to tangible materials and concrete functions, interaction design researchers have also identified some abstract properties as resources for design, for example, ambiguity (Gaver, Beaver, & Benford, 2003) and randomness (Leong, Vetere, & Howard, 2006).
Understanding that functionality and usability are not enough to address human needs in interactive systems, design researchers have stepped further into promoting the aesthetics of interaction (Petersen, Iversen, Krogh, & Ludvigsen, 2004; Petersen, Hallnäs, & Jacob, 2008; Hummels & Overbeeke, 2010) from a perspective of pragmatist aesthetics. Petersen et al. (2004) argue that pragmatist aesthetics can provide a basis for centering on the aesthetics of interaction related to our everyday experiential qualities when engaging in and designing interactive systems. This places user experience at the core of interaction, and also considers it to be a qualitative value and a resource within the interaction design agenda. Presenting the concept of fluency, Löwgren (2007) suggests that designers should create graceful and smooth transitions among multiple artifacts by drawing on an understanding of the nature of digital materials in augmented space. Petersen et al. (2004) point out that the aim of aesthetic interaction is the creation of involvement, experience, surprise, and serendipity in the interactive experience.
In this paper, instead of seeing serendipity only as a consequence of interaction, we place it at the center of interaction design as an experiential quality based on investigating the very nature of the temporal structure of digital materials in cyberspace. To help us better understand new materials, McLuhan’s “rear-view mirror” metaphor is helpful: It suggests that we often try to understand a new medium by relating it to the past, and as a result we might miss or misunderstand important impacts of the new medium (McLuhan & Fiore, 1967). To address this problem, design researchers need to regard computers as emerging materials and media, and to focus on researching through design (Zimmerman, Forlizzi, & Evenson, 2007) as a way to explore the unexpected features of digital materials and to shape the future by harnessing a technology-push, consumer-pull balance.
More and more researchers have begun to think of interaction design as a means for creating computational compositions (Wiberg & Robles, 2010; Vallgårda & Redström, 2007) that are composed of various materials, ranging from physical to digital forms. What makes digital materials differ significantly from other materials is the nature of their temporal and spatial structures. Highlighting the aspects of space and time of an object in a context of ubiquitous computing, Bruce Sterling (2005) proposed SPIME (SPace + tIME), which is a location-aware, self-logging, and uniquely identified object. This inspiring notion poses a definite challenge to interaction designers in light of the coming future of the Internet of Things (Sterling, 2005). Surrounded by an ecology of computational artifacts, which are essentially temporal and spatial, how can we design products and services that are based on the huge amount of emergent information as well as the spatio-temporal features of digital materials? If we consider, as Mitchell (1996) has indicated, that encounters are important resources when cyberspace architects and urban designers build a city of bits, then we have to ask whether chance encounters in terms of both space and time become limited and significant resources that can be allocated with digital materials.
In this paper, the author argues that serendipity is one of the experiential qualities most relevant to the nature of digital materials in interaction design. Rather than discussing computational compositions, this paper will focus on the discussion of digital materials to characterize experiential qualities of mediated serendipity in interaction design. The next section will present an overview of related studies and concepts such as serendipity, randomness, and implicit interactions. Then, we will discuss the emerging need to articulate serendipity as an experiential quality and will describe research methods based on a practical approach to framing serendipity. To illustrate what the author seeks to articulate, three functional prototypes—-Social Radio, Social Clock, and Sound Capsule—will be presented along with a brief summary of the related empirical studies. Further discussion and a conclusion close the paper.
Literature Review
Computer-Based Serendipity
“Serendipity” is a term that was coined by Horace Walpole in reference to a fairy tale, “The Three Princes of Serendip.” He described the three princes as going on adventures in which they were “always making discoveries, by accidents and sagacity, of things they were not in quest of” (Serendipity, n.d.). Today, the word serendipity usually means a “chance encounter” or a “happy accident.” The role of serendipity in inventions is quite significant, often referring to accidental discoveries in science, medicine, and technology, for example the discoveries of Velcro, penicillin, X-rays, and dynamite (Roberts, 1989).
Van Andel (1994) defines serendipity as the art of making an unsought finding. By collecting examples from four domains (science, technology, art, and daily life), Van Andel identifies seventeen serendipity patterns related to finding “new and useful” technologies. Regarding a computer program as a subject, Van Andel argues that the program cannot foresee the unforeseen and it cannot be surprised. However, this paper argues that by seeing a computer as a material, we can employ it to provide serendipity. Where we agree with Van Andel is that the subject of experiencing serendipity is the human rather than the machine.
With the dramatic increase in information in cyberspace, a large amount of research centers on information encounters and regards serendipity as the unexpected encountering of something fortunate (André, Teevan, & Dumais, 2009). Serendipitous information retrieval (Toms, 2000; De Bruijn & Spence, 2008), web browsing (André, Teevan, et al., 2009), and recommender systems (Bellotti et al., 2008; Iaquinta, De Gemmis, Lops, & Semeraro, 2009) are frequent applications in computer science. Serendipity in these applications indicates information-seeking that aims for interesting but not directly related results. Encountering useful information therefore becomes critical when assessing serendipity in these applications. To facilitate fortunate discoveries by accident, data mining systems can be designed to visualize data to support interactive and serendipitous discoveries (Beale, 2007). Intelligent and sophisticated visualization techniques are often used to support dynamic and spatial exploration. Moreover, serendipity can also be regarded as the experience of providing opportunistic and spontaneous interactions for geographically-distributed groups interacting in a shared virtual world (Jeffrey & McGrath, 2000). Evaluation with focus groups has shown that informal communication can facilitate awareness by supporting such unintended types of communication in a workspace. Similar notions have been applied to create serendipitous encounters that can often be keys to developing a strong sense of community (Renduchintala, Kelliher, & Sundaram, 2006; Vyas, Nijholt, & Veer, 2010).
Mobile Serendipity
As the last few years have seen considerable growth in applications of mobile devices in the context of pervasive computing, the term serendipity has become much more significant, reflecting as it does the intriguing nature and the complex combination of space and time of an individual’s experiences in everyday life. Both virtual and physical encounters in synchronous and asynchronous manners turn into encounters of great possibility when people live with mediated technology. Context-aware mobile recommender systems show their practical benefits by predicting users’ needs according to their locations and activities (Bellotti et al., 2008). Perhaps the most attractive recommendations are the social serendipity enabled by many applications of social proximity-sensing software (Eagle & Pentland, 2005). While the need of being able to encounter the right information at the right place becomes evident at work and in our personal lives, serendipitous file exchange systems allow users to share information with other users and devices (Ahn & Pierce, 2005). Thus, in addition to providing information and to allowing for the improvisational combination of computational resources and devices, serendipity also encompasses the ability to interconnect arbitrary devices without prior knowledge of one another (Newman et al., 2002). How serendipitous meetings relate to location and the density of hotspots is also an important issue in designing LBS (location-based services) (Trestian, Ranjan, Kuzmanovic, & Nucci, 2009).
What distinguishes mobile serendipity from other types of serendipity is that, in Danzico’s (2010) words, it forges a strange new self-relationship that spans time. The chance encounter with an old self in the same spot leads to a new awareness of time and a new reflection of the self. Danzico (2010) also argues that when choice is involved, we no longer have Walpole’s intersection of sagacity and accident. Most mobile serendipity systems employ deterministic mechanisms to filter out useful information or resources, as these increase the opportunity for specificity instead of the chances for serendipity (Danzico, 2010). André, Schraefel, Teevan, and Dumais (2009) address this problem by defining serendipity as a combination of chance encounter and sagacity. These two phases of serendipity involve, one, finding unexpected information and, two, making an intellectual leap of understanding. André, Schraefel, et al. (2009) also clarify the role of the computer and the human in serendipity: A computer can provide automation and acceleration for the first half of the serendipitous encounter while the second half, requiring sagacity or wisdom, remains dependent on the human. In this paper, the automation we are concerned with is the mechanism of unexpectedness-generation that is supported by a computer. Following and broadening the problem-framing of serendipity as proposed by André, Schraefel, et al. (2009), here we examine serendipity as an unexpectedness-generating and meaning-making experience, with the former quality resulting from technology mediation and the latter depending on the human factor. We will detail the complete framing and corresponding research methods in a later section.
Randomness
In addition to explicit order and logic, randomness has been regarded as a resource for design (Drew & Haahr, 2002; Leong et al., 2006). Linguists have demonstrated that compositions written in a random order can successfully evoke semantically meaningful images when read. Drew & Haahr (2002) examined this phenomenon in their discussion of Samuel Beckett’s prose piece “Lessness,” (1995), in which Beckett used random permutation to order the 60 sentences of the story; reading the story draws on an appreciation process that is dependent on the reader’s attempts to comprehend and create meaning. Why randomness becomes significant in the experience of the story is, as argued by Drew & Haahr, because the absence of an obvious determinism provides a gap in understanding and thus spurs the reader’s interaction with the piece. They see randomness as a means to create a “blank” that induces and guides the reader’s constitutive activity. For a long time, artists have understood and exploited this concept, using blank space as an implicit expression or creation of ambiguity. Considering the need to provide rich resources for experience, Gaver et al. (2003) propose ambiguity as a resource for design, to be employed in addition to more traditional resources that focus on a concern for clarity and precision. By intentionally creating a space for interpretation, ambiguity allows designers to express their own perspective while at the same time engaging users without constraining how they might respond.
Among interaction designers, how to enrich user experience by employing randomness has gained much attention. Leong, Howard, and Vetere (2008) have argued that harnessing randomness in the design of an unfinalized device has many benefits (Leong et al., 2008). Drawing upon McCarthy and Wright’s (2004) assertion, they indicate that unfinalization invites us to see technology as always becoming. Abdicating choice to a system such as shuffle listening becomes a very significant feature when engaging with a device that holds a large amount of digital media. To create such unfinalization and defamiliarization, Leong et al. (2006) propose randomness as a resource for design.
Although randomness is recognized as a resource for improving the experience of the user, intentional manipulation, management, and exploration are still the main categories of interaction that are taken into account when applying randomness. In the following section, we will address the role of intention in the interaction process, and how it relates randomness and serendipity to deeper concerns.
Revisiting Interaction Design
Interaction design has grown into a well-developed design category in which researchers and designers contribute from various disciplines. Bill Verplank (Moggridge, 2006, pp.124-134) illustrates that an interaction designer has three sequential “how-do-you” questions to answer (how-do-you-do, how-do-you-feel, and how-do-you-know), with a similar process apparent when a user interacts with a device. Researchers in physical computing regard interaction as an iterative process (O’Sullivan & Igoe, 2004) that includes input, processing, and output. Interaction design is modeled as the designing of each of these three parts and, thus, implies a sequential process of human and machine activities. Such a paradigm is very useful when designing and implementing interactive systems, and is, in my opinion, based on the metaphor of the Turing machine. However, there are two important issues that are not addressed in this input-processing-output model. The first is how an interactive system works if there is no intentional input by a user or even if there is no obvious user. The other concerns the problems that arise if the interactive process is not a sequential iteration but a coincidental phenomenon.
Ju and Leifer (2008) present a framework for implicit interactions that characterizes them as being based on “attentional demand and initiative.” Attentional demand classifies interactions into foreground and background interactions, while initiative, which refers to who is initiating an interaction, separates reactive interactions from proactive ones. This framework provides a reference for measuring to what extent of implicitness an interaction enables communication without using explicit input and output. For example, device-initiated interactions (proactive) without attention (background) are defined as ambient agents with the highest implicitness. User-initiated interactions are thought to be reactive interactions that involve users’ intentional input. For examining the role of intentions in the serendipity process, Roberts (1989) has coined the term pseudoserendipity to describe accidental discoveries of ways to achieve an end sought for, in contrast to the meaning of (true) serendipity, accidentally discovering things without having the intention of searching for them. Therefore, as our intention is to explore the nature of (true) serendipity, as defined by Roberts, we believe that the implicit interaction is the most promising category of interaction for eliciting serendipitous experiences.
Ju and Leifer (2008) also argue that implicit interaction focuses on improving the interactions between people and computer-based systems embedded in the world. Dourish (2001), furthermore, indicates that embodied interaction research encompasses two significant issues: social computing and tangible computing. Considering the growing interest in social computing, the interaction process is becoming even more intriguing, resonating with the notion of aesthetic experience as a fifth element of interaction, as discussed by Petersen et al. (2004), when there is an increasing amount of interactions that are initiated by “improvising” social actors. Tsujita, Tsukada, and Itiro (2010) propose their InPhase system, which incorporates the notion of “happy co-incidences,” wherein doors, sofas, and televisions are equipped with sensors and connected to remote equivalents. The simultaneous use of these remote equivalents is communicated to form an implicit interaction, which is hard to analyze using the iterative input-processing-output structure. A sequential perspective of interactive process is no longer valid for such a complex and intriguing synchronicity of emerging interactions.
Serendipity as an Experiential Quality
In order to recognize serendipity as an experiential quality in interaction design, it is necessary to set it apart from serendipity’s role in facilitating usefulness and usability. Data discovery for useful findings and ease-of-use in recommendation systems for certain tasks are often the goals of those systems employing serendipity; this is what is described by Roberts (1989) as pseudoserendipity, as it involves an obvious intention of searching. However, this paper advocates that (true) serendipity is a pure experience in our everyday lives interwoven with a tangled ecology of interactive systems. Rather than seeing serendipity as a result of intentional exploration or a process of game-play, we argue that understanding serendipity in interaction design calls for investigating the momentary experience of a user as he/she experiences serendipity. Acknowledging that people’s experiences of serendipity can be imbued with magic, wonder, delight, and thrill, Leong et al. (Leong, Wright, Vetere, & Howard, 2010) regard the “phenomenon of serendipity” as a starting point instead of a means for assessing user satisfaction with regard to results. Quantitative approaches and lab settings are inappropriate for such studies. Moreover, the mobile computing, mediated social connectedness, and implicit interactions enabled by digital materials potentially pose new challenges in interaction design wherein assessing interactions implies analyzing co-incidence and synchronicity instead of examining interactive systems.
Although growing numbers of designers are considering serendipity as a significant feature of digital materials that should be taken into account when designing interaction, very little attention has been given specifically to the perspective of viewing serendipity as an experiential quality that highlights the momentary understanding of unrelated things. Nam and Kim (2011) attach serendipitous functions to interactive products as “auxiliary” features meant to enhance enjoyment during long-time engagement. The non-deterministic mapping of functions, employed to provide unexpected responses in an interactive system, is considered as serendipity as well as a means for enhancing ludic engagement. Since the interactive systems presented by Nam and Kim are all systems involving explicit interactions, “engagement” becomes quite significant. Here we wonder if there is a possibility that the aim of serendipity is neither engagement nor hedonics. On the other hand, Hallnäs and Redström (2001) present slow technology as a means for engaging in reflection and moments of mental rest, as opposed to the emphasis on performance efficiency that is an overriding feature of much technology. Illustrative projects show that the aesthetics of slow technology could bring forth and make room for reflection by combining complexity of form and simplicity of material along with a period of time to allow for accumulated understanding. In contrast to the aims of slow technology, which evokes understanding and appreciation after thoughtful reflection, this paper argues that serendipity is a spontaneous experience that usually disappears when we reflect on it over a period of time. What we are seeking is to examine the momentary understanding of unexpected things that characterizes serendipity rather than an experience of continuous reflection or reasoning.
Research regarding serendipity as lived and felt experience has only recently emerged. Leong et al. (2010) offer a detailed account of how an empirical study with dialogical methods could help in understanding the key qualities and the richness of serendipity. To better understand the experience of serendipity during music listening, they asked participants to carry out dice-led listening activities. A diary, as a retrospective medium, was employed to gather data for further dialogue in interviews. Rather than discussing randomness as a design resource (Leong et al., 2006; Leong et al., 2008), Leong et al. (2010) shift to understanding the experience of serendipity, particularly with regard to their finding that “at the heart of the experience of serendipity was the emergence of powerful personal meanings.”
In developing an understanding of embodied interaction in terms of personal meanings, Dourish (2001) advocates a phenomenological approach to studying embodied interaction, regarding our experiences as embodied actors interacting in the world. Drawing upon Bachelard’s phenomenological thought as a source of inspiration, this paper discusses serendipitous interactions by reflecting on the poetic image that Bachelard regards as an emergence from language: “By living the poems we read, we have the salutary experience of emerging” (Bachelard, 1994).
The focus of this paper is on how digital materials bring forth the experience of serendipity in a mixture of cyberspace and physical space. Rather than seeing serendipity only as meeting with unexpected but useful contents, an approach that usually involves an explicit mechanism of presenting uncertain data, this paper addresses how we live with serendipity in augmented spaces, and “how technologically mediated action is lived and felt,” as McCarthy and Wright have discussed (McCarthy & Wright, 2004). This perspective further characterizes the feltness of experience in our daily practices.
The always-becoming experience of technology that we live with sets the ground for this research, laying stress on how the expression of digital artifacts infuses the experience of serendipity into our everyday lives. Here we take a research-through-design approach (Zimmerman et al., 2007) in exploring how digital materials can create serendipitous experience in a context of personal mobility and mediated connectedness. Instead of analyzing the experience of using available commercial products (Leong et al., 2010; Leong et al., 2006), this research involves presenting original concrete design works and through these articulating the role that serendipity plays in interaction design when experiential quality is concerned. Drawing upon Löwgren’s argument that interaction design needs its own concepts in order to capture and explain the key experiential qualities involved in using digital products (Löwgren, 2007), we propose that experiential concepts for describing how an interaction feels can be articulated as statements forming potential contributions to a discursive community.
Research Methods according to the Framing of Serendipity
To study such a felt experience, there is an urgency to frame serendipity as well as to propose appropriate research methods. It is not our intention here, however, to propose a definite conceptual framing of serendipity, but rather to present a practical or even simplified framing that will facilitate the creation and investigation of the felt experience of serendipity for the purpose of serving as an operational framework to ground this research. In accord with Walpole’s description of serendipity as “discoveries made by accidents and sagacity,” we deconstruct the process of serendipity into two essential parts: accidents and discoveries. Basing serendipity on these two parts, interaction designers should address how to create digital accidents and how to evoke sagacious discoveries that could possibly elicit felt serendipity (Figure 1). These two parts happen to be in line with the definition proposed by André, Schraefel, et al. (2009): the finding of unexpected information (digital accidents) and the making of an intellectual leap (sagacious discoveries).
Figure 1. Two important parts are involved in eliciting felt serendipity with digital products.
To explain this framework further, one part of the serendipitous experience involves generating unexpectedness (with regard to various elements, including time, space, people, events, and contents), which creates digital accidents or unexpected information, and the other part involves making meaning in the context of everyday life, which indicates sagacious discoveries or an intellectual leap. How to generate unexpectedness relevant to the experience of serendipity can be regarded as a design problem that involves making appropriate design choices with regard to materials and functions. The research-through-design method allows researchers to remain open to design choices while working towards understanding. Moreover, its strength is it presents concrete design outputs for further articulation. Therefore, the first part of our framing implies presenting design works that can best elicit serendipitous experiences. On the other hand, the second part of the serendipitous experience, meaning-making, which depends mainly on the human factor, requires a qualitative inquiry into a particular experience. Empirical study of this phenomenon is appropriate for explicating the user experience of serendipity during the use of digital media (Leong et al., 2010). In short, what this paper aims to articulate with regard to felt serendipity will be illustrated by research-through-design and explored through empirical study.
The usual approach in attempting to create the effect of unexpectedness, the first part of the serendipitous experience, is to employ randomness. However, randomness is not in fact a proper term to describe this experience since people do not report that they feel they are experiencing something “random” but rather that they are experiencing a feeling of “serendipity.” For example, people often report experiencing serendipity while shuffle-listening (Leong et al., 2006). In other words, randomness is only one of the means among design resources for generating unexpectedness; it is not an experiential quality that we can investigate. Computer-generated randomness and improvisational activities by social actors are also possible means for providing unexpectedness. With this in mind, and given the strong random quality of unexpectedness in serendipity, we designed three activities to explicate experiences of serendipity during people’s use of digital media.
Regarding meaning-making, McCarthy and Wright (2004) argue that the meaning of a situation is never pre-given. Rather, the understanding of a situation involves dialogical sense making, which will act relationally to bring expectations to the situation and create ways of looking at it. In this paper, therefore, we explore the particular human experience of serendipity by understanding this lived experience as a phenomenon as McCarthy and Wright assert it to be in their conceptual foundation. To approach experience empirically, the “ethnographic stance on interpretation of cultures” suggests that the investigation of serendipity during experiences of cultural activities should be conducted within the everyday settings of people’s lives where serendipitous experiences are reported (McCarthy & Wright, 2004). Therefore, qualitative methods conducted in the field are most effective for capturing the lived experience of people’s activities.
Of special notice is that our framing of serendipity does not allow us to guarantee success in creating serendipitous experiences, nor is it intended to do so. Instead, we seek to shape the processes that are necessary for investigation and observation of the emergence of a serendipitous experience in an individual’s life.
Illustrative Products and Projects
The purpose of this section is to unfold the notion of serendipity that the author is seeking to examine by means of presenting and discussing several example products. The order of these examples will show a sequential process in which we gradually address the issue of serendipity through design and discourse. The discussion of the examples will also pursue an understanding of how serendipity is an experiential quality of the digital artifacts that people live with.
Crocodile Dentist
To illustrate how randomness plays a role in social practices, a popular mechanical toy, Crocodile Dentist (Figure 2), serves as a good starting point. This toy is designed as a game in which the players take turns pressing the crocodile’s teeth, one of which is the “sore tooth” that will cause the mouth to shut. The player who presses the sore tooth and gets bitten is the loser. Each new game is started by opening the mouth of the toy, which causes a mechanism to randomly determine the sore tooth.
Figure 2. Crocodile Dentist mechanical toy.
In examining the process of playing this game, we can see that the toy provides a preliminary understanding similar to the framing above. First, Crocodile Dentist creates and provides an uncertain entity, a sore tooth, by a random function generated by a physical mechanism. This “hardware random number generator” serves as an unprejudiced numerical fact, the pattern of which cannot be predicted by the players. The second relevant aspect of the game is the meaning-making process that it involves, wherein the participants interact in turn with the artifact as well as with each other embodied in a social setting. As with other toy-based games, it is the embodied social interaction rather than the artifact itself that creates the meaning that becomes associated with the game. We can imagine that playing the game alone would be far less interesting than playing it with a group of people. However, although two parts of our serendipity framing appear in relation to Crocodile Dentist, those who participated in the game rarely reported any experience of serendipity, indicating that the framing is a necessary rather than a sufficient condition of the emergence of a serendipitous experience.
The interaction pattern described above is an explicit interaction, as Ju and Leifer (2008) would frame it. Although the uncertainty occurs as a result of information provided by the product itself, the interaction involved is surely initiated by the users and thus forms an explicit interaction, one which is intentionally performed by the players as a result of their focus of attention on the game. If serendipity is a quality that involves a seeking for some kind of uncertainty, implicit interactions surely provide much more unknown-ness than explicit interactions. Therefore, this paper argues that the nature of an implicit interaction that involves some kind of uncertainty in terms of information, attention demands, and initiatives, is in line with what serendipity comprises in the first part of our frame, unexpectedness-generating. Of notice is that the implicit interaction in the framework of Ju and Leifer (2008) is not a definite notion, but rather one within a spectrum ranging from a very reactive and foreground job, direct manipulation, to a very proactive and background one, say, an ambient agent. Ju and Leifer (2008) also assert that it is more accurate to regard interactions as being more or less implicit. This allows us to posit that the aesthetics of serendipity in interaction design entails seeing and perceiving unexpected experiences with interactive artifacts by emphasizing the quality of implicitness that occurs when a user senses and learns the meaning and intention of expression.
In short, Crocodile Dentist is a physical toy with a mechanical randomness-generator that invites users to initiate explicit interactions in a casual situation, one in which the participants simply want to have fun. Thinking beyond the physical material of this example, we wonder whether digital materials could provide more powerful functions of unexpectedness-generating, in the broader sense in which serendipity is located. If so, could the serendipity that is enabled by digital artifacts be interwoven into our everyday life more easily and frequently because of its flexibility and pliability? Moreover, how implicit would the interaction be when conveying serendipity in this context? Unlike the experience of fluency, which is achieved by designing a smooth and graceful transition, how can serendipity, in which the emphasis is on a surprising gap within ongoing contexts, be an experiential quality that we live with in our daily lives? Would it be possible to use mediated social connectedness as a resource for designing serendipity?
In order to address these questions, we will investigate serendipity in interaction design by presenting and discussing the following three projects, all of which are functional prototypes backed up by long-term empirical studies (Liang, Tseng, Lee, & Cheng, 2009; Chang & Liang, 2011; Hsieh, Liang, & Chen, 2011). The focus of this paper is not to detail the empirical data related to each of these projects, but to reveal how each one represents a certain aspect of experiences related to the diversity of serendipity.
Design Works that Articulate Serendipity
The following sub-sections introduce three design works, which are presented in order to articulate serendipity as an experiential quality in interaction design. Each of these projects—a radio, an alarm clock, and a time capsule made of digital materials—is centered on serendipitous experience. They are introduced here sequentially so as to tackle emerging issues. Their presentation in this study follows a research-through-design (Zimmerman et al., 2007) approach, which allows us to contribute actual design works to the field as well as an understanding of serendipity that can be leveraged in interaction design communities. The strength of such a method of inquiry is found in the fact that it gives the researcher the ability to tackle complex discourses through explorations in design. We produce novel integrations of HCI research to “make the right thing: a product that transforms the world from its current state to a preferred state,” as Zimmerman et al. (2007) advocate. In other words, instead of merely presenting an analysis and discourse on available commercial products (Leong et al., 2006; Leong et al., 2010), we seek to design artifacts as outcomes that can transform the world from its current state, in which deterministic functions are provided, to a preferred state in which interaction evokes an experience of serendipity. The artifacts presented in this paper are meant to become design exemplars, providing an appropriate conduit for understanding serendipity as well as for discoursing on our research findings.
We also believe that in order to articulate serendipity as an experiential quality, designing interactive systems to elicit such experiences is critical for voicing and investigating serendipity, which is a very subjective phenomenon. We regard this phenomenon as a problematic one in that it can never be accurately modeled, thus an engineering approach to addressing it would fail.
The sequential order of the following descriptions also documents the process, one of four lenses for evaluating an interaction design research contribution proposed by Zimmerman et al. The description of each project is structured with an introduction, a function/form analysis, a discussion of empirical evidence of user experience, a consideration of the project’s potential impact, and an assessment, in accord with the four lenses criteria of the research-through-design method.
Social Radio
The first project we developed to investigate serendipity was aimed at designing an everyday object for social interaction using ambient forms. The question framed in this case was: How can digital materials and implicit social interactions be embedded in our daily practices so as to create serendipitous experience? Notions of sharing and coincidence, and concepts such as the jukebox and social networks, were proposed and discussed in the early stage of this design. Inspired by and centered on the interaction pattern of calm technology (Weiser & Brown, 1997) while examining daily practices, the idea of redesigning an Internet radio emerged (Figure 3).
Figure 3. Social Radio scenarios.
The result, Social Radio (Liang et al., 2009), is an Internet radio that allows users to upload music and text comments within a social group (Figure 4). Music uploaded by users who know each other is streamed by the server, which automatically schedules the broadcast with a random mechanism. The server of Social Radio is designed as an automatic DJ that keeps the broadcast streaming whenever the server is running. A simple aging scheme is applied to lower the probability of an older song being scheduled on the play-list; in other words, a newly uploaded music clip will be highlighted by means of a higher frequency of appearance.
Figure 4. Social Radio concept.
Implicit interactions can be observed in this project in the following aspects. According to the framework of Ju and Leifer (2008), the interaction that occurs in listening to Social Radio is proactive and mostly background, and thus could be regarded as an ambient agent, which is of the highest implicitness. Changes in broadcast programs are initiated by the system and by other users over time and space. Even though a user could upload a music clip with the desire to influence the broadcast, there is no guarantee that the server will play it immediately, since a random mechanism with an aging strategy is applied. Such an indirect response to what is uploaded and the uncertain time of playback make the interaction ambiguous to a certain degree. Moreover, when we consider that listening to the radio is usually a background activity that accompanies other foreground activities, this shifting of the attentional demand indicates a fluent transition between explicitness and implicitness. Above all, the expression that a user makes by uploading music—reflected for example in the lyrics of a song or its melody, and the literal or metaphorical meanings that these might encompass—potentially forms an implicit communication among participants.
Two necessary parts of serendipity that we have framed can be found in Social Radio. First, unexpectedness-generating is represented by the digital shuffle of music by the server and the improvisational uploads of participants. The implicitness of the interaction described above makes this first part even more intriguing. Second, the meaning-making process that each participant takes part in is experienced in the context in which that person is situated. The experiencing of serendipity in this case relates to the user’s momentary and improvisational linking and understanding of the music, the expression of the music itself, the intention of a specific uploader, and the surrounding contexts.
The preliminary empirical study of this design (Chang & Liang, 2011) allows us to detail the following findings. What makes Social Radio attractive is the implicit interaction of unexpected choices of music contributed by acquaintances. Immediate and spontaneous serendipitous experience occurs very often and can be very rich among a group of people who know each other. Social Radio also supports the idea of food-for-talk, as it can serve as a catalyst for embodied social interaction afterwards. In contrast, participants in a control group that consisted of strangers showed far less experience of serendipity, very similar to the experience of listening to ordinary Internet radio. Unexpectedness becomes too diverse for people to be involved in making meaning and thus fails to create serendipitous experience if the underlying linkage (in this case, a social linkage) is not supported or is of a nature that makes it difficult for the user to make associations. This indicates that successful design for serendipitous experience depends on fine-tuning the degree of unexpectedness so as to support meaning-making in context. In this case, the underlying social linkage serves as a good cue for association and interpretation related to the unexpectedness that is generated, since people are generally curious about the intentions of expressions made by those that they know.
Implications for designing serendipity: Designing successful serendipitous experience depends on how well unexpectedness is generated in a way that supports spontaneous and improvisational meaning-making in context. In this case, randomness is shown as a useful means for creating unexpectedness. A certain degree of correlation between the unexpectedness and what is known, which is a design choice an interaction designer has to make, can provide a cue for interpretation.
The criteria for the research-through-design approach proposed by Zimmerman et al. (2007), including process, invention, relevance, and extensibility, provide a good framework to evaluate interaction design research. The process of designing Social Radio as documented above represents a practical design that integrates web and mobile technologies to create an invention that provides a higher possibility of serendipitous experience in daily life. The relevance of this design is articulated by the concept that living with possible serendipity evoked by music listening is an experiential quality as well as a new way of interaction with a large amount of digital media with implicit social meanings. Social Radio also shows extensibility in that it could be incorporated into future smart mobile devices, imparting to these devices a means for impromptu social connectedness in everyday life.
Social Clock
To further explore the possibility of embedding serendipity in our daily life, we turn now to searching for the kind of daily experience that is a ritual or a regular task. Unlike the ubiquity of Social Radio, such a ritual would usually be situated in a fixed setting, in terms of time and space. Moreover, as more and more people enjoy posting or checking their friends’ posts on sites such as Twitter and Facebook, we wonder if these digital contents might be employed as resources for interaction design. This led us to consider what it would be like if a daily highlight tweet could randomly appear in the physical space of our domestic setting. Linking the idea of a daily ritual with the serendipity of digital posts, our concept gradually converged on the idea of designing a serendipitous awakening experience that was based on addressing the following questions (Figure 5). Who or what awakens a user? How are users awakened? What do users experience when awakened? And, above all, how can an awakening experience be embedded with serendipity?
Figure 5. Social Clock scenario.
Social Clock (Liang et al., 2009) is an alarm clock that allows users to upload sounds when they set the alarm time, while the sounds that are heard when the alarm goes off come randomly from sounds uploaded by other members within a social group (Figure 6). The major expression that occurs when one is woken up by Social Clock is not the result merely of a mechanical function with a typical-sounding alarm. Rather, upon hearing the surprise sound clip uploaded by an acquaintance, the awakening process becomes potentially rich and serendipitous. Social Clock thus acts as an artifact that performs the function of an alarm clock as well as conveys personal expressions in an implicit manner.
Figure 6. Social Clock concept.
If we see the awakening process in general as an interaction, the variety of this experience could result from different initiatives, ranging from an alarm clock, to a telephone call, to one’s own mother. Nevertheless, this interaction is clearly a proactive one as it is impossible for one to have an intention just before being awakened, no matter who initiates the interaction. On the other hand, this interaction is a foreground activity, one that demands our attention. Also, as it falls into the category of alerts in Ju and Leifer’s framework, it is an implicit interaction with a greater degree of explicitness than that found with Social Radio. The certainty of the alarm time and the foreground attentional demand provide Social Clock more explicitness.
The qualities of serendipity—unexpectedness-generating and meaning-making—are quite definite in Social Clock. A randomly chosen personal audio post constitutes an uncertain alarm that is responsible for waking an individual at a pre-determined time. The moment that the Social Clock alarm goes off, one is forced to wake up as well as to experience a moment of relating the alarm sound to an acquaintance. The serendipity potentially evoked by Social Clock is thus a rich experience emerging at the moment of being awakened. Although the set time is definitely explicit, the uncertain alarm and the lack of time for preparation just before it goes off make the serendipitous experience dramatic and significant compared to Social Radio. The momentary characteristic of serendipity in a situation in which one is not wholly conscious, as when waking up, contributes in a sense even more ambiguity to the interaction process.
A preliminary empirical study (Chang & Liang, 2011) that included having participants experience using Social Clock in daily life and examining their responses in a qualitative inquiry (Figure 7) shows the following findings. Social Clock turns a moment that might inherently be trivial and boring into something pleasurable, anticipated, and serendipitous. This moment is quite a short span of time in which almost all participants experience an instant and spontaneous process of meaning-making; whereas reflection and interaction afterward might be possible, the serendipitous moment has passed away quickly. It is this moment of serendipity that is designed into Social Clock for a user to experience as part of a daily routine. Social Clock also initiates rich interaction within a group of acquaintances, reported by participants as the experience of feeling a chance encounter elicited from the unexpected alarm. The alarm sounds thus become alternative materials for people to use for communicating, with a wide diversity of contents possible, drawing for example from pop music, personal voice recordings, jokes, fake broadcasts, dialogues, or humorous singing performances. For instance, participants often uploaded music that related to a social event or other context, such as a recent birthday party or an upcoming meeting, thus spurring the receiver to experience a meaningful interpretation while encountering unexpectedness. On the other hand, although participants in a group of strangers reported a certain degree of interest and expectation, there was rarely any serendipity experienced in this case. Uploaded contents tend to cover a very narrow scope of music if the participants are not familiar with each other. This again echoes the above implication that successful meaning-making of uncertain things is critical for creating serendipitous experience.
Figure 7. Social Clock empirical study: Experiencing the Social Clock in daily life (left); Questionnaire answering and semi-structured interview for the study (right).
Implications for designing serendipity: Serendipity can enrich the experience of daily routines, and serendipitous experience is a moment that a designer can possibly create by working with daily routines, digital materials, and social connectedness.
The four lenses of research-through-design can also be examined in this case. The design of Social Clock indicates a process that is characterized by the momentary experience of serendipity, in contrast to Social Radio. The process documented also describes how the examination of a daily routine can lead to ideas for placing serendipity in a context of embodied interaction. All of the functions of Social Clock are built on an emerging mobile platform, Android; and the form is given a wooden frame so as to identify it in the role of a domestic artifact (See Figure 8). The integration of emerging technologies and crafting skills complete the expression and meet the invention criterion of research-through-design. Social Clock also clearly articulates a promising direction for embodying serendipity in our everyday routines and rituals, meaning that it meets the relevance assessment. Finally, the extensibility of Social Clock is suggested in that the process of social connectedness and random mechanisms emerging in a very short span of time could be a significant resource for designing serendipitous experience in the future.
Figure 8. Social Clock prototype: In a wooden frame (left) and mobile app in use (right).
Sound Capsule
In order to investigate the very nature of serendipity and to articulate its temporal and experiential qualities in a context of mobile pervasion, we further explore whether serendipity could be elicited with digital technologies that might be interwoven anytime and anywhere into our daily life. Instead of there being a pre-set time for the occurrence of serendipity, as with Social Clock, would it be possible to evoke serendipitous experience at an uncertain time? This design also concerns how to create a serendipitous moment that could not be predicted by the user in relation to a common daily activity or a serendipity trigger right before the moment. Noticing that the dramatically increasing amount of digital recordings in our lives makes it impossible for anyone to revisit all digital data that has been personally significant in his or her life, we are interested in the notion of a time capsule that could provide a digital review and an experience of reminiscence serendipitously.
Searching for the appropriate materials to put into such a time capsule, we first went about asking how we might successfully build such a device, and thus identified two significant necessary characteristics: (1) a defamiliarization process, such as a long enough period of time that the user would forget the contents of the time capsule, and (2) a moment of evocation that would create instant surprise and reminiscence afterwards. Our previous study (Hsieh et al., 2011) showed that impressions of audio contents are easier to become vague in a person’s memory, and thus defamiliarized, and at the same time, they can evoke more details than visual content when revisited. These findings suggested to us a design choice of employing digital audio files as the materials of a digital time capsule.
The expression of Sound Capsule is that an individual might receive a phone call unexpectedly from a server, with the content of the call being a clip of audio saved in the past by oneself or an acquaintance (see Figure 9). The sound clip, which would include a recorded voice and surrounding sounds, would have been saved in the Sound Capsule’s server database. The clip is chosen randomly and the phone call is made at a random time. In our preliminary study, it was found that a social-network-enabled Sound Capsule made the investigation too complex in its current state of development. Thus, the current simplified application is implemented on a mobile platform, Android, that supports personal sound capsules instead of social sound capsules; this means that, in the current version, one will only receive one’s own recordings. Sound Capsule is designed to make exactly the same ring tone as the mobile device, and the resulting form of interaction—pressing a button to answer a call—is also identical to that involved in a normal phone call.
Figure 9. Sound Capsule concept.
The randomness of the phone call, encompassing a random time and the delivery of a random sound clip, is what accounts for the unexpectedness-generating quality of the Sound Capsule experience. Moreover, the meaning-making process happens almost at the exact moment that the user presses the answer button, making it situated in a daily practice. In other words, the serendipitous experience arises within an everyday context, in which interpretation and meaning are made in an impromptu fashion and in relation to a dynamic and uncertain happening in one’s surroundings. This design potentially creates the experience of unexpectedly meeting an older self interwoven with a present self in a current time and space. It is interesting that the two characteristics of a time capsule are almost in line with the two significant parts of serendipity framed earlier. The defamiliarization characteristic of the capsule helps to prepare materials for unexpectedness-generating, with the unexpectedness in a broader sense including time, space, people, and contents. On the other hand, a moment of evocation is experienced as the time capsule creates an instant surprise that builds on meaning-making in a context of daily practice instantly and spontaneously. Therefore, the implicit form of a time capsule is very close to the nature of serendipity. Sound Capsule, as an adoption of a time capsule with digital technology, shows much more flexibility and potential in terms of creating serendipitous experiences within everyday activities rather than in a lab- or instruction-based task.
Since defamiliarization of known things is critical to creating an effect with a time capsule, we conducted a study in which the clips used are ones that were saved at least three months earlier (Hsieh et al., 2011). However, informal observation shows that three days are actually enough for most people to become unfamiliar with the sounds they have recorded. The semi-structured interviews in our empirical study, which involved analyzing and discussing data collected in booklets that were handed out to the participants for keeping retrospective diaries (see Figure 10), demonstrated serendipitous experiences that were rich beyond our expectations. These experiences were shown to give rise to a wide range of emotions, including envy, compassion, happiness, and reflection. For example, a feeling of envy came to the mind of one participant upon hearing a conversation between his girlfriend and a “strange man”—the “strange man” being the participant himself, as he momentarily did not recognize his own voice. He misunderstood the phone call to be an unintentional call by his girlfriend, who he thought had accidentally pressed the dial button on her phone at the wrong time. When the memory of his past conversation gradually emerged, the participant’s mood suddenly switched to one of happiness and self-reflection. The unexpected encounter with a dialogue that his older self had had with his girlfriend led this participant to report a serendipitous experience that involved finding “happiness by accident.”
Figure 10. Data collection in the empirical study of Sound Capsule: A sample page of a participant’s retrospective diary (left) and a translation of the written text (right).
The serendipitous experience in this case is much richer than that with other interactive systems, as it involves spontaneous meaning-making. Other participants also reported that the serendipity triggered with Sound Capsule involved great diversity and complex experiential qualities. The following are examples of some other rich serendipitous experiences found in our study: Recordings made during a happy trip with relatives three months earlier, which had almost been forgotten, successfully created a serendipitous association and a happy reminiscence of a special family time. Recordings made three months earlier during a quarrel created a serendipitous experience of reconciliation, with the participant reporting an unexpected and dialogical understanding of himself and his relation to others. Others reported that, in addition to serendipity, other emotions were evoked when a recording was heard in a similar location or situation in which it was made. One participant, for example, felt somewhat stunned when he heard a recording he had made to the time capsule about his desire to find a new job: He heard the recording in the same situation as he had made it, while eating a bowl of instant noodles, and he realized that he was still looking for a job. In this case, serendipity is not necessarily a pleasing experience. Above all, however, these felt experiences of serendipity resonate with the creative understanding that is never ready a priori but must be finalized dialogically, as Bakhtin describes it (McCarthy & Wright, 2004).
Implications for designing serendipity: By living the serendipity we encounter, we can have the “salutary experience of emerging.” Intentional defamiliarization of one’s own belongings, by means of generating unexpectedness in time, space, contents, and people, followed by an evocation, within which meanings are made spontaneously, can possibly create a great diversity of serendipitous experiences.
Sound Capsule successfully meets the criteria of research-through-design. Its design indicates a process that is characterized by an uncertain moment of evocation, in contrast to Social Clock. The process documented above also highlights the complexity and dynamics of embedding serendipity in everyday life experiences. All functions of Sound Capsule are built on the Android system, integrating cutting-edge technology and the emerging needs of people in a daily context (See Figure 11). To gather enough user data in in-situ experiencing of serendipity in real everyday settings for analysis, this functional prototype not only works but also works robustly, reliably, and continuously for a long period of time. Instead of creating serendipity within daily rituals, as Social Clock does, Sound Capsule articulates a novel potential to implicitly initiate serendipity and to provide a spontaneous experience of, for example, surprise, understanding, reminiscence, or some other emotional evocation. In terms of relevance, what Sound Capsule demonstrates is a better projection related to the current state of the art. Finally, concerning extensibility, Sound Capsule itself stands for a characteristic prototype for further discourse and suggests the possibility of a new type of interaction in the future that will emphasize serendipitous experiences based on coupling personally known things over time and space.
Figure 11. Sound Capsule interfaces: Sound Capsule App icon (left) and homepage of Sound Capsule (right).
A Brief Summary of the Three Design Works
Although the details of the qualitative studies of these projects can be found in our previous works (Chang & Liang, 2011; Hsieh et al., 2011), there is still a need to conclude here with a summary table (see Table 1) for a quick comparison. A sequential developing process has demonstrated a clear trend and shift in terms of the richness of serendipitous experience, inquiry methods, numbers of participants, and so on.
Table 1. Comparison of the three prototypes.
In order to clarify that Social Radio provides better user satisfaction over Internet Radio (Chang & Liang, 2011), we recruited ten subjects to perform a pilot study, which showed significant differences in terms of satisfaction and preference. Then, we invited ten acquaintances to take part in an experiment group and ten strangers to take part in a control group, with both groups experiencing Social Radio for one week (see Table 1), to further explore whether serendipitous experiences would emerge in both of these two groups. Measuring instruments included a questionnaire and unstructured interviews. Similarly, we conducted a pilot study, recruiting twenty-four participants in two groups, to determine whether Social Clock provides a better experience of awakening than a typical alarm clock. To further elucidate whether experiences of serendipity appeared, five acquaintances and five strangers were invited to experience Social Clock in their living environment for two weeks, during which time the researchers would email and remind the participants to perform the task and to keep notes on the details of their experience. A questionnaire and semi-structured interviews were employed to investigate the experiential qualities of serendipity. Based on the above two projects, we gradually figured out appropriate methods (Light, 2006) for inquiry into felt experiences of serendipity that could be adapted for use with Sound Capsule. After collecting and interpreting five participants’ diary entries, we sought to gather detailed retrospective accounts by discussing and making sense of them with semi-structured interviews (see Table 1). Thus, instead of seeing serendipity as a measurable degree of user satisfaction, as a way of eliminating frustration and improving performance, our understanding from studying serendipity suggests that there is value in looking at the experiential dimensions of the felt serendipity evoked by technology.
While the first two projects partially tried to deal with experimental or controlled comparisons, the depth and richness of the serendipitous experiences that were evoked seemed to be lower than in the case of the last project. We modified our inquiry method in investigating Sound Capsule so that it would be useful not only for articulating serendipity as an experiential quality, but also for dealing with the richness of felt serendipity. Moreover, following the method described above, we were able to explore serendipity in ways that reveal insights into “split seconds of interaction” and point to “shifts that are momentary and subtle” (emphasis added) (Light, 2006).
As the inquiry methods changed, the number of participants we recruited in each project also varied. It seems reasonable that the results of experiential studies involving greater depth would come from and focus on fewer subjects under more limited research resources.
Our design works unexpectedly address issues of dialogicality that are described by Bahktin. Although the serendipitous experiences in the three projects all encompass a process of dialogical meaning-making, which always occurs in the tension between self and other, as Bakhtin explains it (McCarthy and Wright, 2004), the focus of each project is slightly different. Social Radio and Social Clock emphasize how an individual relates to others, while Sound Capsule highlights how one relates to one’s own history of selves.
Our investigation also shows, interestingly, that user satisfaction and depth of feltness seem to contradict each other in these three projects. Social Radio meets the highest degree of user satisfaction, while Sound Capsule meets the lowest. On the other hand, the experiences evoked by Sound Capsule are of the maximum depth of feltness, while those of Social Radio are the lowest. Social Clock ranked medium both in terms of satisfaction and feltness.
Discussion
As Bachelard (1994) asserts, a sincere impulse toward admiration, rather than critical consideration, is always necessary if we are to receive the phenomenological benefit of a poetic image. Arresting this impulse by putting the mind in second position destroys the primitivity of the imagination. Drawing on Bachelard’s assertion, we argue that serendipity in interaction is an experience of momentary and spontaneous understanding beyond the slightest critical attitude. Any critical consideration will put the mind in second position and destroy what articulates serendipity as an experiential quality.
“Lessness,” Beckett’s randomly ordered prose piece, demonstrates how random permutation of sentences creates serendipitous experience in reading. In contrast, our design works illustrate how random permutation of implicit interactions creates serendipity in our everyday practices. Rather than designing a serendipitous experience while reading and interpreting, we suggest that interaction designers should address how to design serendipity as something we live with, that is, seeing serendipity as lived experience situated in everyday contexts, as McCarthy and Wright (2004) explain.
In developing methods for understanding the experience of serendipity, Leong et al. (2010) have demonstrated how serendipity can be understood as felt experience and how collected accounts can be analyzed to see how people make sense of what they experience. Their study implicitly encompasses designing a process that uses a random mechanism to elicit serendipity and conducting a retrospective and dialogical investigation as a way to understand the moment of felt experience through an act of sense-making. Although they did not explicitly identify them as such, these two parts would be consonant with how we have framed serendipity as two necessary processes: unexpectedness-generating and meaning-making. Moreover, in order to perform research-through-design, what we investigated in our efforts to understand serendipity were three digital artifacts that we ourselves designed, in contrast to investigating a commercially available product, as was the case with Leong et al.
Another important aspect that our designs demonstrate is that implicit interactions will better resonate with the nature of serendipity. Although a dice-led music listening experience involves a random mechanism for eliciting serendipity, the activity that Leong et al. (2010) asked participants to perform was a foreground, explicit and intentional task, one usually situated in front of a desk so that the user could keep steady while looking at photos and writing notes. It is very difficult to persuade us that such steady foreground activities are lived and felt experiences in our everyday practices. The point here is that designing serendipity we live with is not as easy as designing an interactive serendipitous system. Similarly, Peesapati et al. (2010) propose the Pensieve system, which randomly forwards by email messages coming from one’s previous posts on social websites, with the aim of “supporting spontaneity and serendipity on reminiscing.” However, the activity of checking one’s email is also a foreground and explicit activity, which provides a structure regarding unexpectedness-generating. Our intention is not to exclude these projects in terms of serendipity, but to highlight the challenge of lived and felt serendipity being interwoven into everyday practices such as listening to music while walking, using an alarm clock to wake up, and answering a phone on the street. The artifacts we designed suggest that implicit interactions coupled with unexpectedness-generating and meaning-making processes will be highly geared toward creating lived serendipity.
Engagement and awareness are always a dilemma when being engaged with an interactive system. Continuous engagement with interactive systems often causes users to feel decontextualized in their surroundings, as well as less aware of where they are situated. Serendipity is neither intended to address the dilemma of engagement and awareness, nor is its aim to provide fluent experiences in interaction. Experiencing encounters in-situ always requires a background of everyday life practices instead of engagement with an interactive system. Nevertheless, most context-aware systems are in fact another type of engagement system. Serendipity in this research indicates neither an engaging system nor an awareness system. Rather, serendipity, being an interruptive and discontinuous experience, happens in a sudden moment that might break the engagement, awareness, and even fluency of interactive systems. The coincidence of such experiences is a promising direction for future investigations into serendipity.
These three projects present a series of design outputs developed as research-through-design, a process that shows its strength in doing and articulating, and through which understanding of serendipity emerges as concrete knowledge instead of conceptual notions. Above all, the design artifacts are made to articulate serendipity experienced with digital products as well as to provide a “catalyst and subject matter for discourse” (Zimmerman et al., 2007) in the interaction design community. These artifacts have become common examples for discussion within our own laboratory, and it is believed that they can inspire other interaction designers to learn about and integrate serendipity into their designs. The prototypes provide concrete examples of eliciting experiences of serendipity in our daily life, and evaluations of the prototypes revealed a shift from functional usage of digital materials to a new way of interacting with digital media. These prototypes also connect with the increasing need in daily life to enable users to experience serendipity in a context of pervasive computing, while regarding users as going beyond the roles of receivers and producers of digital data.
Above all, rather than seeing serendipity as a deterministic outcome of an interactive system, we understand it as a phenomenon that can be observed and that can possibly be elicited by shaping appropriate processes. There is no guarantee of the emergence of serendipity, nor are we able to measure or compare the intensity of serendipitous experiences.
Conclusion
This paper has presented the findings of a continuous design effort over the past two years to explore a new possibility that places serendipity at the center of experiencing interactive processes with artifacts that use digital materials. In line with Löwgren’s (2007) cultural approach to addressing experiential qualities in interaction design, knowledge about serendipity has unfolded in the process of selecting and examining certain examples. In addition to assuming the perspective of a critic, by selecting and reading the examples, this paper also presents in the selection process real design cases undertaken and tested in living contexts, articulated according to a research-through-design approach (Zimmerman et al., 2007), which allows us to employ our strongest skills in integrating digital materials and making a research contribution. What this paper contributes might be seen in how it shows a new combination of roles for the interaction designer: designing, criticizing, and discoursing, while investigating experiential qualities.
Although qualities such as fun, enjoyment, aesthetics, and experience have been examined by researchers in the humanities and social sciences (Blythe et al., 2010), serendipity as felt experience is rarely investigated, probably because of the difficulty of systematically eliciting such lived experiences in an amount necessary for valid investigation. With this in mind, our design artifacts demonstrate that digital materials show great potential for creating interactions that evoke the emergence of serendipitous experiences in everyday life.
The final result of this study includes a concrete problem framing and articulation of the experiential quality of serendipity, and a series of artifacts accompanied with documentation of the design process. Moreover, we have designed artifacts that serve to identify opportunities for new technology that could have significant impact on efforts to elicit a very subjective phenomenon, serendipity. In order to evaluate the complex effects of these artifacts situated in our daily life, we have presented a process of seeking appropriate research methods. We also discovered unanticipated effects of felt serendipity and provide implications for linking the general concepts of serendipity to specific contexts and experiences of target users. While the contributions of this paper, following a research-through-design process, are artifacts that demonstrate significant inventiveness, we have ignored any attempt to make commercially successful things, focusing instead on making the right things.
According to what we have framed and the findings related to our design artifacts, it appears that serendipity could arise as an experiential quality interwoven into daily life by employing digital materials with appropriate forms in terms of time and space. Unexpectedness with a certain degree of association for meaning-making indicates the challenges of design choices that are made with the intention of creating serendipity we live with.
Beyond the role of being a tool for communication, language is also a material used in composing a poem. If we search for the next step in interaction design by making an analogy between language and interaction, it seems we can regard interaction as a material that goes beyond its original functionality. Acknowledging the rich body of phenomenological research, for example, Bachelard’s (1994) assertion that “poetry puts language in a state of emergence, in which life becomes manifest through its vivacity,” the serendipity articulated in this paper could be one aspect of what places interaction in a state of emergence. Thus, it is expected in the future that interaction designers will create experiential qualities that place interaction in a state of emergence, in which our everyday life becomes manifest through its vivacity.
References
- Ahn, J., & Pierce, J. S. (2005). SEREFE: Serendipitous file exchange between users and devices. In Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices Services (pp. 190-198). New York, NY: ACM Press.
- André, P., Schraefel, M. C., Teevan, J., & Dumais, S. T. (2009). Discovery is never by chance: Designing for (un)serendipity. In Proceedings of the 7th ACM Conference on Creativity and Cognition (pp. 305-314). New York, NY: ACM Press.
- André, P., Teevan, J., & Dumais, S. T. (2009). From x-rays to silly putty via Uranus: Serendipity and its role in web search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2033-2036). New York, NY: ACM Press.
- Bachelard, G. (1994). The poetics of space. Boston, MA: Beacon Press.
- Beale, R. (2007). Supporting serendipity: Using ambient intelligence to augment user exploration for data mining and web browsing. International Journal of Human-Computer Studies, 65(5), 421-433.
- Beckett, S. (1995). Lessness. In S. E. Gontarski (Ed.), Samuel Beckett: The complete short prose, 1929-1989 (pp. 197-201). New York, NY: Grove Press.
- Bellotti, V., Price, B., Rasmussen, P., Roberts, M., Schiano, D. J., Walendowski, A., Begole, B., … Walendowski, A. (2008). Activity-based serendipitous recommendations with the Magitti mobile leisure guide. In Proceedings of the 26th Conference on Human Factors in Computing Systems (pp. 4521-4524). New York, NY: ACM Press.
- Blythe, M., McCarthy, J., Light, A., Bardzell, S., Wright, P., Bardzell, J., & Blackwell, A. (2010). Critical dialogue: Interaction, experience and cultural theory. In Proceedings of the 28th Conference on Human Factors in Computing Systems (Extended Abstracts, pp. 1157-1166). New York, NY: ACM Press.
- Chang, H. -M., & Liang, R. -H. (2011). Express yourself: Designing interactive products with implicitness to improve social interaction. In J.A. Jacko (Eds.), Human-computer interaction: Towards mobile and intelligent interaction environments (Vol. 6763, pp. 175-184). New York, NY: Springer.
- Danzico, L. (2010). The design of serendipity is not by chance. Interactions, 17(5), 16-18.
- De Bruijn, O., & Spence, R. (2008). A new framework for theory-based interaction design applied to serendipitous information retrieval. ACM Transaction on Computer-Human Interaction, 15(1), 1-38.
- Dourish, P. (2001). Where the action is: The foundations of embodied interaction. Cambridge, MA: The MIT Press.
- Drew, E., & Haahr, M. (2002). Lessness: Randomness, consciousness and meaning. In Proceedings of the 4th International Conference on CAiiA-STAR Research. Retrieved May 1, 2010, from http://www.random.org/lessness/paper/
- Eagle, N., & Pentland, A. (2005). Social serendipity: Mobilizing social software. IEEE Pervasive Computing, 4(2), 28-34.
- Gaver, W. W., Beaver, J., & Benford, S. (2003). Ambiguity as a resource for design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 233-240). New York, NY: ACM Press.
- Hallnäs, L., & Redström, J. (2001). Slow technology: Designing for reflection. Journal of Personal and Ubiquitous Computing, 5(3), 201-212.
- Hsieh, P. -C., Liang, R. -H., & Chen, H. -C. (2011). SoundCapsule: The study of reminiscence triggered by utilizing sound media and technology. In Proceedings of the 4th World Conference on Design Research [CD-ROM]. Delft, the Netherlands: Delft University of Technology.
- Hummels, C., & Overbeeke, K. (2010). Special issue editorial: Aesthetics of interaction. International Journal of Design, 4(2), 1-2.
- Iaquinta, L., De Gemmis, M., Lops, P., & Semeraro, G. (2009). Recommendations toward serendipitous diversions. In Proceedings of the 9th International Conference on Intelligent Systems Design and Applications (pp. 1049-1054). Los Alamitos, CA : IEEE Computer Society.
- Jeffrey, P., & McGrath, A. (2000). Sharing serendipity in the workplace. In Proceedings of the 3rd International Conference on Collaborative Virtual Environments (pp. 173-179). New York, NY: ACM Press.
- Jordan, P. W. (2000). Designing pleasurable products. London, UK: Taylor & Francis.
- Ju, W., & Leifer, L. (2008). The design of implicit interactions: Making interactive systems less obnoxious. Design Issues, 24(3), 72-84.
- Kolko, J. (2010). Thoughts on interaction design. Burlington, NC: Morgan Kaufmann.
- Leong, T. W., Howard, S., & Vetere, F. (2008). Take a chance on me: Using randomness for the design of digital devices. Interactions 15(3), 16-19.
- Leong, T. W., Vetere, F., & Howard, S. (2006). Randomness as a resource for design. In Proceedings of the 6th Conference on Designing Interactive Systems (pp. 132-139). New York, NY: ACM Press.
- Leong, T. W., Wright, P., Vetere, F., & Howard, S. (2010). Understanding experience using dialogical methods. In Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction (pp. 256-263). New York, NY: ACM Press.
- Liang, R. -H., Tseng, K. -C., Lee, M. -Y., & Cheng, C. -Y. (2009). Social radio: Designing everyday objects for social interaction with ambient form. In Proceedings of the 5th International Workshop on Design and Semantics of Form and Movement (pp. 82-91). Eindhoven, the Netherland: Philips.
- Light, A. (2006). Adding method to meaning: A technique for exploring peoples’ experience with technology. Behaviour Information Technology, 25(2), 175-187.
- Lin, Y. -C., Chang, H. -M. & Liang, R. -H. (2011). Designing poetic interaction in space. In J.A. Jacko (Eds.), Human-computer interaction: Towards mobile and intelligent interaction environments (Vol. 6763, pp. 559-568). New York, NY: Springer.
- Löwgren, J. (2007). Fluency as an experiential quality in augmented spaces. International Journal of Design, 1(3), 1-10.
- Löwgren, J., & Stolterman, E. (2004). Thoughtful interaction design: A design perspective on information technology. Cambridge, MA: The MIT Press.
- McCarthy, J., & Wright, P. (2004). Technology as experience. Cambridge, MA: The MIT Press.
- McLuhan, M., & Fiore, Q. (1967). The medium is the message: An inventory of effects. Corte Madera, CA: Gingko Press.
- Mitchell, W. (1996). City of bits: Space, place, and the infobahn. Cambridge, MA: The MIT Press.
- Moggridge, B. (2006). Designing interactions. Cambridge, MA: The MIT Press.
- Nam, T., & Kim, C. (2011). Design by tangible stories: Enriching interactive everyday products with ludic value. International Journal of Design, 5(1), 85-98.
- Newman, M. W., Newman, M., Sedivy, J. Z., Neuwirth, C. M., Edwards, W. K., Hong, J. I., Izadi, S., et al. (2002). Designing for serendipity. In Proceedings of the 4th Conference on Designing Interactive Systems (pp. 147-156). New York, NY: ACM Press.
- O’Sullivan, D., & Igoe, T. (2004). Physical computing: Sensing and controlling the physical world with computers. Boston, MA: Thomson Course Technology.
- Peesapati, S. T., Schwanda, V., Schultz, J., Lepage, M., Jeong, S. -Y., & Cosley, D. (2010). Pensieve: Supporting everyday reminiscence. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2027-2036). New York, NY: ACM Press.
- Petersen, M. G., Hallnäs, L., & Jacob, R. J .K. (2008). Introduction to special issue on the aesthetics of interaction. ACM Transaction on Computer-Human Interaction, 15(3), Article 10.
- Petersen, M. G., Iversen, O. S., Krogh, P. G., & Ludvigsen, M. (2004). Aesthetic interaction: A pragmatist’s aesthetics of interactive systems. In Proceedings of the 5th Conference on Designing Interactive Systems (pp. 269-276). New York, NY: ACM Press.
- Renduchintala, A., Kelliher, A., & Sundaram, H. (2006). Creating serendipitous encounters in a geographically distributed community. In Proceedings of the 1st ACM International Workshop on Human-Centered Multimedia (pp. 45-54). New York, NY: ACM Press.
- Roberts, R. M. (1989). Serendipity: Accidental discoveries in science. New York, NY: Wiley.
- Serendipity. (n.d.). Retrieved July 26, 2011, from http://en.wikipedia.org/wiki/Serendipity
- Sterling, B. (2005). Shaping things. Cambridge, MA: The MIT Press.
- Toms, E. (2000). Serendipitous information retrieval. In Proceedings of the 1st DELOS Network of Excellence Workshop on Information Seeking, Searching and Querying in Digital Libraries (pp. 11-15). Zurich, Switerland: ERCIM.
- Trestian, I., Ranjan, S., Kuzmanovic, A., & Nucci, A. (2009). Measuring serendipity: Connecting people, locations and interests in a mobile 3G network. In Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement Conference (pp. 267-279). New York, NY: ACM Press.
- Tsujita, H., Tsukada, K., & Itiro, S. (2010). InPhase: Evaluation of a communication system focused on “happy coincidences” of daily behaviors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2481-2490). New York, NY: ACM Press.
- Vallgårda, A., & Redström, J. (2007). Computational composites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 513-522). New York, NY: ACM Press.
- Vallgårda, A., & Sokoler, T. (2010). A material strategy: Exploring the material properties of computers. International Journal of Design, 4(3), 1-14.
- Van Andel, P. (1994). Anatomy of the unsought finding. Serendipity: Origin, history, domains, traditions, appearances, patterns and programmability. The British Journal for the Philosophy of Science, 45(2), 631-648.
- Vyas, D., Nijholt, A., & Veer, G. V. D. (2010). Supporting cooperative design through “living” artefacts. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (pp. 541-550). New York, NY: ACM Press.
- Weiser, M., & Brown, J. S. (1997). The coming age of calm technology. In P. Denning & R. Metcalfe (Eds.), Beyond calculation: The next fifty years (pp. 75-85). New York, NY: Copernicus.
- Wiberg, M., & Robles, E. (2010). Computational compositions: Aesthetics, materials, and interaction design. International Journal of Design, 4(2), 65-76.
- Zimmerman, J., Forlizzi, J., & Evenson, S. (2007). Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 493-502). New York, NY: ACM Press.
