Slipping Into the Breaks

Janice Ahn, Lead Interaction Designer

Relational Storytelling in the IoT

Story is core to how we make meaning. Each of us lives amidst tangles of narratives—we weave them together constantly as we live day-to-day and as we reflect on the past.

Before moving into user experience design, the focus of my life was story. I crafted stories by writing and directing films, and studied them by way of English literature and performance. I was steeped in story and form across media. After diving into user experience design, my ears perked up at story-related terms like “user stories,” “scenarios,” and “personas.” After some mental gymnastics, I figured out that these terms have a tenuous connection to the storytelling I’d studied and practiced. I’ve kept trying to reconcile the story I know with the UX/digital product world. The IoT now provides an opportunity—and an imperative—for UX designers to engage story more deeply.

As I work on more and more complex UX challenges, I realize that designers can learn a lot from storytellers. If you think about it, storytellers are designers working within complexity: they simultaneously have a macro purview of various parties, circumstances, behaviors, consequences, outcomes, arcs—and a micro purview of how actions, facts, and intentions would influence the story plot, as well as beats, rhythms, nuance, and meaning. In different situations, storytelling concepts and techniques can provide the unique insight needed to design for more meaningful experiences. For example, even in developing VR/AR/MR experiences, it makes no sense to bypass story—technology is not powerful as spectacle, it’s powerful as a medium.

The IoT offers a unique opportunity to put storytellers’ techniques into practice. Looking back, we see that leading up the IoT, we’ve been designing primarily for one brand, for walled gardens of interactivity, and for comparatively less data-complex experiences. The “happy path” for users has been relatively linear. But today, we have a growing ecosystem of connected services and platforms that are highly complex.

IoT Ecosystem

Within this exciting and tangly ecosystem, connected devices are collecting and sharing data at a new magnitude of complexity. Experiences move across screens, between screens, and off the screen. A story can no longer live within one brand experience; it now lives across brands, platforms, and experiences. In the IoT design challenge space, we must consider that: devices are specialized with a range of capabilities; the IoT spans both physical and digital contexts; devices and services function across intersecting systems; connectivity is varied; and experiences are non-linear. (In Designing Connected Products, Claire Rowland and her co-authors speak about this in more detail.)

Story has the amazing capacity to order incredible complexity. While the term "relational storytelling" may seem redundant, it best suggests the strength of story to define relationships between elements in a system. Because story distills relationships, it can scale up. And so it can be used to meet some of the unique design challenges that come from the increased complexity of the IoT.

Take, for example, how the storytelling technique of “premise” functions. The story premise is the underlying “spine” of the story, the motivating force that leads to the story’s inevitable conclusion. In Romeo and Juliet, the premise might be that “Great love defies even death.” The premise is not merely an idea or a theme. As Lajos Egri asserts in The Art of Dramatic Writing, “The author using a badly worded, false, or badly constructed premise finds himself filling space and time with pointless dialogue—even action—and not getting anywhere near the proof of his premise. Why? Because he has no direction.” A strong premise gives the writer—and the story—direction. The writer hangs characters’ “spines” off of the premise. (Romeo’s character spine might be, “To win great love at any cost.”) A strong premise gives direction to anyone participating in the “design” of the story production, e.g., directors, production designers, and actors.

The director, for example, uses his or her understanding of the distilled, core story to make innumerable decisions, often in collaboration with creative partners. And an actor goes through extensive script analysis to identify, clarify, and embody the story premise and his or her character’s spine, in order to make acting decisions, often in collaboration with others actors and the director.

So, you might be asking: How in the world does this apply to the IoT? The thing is that storytelling is not abstract or theoretical—it’s incredibly practical. It grounds the work of the entire production, and it defines the final “product.” As designers, we can use premise as a construct to make decisions about how a connected product behaves and how it works within the larger context of the IoT landscape. What if we undertook to articulate a premise? While it seems like it would be easy, it takes a lot of thought, work, and discipline to articulate a strong premise. Yet doing so would force us to define the story and would enable us to constantly check our subsequent decisions against the premise. It would serve as a framework for our decisions, and it would shape our product fundamentally.

Another storytelling concept is “diegesis,” namely diegetic and non-diegetic spaces. Diegesis is perhaps most commonly referenced when it comes to sound in film. Diegetic sounds are those whose sources are visible or implied on screen; they come from the fictional story world—the diegesis. Think of Mo’ Better Blues by Spike Lee (or any other film featuring a musician). In it, the trumpet music that Denzel Washington’s character plays on-screen is diegetic sound. Non-diegetic sounds are those that come from outside of the fictional world, like a film score or voice-over. So there’s a distinction between the fictional story world and the non-fictional world. But what’s more interesting is how that distinction is traversed, how the wall is permeated. For example, Denzel’s character “breaks the fourth wall” as he practices his trumpet while staring straight into the camera. [embed youtube clip here] We’re made aware of film as a medium, and of a storyteller’s hand at work. It’s this crossing of spaces that applies to the design considerations of the IoT.

Earlier, I mentioned that one of the considerations in designing for the IoT is that experiences move between screens, and that connectivity is varied. The walled garden of interactive experience is permeated. In the context of the IoT, a person is both the character and the audience—he or she moves between spaces. You could say that in the IoT, the diegesis is the the world in which the user actually interacts with the connect product or service while the non-diegetic space is everything else. Using these constructs enables designers to be more intentional in designing for a person’s experience of movement between these diegetic and non-diegetic spaces.

One theme that’s emerging is “between-ness” or what’s underneath. To riff on this idea, Invisible Man by Ralph Ellison comes to mind. Our nameless main character and narrator says,

Invisibility, let me explain, gives one a slightly different sense of time, you’re never quote on the beat. Sometimes you’re ahead and sometimes behind. Instead of the swift and imperceptible flowing of time, you are aware of its nodes, those points where time stands still or from which it leaps ahead. And you slip into the breaks and look around.

The character, speaking to us (crossing diegetic/non-diegetic space) from a “border area” of Harlem, in a basement sectioned off and forgotten since the 19th century, is literally and figuratively situated both in-between and underneath. This passage has resonated with me for years. Part of the character’s power comes from occupying that space—from it, he finds understanding and perspective.

Likewise, we can find power in subtext, which is what lies beneath spoken words of dialogue. It’s what a character really thinks, feels, and believes. It’s through practicing human-centered design that we can listen to subtext, which develops our empathy for the real people engaging the IoT; that empathy in turn equips us to design for more meaningful experiences. When I hear the phrase “Internet of You” in lieu of “Internet of Things,” I imagine that the “you” references real people as the main characters of their stories. Because main characters get to be complex. Unlike supporting characters or extras, they get to have more depth.

All designers will benefit by drawing deeply from story, not only to communicate with audiences, but to unpack complexity using narrative techniques. Within the complex ecosystem of the IoT, we have the opportunity to leverage the power of “story”—to the benefit of the design process, of team collaboration, and of the product itself, and ultimately for the real people for whose experience we are privileged to design.

This article is adapted from Janice Ahn’s “Narrative Techniques in the IoT” (presentation). Interaction 16 Conference, 02 March 2016, Finlandia Hall, Helsinki, Finland.

Share this article