Story Research

This section deals first with story and narrative theory, then describes some of my own research conducted in completion of doctoral studies at Northumbria University, UK (2020) and a masters degree at Sunderland University, UK (2008).

Useful Tips for Design Teams

The following practical and theoretical insights about design storytelling were drawn from empirical studies and reflective experiments conducted on the subject of design story work reported in the research thesis Making Scenarios More Worthwhile: Orienting to Design Story Work (below).
Practical insights

• Come to know the purpose of the story by asking the right questions.

• Be concrete. Get down to the details.

• Look for conflicts or confrontations and motivations.

• Identify the story’s keystone idea (find it through telling, by diligence in looking for universal connectivity – the coherent big idea).

• Use a canonical framework to guide story spinning. Dial-a-Plot helps with new stories. StoryFrame helps with a wide range of spinning and evaluation activities.

• Draw on simple, tangible, non-intrusive resources (props) to invigorate ideas and prompt actions.

• Understand where human lack is and how it differs from the worldly ‘problem’.

• Abandon all preconceptions about the way the story should be expressed; let the story and the telling unfold in a way that fits with the people involved in the story work and the story work setting.

• Story spinning consists of a series of retellings.

• Test the story in different media (multi-modal language or separate language tellings), and different tenses.

• Progress in a rhythm of deliberative and reflective moves.

• Accept that during storytelling there will be doubts and episodes of apparent inaction.

• During such episodes rethink, reframe and refit everything; introspective viewpoints, methods of working, contextual information, the story, the design.

Theoretical insights

• Story spinning is a narrative activity that concerns itself with both content (the story being told) and expression (the way it is told). Teams have to learn more than just what the story is; they have to learn how it can be told.

• Story and narrative are inseparable. In design work the story has traditionally been the goal of story spinning. Yet questions about the way it should be told (the narrative) can challenge a design team’s working methods. Such questions should not be dismissed. Rather they should be embraced.

• Though mediated by strategic conversations, stories are the product of the imaginings of those who engage as both tellers and listeners. Allow the imaginings. Before launching into uncharted waters, invite an expert on-board.

Story and Narrative Theory

Though, in common parlance the terms story and narrative may be used interchangeably, in fields such as literary studies and disciplines such as narrative inquiry it is necessary to distinguish one from the other. For example, in literary formalism the basis of story, the ‘fable’, is ‘the set of events tied together which are communicated to us in the course of the work’ or ‘what has in effect happened’ (Tomashevsky, 1965 in Chatman, 1980:20). However, the structuralist perspective views story as a chain of events, characters and settings that form the content of narrative. Carpenter and Emerald (2009 cited in Dwyer & Emerald, 2016:4) add that ‘stories, in the main, provide meanings for past events, that is, they are a context for knowledge production’. While Bremond (in Chatman, 1980:19–20) adds that story has a ‘layer of autonomous significance, endowed with a structure that can be isolated from the whole of the message’.

Narrative theory

Etherington (2008:4) contends that ‘etymologically... “narrative” combines recounting of events with a particular kind of knowledge or understanding of them’, adding that in Martin’s view (2008, in ibid.) ‘[t]his indicates the characteristics of narrative which go beyond sequencing of events and towards meaning-making’.
Narratology, the study of the form and function of narrative, is a vast subject (Prince, 1982). It is widely accepted that narrative consists of at least two essential parts, though descriptions of the parts differ from one school of thought to another. Formalist interpretations of narrative take neither the author of a work nor the work’s expressive form into account. A sequence of events, characters and settings unfolds as a ‘fable’ or fabula, which addresses ‘what has in effect happened’, and a ‘plot’ or sjuzet, which addresses ‘how the reader becomes aware of what happened’ (Chatman, 1980:20). Referring to Russian Formalism, Bruner (2004:696) gives a rough interpretation of fabula as ‘theme’, the timeless aspect of story, sjuzet as ‘discourse’, the sequenced aspect of story, and forma, the third aspect of Formalist narrative theory, as ’genre’.
Structuralist theory, on the other hand, argues that both the author of the work and the form that the work takes are essential elements of narrative. In any given narrative the story is necessarily allied to a discourse which, by reason of it being expressed by someone in a particular way, takes on a particular form (Chatman, 1980:20). Things that have come to characterise contemporary narrative as literary forms have evolved include a steady move ‘toward an empowerment and subjective enrichment of the Agent protagonist’ (Bruner, 2004:698), and, citing (Grimas & Courtes, 1976), ‘a landscape of consciousness, the inner worlds of the protagonists involved in the action’ that complement a landscape ‘of action on which events unfold’(Bruner, 2004:698).
Through plot selection, narratives bring absent or distant actants, events and settings related in stories into the present where they are imbued with new meaning. Thus, ‘[e]very narrative… is constructed on the basis of a set of events which might have been included but were left out’ (White, 1980:17). The term ‘actant’ is used to refer to ‘something that acts or to which activity is granted by others’ (Latour, 1996:7). The term is used to refer to actor categories or archetypes (ibid.). The term ‘actor’ is reserved to refer to particular individuals. According to White (1980:15), narrative representation acknowledges and partakes in the co-construction of culturally determined ‘social systems’ through which judgements are made about ethical and moral significance in lived experience.

Characteristics of narrative

Hermeneutics; the analysis of historical texts, offers theory on the characteristics of narrative. Gergen (2005:100) provides an overview of attempts made in a number of domains to characterise well-formed narrative. The following criteria are deemed to be central:

demarcation signs (ibid.:65, 82),

establishing a valued endpoint,

selecting events relevant to the endpoint,

the ordering of events,

stability of identity, and

causal linkages (ibid.:100–104).

White (1980:9) separates narratives from annals and chronicles on the basis of the contestability of events and whether the author has written with an authoritative voice. Since annals purport to relate incontestable events that are eminently rational there is little need for the text to take up an authoritative voice. However, chronicles are written from the point of view of individuals whose authority may be contested. Thus, an authoritative voice is used to persuade readers of the plausibility of events.
Annals are characterised as having no plot, identified author, date associated with authorship, stated rationale, opinion or beginning or ending. Also, they do not claim to be either factual or fictional. Chronicles, on the other hand, are characterised as having greater comprehensiveness, greater narrative coherence, thematic organisation of materials, a central subjective, reveal the author’s hand or point-of-view, and are written with an authoritative voice. However, because chronicles act as a record rather than a rationale they simply terminate and draw no conclusions. In contrast, narratives feature a kind of moral closure that is not present in either annals or chronicles (White, 1980:26).
Figure 2.6. Chatman’s model of narrative.
For further insights, Chatman (1980:26) provides a model of narrative structure that draws on Saussure and Hjelmslev. Described here as a branching tree diagram, the model describes how narrative stems from a combination of story, which deals with elements of content such as events and existents, and discourse, which deals with elements of expression, such as the structure of transmission and manifestation. This subdivision of elements can be grouped into four categories (Table).
Table. Narrative elements.
Form of Content (FC): Is ‘the abstract structure of relationships which a particular language imposes […] on the same underlying substance’ (Chatman, 1980:23). Form is a characteristic of narrative that, in the case of content, structures components such as events that include actions and happenings and existents that include characters and settings, in a particular way. For example, the story structure and form of the movie Easy Rider stems from the friendship between Wyatt and Billy, their affiliation with hippies and drugs, and their search for spiritual truth.
Substance of Content (SC): Is the substantive elements of story, consisting of selected ‘thoughts and emotions common to mankind’ (ibid.:23). Elements such as ‘people, things, etc., processed by the author’s cultural codes’ (ibid.:26) that are particularised by Form of Content. For example, the substantive content of the movie Easy Rider is that of motorcyclists, tour trips, rebellion and struggles to find happiness.
Form of Expression (FE): Refers to ‘[t]he structure of narrative transmission’ (ibid.:26), that is, how the substance is put together or presented. It is the form given to a particular manifestation of narrative discourse that imparts special meaning. For example, sound techniques, editing and cinematography give particular qualities of form to the substance of a cinematic narrative. Elements of expression (the discourse) impart particular meanings to elements of content (the story).
Substance of Expression (SE): Refers to the media or mode of conveyance by which narrative expression is achieved. ‘In languages, the substance of expression is the material nature of the linguistic elements, for example, the actual sounds made by voices, or marks on paper’ (ibid.:22). Manifestation of the substance of expression may be verbal, cinematic, balletic, etc., (ibid.:26). For example, The story of Wyatt and Billy’s friendship is told through the media of cinema. The Substance of Expression is therefore cinematic.

Discourse theory

For Cohan and Shires (2005:19), ‘[d]iscourse is where meanings actually get produced’ . The value of stories and scenarios, therefore, does not lie in the objects used to describe them or in the stories and scenarios themselves, rather ‘the power is in the dialogue and potential for creating shared understanding’’(Blandford et al., 2007:18). ‘One common problem is that people often focus on the scenarios themselves, while the benefit needs to derive from the process gain. All emphasis must be on the quality of the ‘strategic conversation’ (Van der Heijden et al., 2002:3). For Schwartz (1991:xv), an authority on strategic planning, strategic conversations are those ‘that, in themselves, lead to continuous organizational learning about key decisions and priorities’. The focus of strategic conversations in design story work, therefore, needs to revolve around questions that the design team seeks answers to in order to understand what to design for and how to design for it.

On the difference between story and narrative

Narrative is desirable in contemporary design work because it embodies features of story and discourse that are capable of conveying the complexities and subtleties of lived experience (McCarthy & Wright, 2007). For designers, narrative affords privileged views of human desires, motivations and beliefs that mere chronicles and annals do not.
If story is likened to what is described on an annotated map, such as a street map or even a map of the human body, narrative can be thought of as the kinds of descriptions that might arise from someone walking those streets or being instructed on the inner workings of the human cardiovascular system. In this view, story couches content, such as settings, happenings and actions, in abstract terms. In story, both the content and the structure of the content become defining characteristics. Changes made to key aspects of a story can become defining characteristics of another story. For example, if the river Thames was taken out of a map of London and the streets were rearranged as if it had never been there, it would no longer stand as a map of London, but rather as a map of a very different London.
Narrative, on the other hand, consists of a combination of story and discourse (Chatman, 1980). It concerns itself with one of many focalisations, ‘points of view’ or tellings of a story. Here, a narrative might be thought of as a paths through or interpretation of a story. For example, Robert Wise’ 1961 film adaptation of West Side Story and issue 21 of Marvel’s comic, Deadpool, are distinctly different narratives. Yet they are both based on the same story, that of William Shakespeare’s Romeo and Juliet. What we might conclude from this is that where stories give a general rendering of characters and events, etc., narrative gives a very particular interpretation through a particular form of expression. It is the discursive aspect of narrative that couches story in human terms that imbue it with meaning.

Storytelling

‘The telling of a story requires skill’ (Eisner, 1996:7).
Storytelling consists of a story, a teller and at least one listener. By way of example, think of a political statement (story) made by a politician (teller) to a journalist (listener). The act of telling a story generates a narrative. The politician’s narrative refers to the particular way in which the policy is described, whether persuasive, defensive, contrite or inspired. In turn, the journalist will create a narrative based on their understanding of the story.
Though the story and its meaning may be clear to the teller, interpretations made by listeners may differ. The most successful storytelling, therefore, occurs when stories are co-created. Such arrangements are contractual. The teller offers narrative prompts. If the listener accepts the offer, their response may be to fill-in, complete or add to parts of the narrative. What happens in these situations is that, in effect, the listener is engaged in reconstructing the story by telling it to themselves. Thus, the skill of the storyteller is linked to the contract they establish with listeners, and the resonance of their story depends on the quality of the narrative prompts they offer.

Well-formed narratives

Citing Lippman’s (1986) study of recall in courtroom testimonials, Gergen (2001:253) describes how well-formed narratives, which;

‘demonstrated the selection of events relevant to an endpoint, the causal linkages between one event and another, and the diachronic ordering of events’ were believed to be genuine, whereas ill-formed narratives were considered to be false. Narratives are coherent and persuasive when their value and purpose are recognisable, and these are more easily recognised when their structure is somewhat familiar. ‘[W]hen events within a narrative are related in an interdependent fashion, the outcome approximates more closely the well-formed story’ (Gergen in Straub, 2005:99).

With so many challenges facing designers, guidance in authoring a well-formed story may be one area where the research can make a difference. ‘Narratives are conversational resources, constructions open to continuous alteration as interaction progresses’ (Gergen, 2001:249).

Scenario Theory

Theoretically, scenarios are closely related to schemas, frames and scripts, the origins of which pre-date the emergence of scenario-based design. As such, schemas, frames and scripts provide a good starting point for understanding how scenarios have been theorised and how new perspectives might be brought to bear on scenario theory.

Schemas

The notion that individuals develop and use mental schemas to make sense of the world has been attributed to the work of Bartlett (1932), whose work is concerned with understanding human relations and interactions. Schema is a ‘term used in psychology literature which refers to memory patterns that humans use to interpret current experiences’ (Bartlett 1932:254 ct. in Herman, 2002:89). To study memory patterns, Bartlett conducted a number of studies, one of which looked at a story that is challenging to remember.
The term schema was popularised by Piaget, whose work in developmental psychology underpins current understandings about how intellect – the ability to know and understand – grows throughout childhood. According to Piaget (1952:6), in the early stages of cognitive development, children create simple cognitive schemata to store and retrieve information about things they encounter in the world.
Bartlett’s (1932) schemata was later subdivided into ‘static’ and ‘dynamic’ knowledge representations; the former ‘stereotypic states of affairs or situations’ called ‘frames’, and the latter ‘stereotyped sequences of events’ called ‘scripts’ (Herman, 2000).

Frames

Cognitive thinking and the computational theory of mind, which views the human brain as an information processing system that performs computer-like operations, can be traced back to Descartes. Exemplified in the work of Broadbent (1958), this view underpinned emergence of the fields of cognitive psychology, cognitive science, and Human Computer Interaction (HCI).
In an early paper that begins to tackle the problem of imparting to machines the human capacity for common sense reasoning, Minsky responds to propositions posed by Piaget (1952), Bartlett (1932) and others on how frame theory might address them: When one encounters a new situation (or makes a substantial change in one’s view of a problem), one selects from memory a structure called a frame. This is a remembered framework to be adapted to fit reality by changing details as necessary. (Minsky, 1975:1)
Building directly on Bartlett’s (1932) schemas, Rumelhardt (1980) emphasises the importance of the way frames form interlinking conceptual networks that help constitute ‘the individual’s view of the world, enabling the individual to meaningfully interpret events, objects and situations’ (in Hoyle, 2001:10).

Scripts

A script differs from a frame in that it [a script] represents a set of expectations... a sequence of events that take place in a time sequence. (Bartlett 1932:255 in Herman, 2002:89). Throughout the 1970s, the Computer Science Research Group at Yale was instrumental in laying the groundwork for story-understanding and story-generation systems that made significant contributions to the field of artificial intelligence. The work was underpinned by script theory. According to Schank and Abelson (1977:248), a script is a ‘frequently recurring social situation involving strongly stereotyped conduct, such as a visit to a restaurant’. Scripts are used in computer programs where their function is to act on a pre-set number of inferences or assumptions about stereotypical situations that may be encountered in the course of processing narrative texts. Like Minsky (1975), Schank’s (1981) conceptualisation of how memory works includes the notion that memory is stored in container-like spaces that can be accessed according to a hierarchical arrangement, and his description of Memory Organization Packets (MOPs) speaks of higher levels that accommodate access to ‘generalized scenarios’, and lower levels that store ‘specific details of particular events’. (ct. in Hoyle, 2001:12).

Making scenarios more worthwhile: Orienting to design story work

A summary of research conducted in completion of a doctoral thesis at Northumbria University, UK (2020).

Author:

Malcolm Jones

Supervisors:

• Gilbert Cockton(primary).

• Mark Blythe (secondary).

Download the complete thesis here: Media:Jones_M_Making_Scenarios_More_Worthwhile.pdf

Abstract

Figure 1. ‘Stories are basic to all human culture, the primary means by which we structure, share, and make sense of our common experience.’ :(Jenkins, 2006: 120/121)

Increased complexity in contemporary design work has led designers to place greater dependence on the use of story and narrative. Though many consider story and narrative a fundamental part of design, use continues to present challenges and efficacy is poorly understood. With regards to use, challenges stem from a lack of support in directing strategic conversations towards getting the right stories and to getting the stories right. With regards to efficacy, poor understanding stems from a lack of research and unifying theory.

Designers turn to story and narrative in response to challenges they face in a world where rapid change and increased complexity have become the norm.
Story and narrative have long been used in design (Don, 1990; Laurel, 1991; Carroll, 1995; Erickson, 1996; Muller, 1999; Grimaldi, Fokkinga & Ocnarescu, 2013). Most designers have a pretty good idea of what stories are and have a good grasp of how stories help them represent, explain, understand and comprehend lived experience (Cochran, 1990:73, in Moller, 2013:28).

For Erickson (1996), who has made a strong case for design as storytelling, stories benefit designers in a number of ways; they are memorable, informal yet concrete, persuasive and informative.

Yet, stories are complex things that consist of elaborate assemblages of narrative elements and discourse. They bring such disparate things as people, objects, actions, happenings, settings and events (Chatman, 1980) together and organise and present them in ways that are at one-and-the-same-time comprehensible and viscerally resonant. In that they always take a particular stance or espouse a particular point of view they are also political and potentially persuasive. For example, they can be used to deceive as easily as they can be used to express views of shared or individual truths, facts or realities. They are an integral part of social interaction (Figure 1) and as such are intertwined with language and communication. For designers, authoring stories that help them design can be challenging. Co-authoring good stories, i.e., ones that either inform or invigorate design work, requires specialised skills and expertise, and the right kinds of resources.
Research conducted over the past 50 Years into Scenario Planning and Scenario-Based Design represents the largest body of knowledge on the use and efficacy of story and narrative in design. The first step taken in search of ways to support design story work looked at the worth and limitations of theory and practice in design scenarios and storyboards.

Aim

• To make scenario and storyboard use more worthwhile.

Objectives

• Understand the current use and limitations of scenarios and storyboards.

• Co-design approaches and resources that support design story work.

• Empirically test and refine story-based approaches and resources.

• Put practical knowledge and empirically tested resources directly into the hands of designers.

Research questions

Figure 2. Matrix of four key research questions

Two complementary lines of inquiry were taken up to help answer research questions.
Understanding Practice concerned itself with how designers work with story, narrative, and narrative resources (Figure 2; A & C), while Building Theory concerned itself with how story, narrative, and narrative resources work for designers (Figure 2; B & D).

Findings from the literature

• A lack of unifying theory to deal with design story work as a whole.

• Gaps in tool and method support in areas of story work that are conceptually challenging.

• Failings in practice; scenarios do not always work as expected.

• Design story work can be visually represented and analysed to gain insights into how designers tell stories.

• Using appropriate supportive resources, design workers can learn to recognise 'keystone ideas' essential to design while engaging in story-based strategic conversations.

Research framework

Epistemology
With regard to views of reality (epistemology), the research followed constructivist principles. Constructivism, in this sense, takes up a middle-ground between objectivism and subjectivism (Fig. 3). For Given (2008:116) 'constructivism [...] disallow[s] the existence of an external objective reality independent of an individual from which knowledge may be collected or gained. Instead, each individual constructs knowledge and his or her experience through social interaction'.

Theoretical perspectives
Pragmatism was taken up as the primary theoretical perspective. Holding to some of the principal tenets of pragmatism (Baert, 2009:24) and constructivism, perspectivism was taken up as a secondary theoretical perspective. For Nietzsche, perspectivism is a philosophical stance that ‘rejects the idea of objective or absolute truth and argues that there are multiple ways to view the same phenomenon or object’ (Given, 2008:481).

Social constructionism, with its concern for the co-construction of meaning through mindful engagement with the world, provided a theoretical lens through which to view and analyse story work.

Methodology
The methodology can be summarised as Research into Design, for Software, Interaction and Experience Design, through Graphic Design.

Methods
Methods used to conduct qualitative research drew on the principles of ethnography and narrative inquiry. Narrative resources were conceptualised through ideation and sketching exercises, and further refined and developed through a series of tests and self-reflective experiments. Interrogation of how the narrative resource worked in design practice was achieved by conducting seven co-creation workshops. These took place in an 18-Month period from 2012 to 2013. Protocol analysis was used to analyse the data.
Forms of data included representations that recreate past events, drawn from;

• preparatory and planning materials,

• working and development materials, and

• reflective materials,

as well as forms of expression, such as;

• gestures,

• utterances, and

• interactions.

Studies

Six studies were conducted over the course of two years.
Study 1. One story, three narratives
An empirical study conducted with second year image-making students at Northumbria University, Newcastle, UK. The aim of the study was to test for the first time the efficacy of concepts for narrative resources and approaches to story work. Outcomes from this study informed ongoing refinement of narrative resources and expansion of the resource repertoire.
Collaborative authorship of a future scenario
An empirical study into story authorship with narrative resources conducted with two partner/owners of a small design agency.

By integrating novel narrative resources with traditional methods, the Pilot study sought to observe the entire life cycle of a story from the planning and authoring stage to creation of a presentation-quality storyboard. 134 The last case study in this series comprised two Innovation workshops. Two teams addressing a novel research question provided the opportunity to make prolonged and rigorous observation of resource-supported story work.

Charting the Digital Lifespan of Tomorrow: Research Through Design Fictions.
Media:Charting_the_Digital_Lifespan_Report.docx

Claims for contributions to knowledge

In the area of Design practice

1. Specialised tool support by the provision of a suite of narrative resources.

2. Method innovation through guidance to support independent development of narrative resources.

In the area of Design theory

3. Revision in the way resources are viewed and theorised.

4. Advancement of theory supporting a view of design as storytelling.

In the area of Research practice

5. Provision of two novel, empirically evaluated visualisation techniques that serve as aids to narrative analysis:

• storyboard transcription, and

• Resource journeys.

6. Furthering methods of analysis.

Resource Journeys

Figure 3. Resource journey map

Resource Journeys are graphical representations of collaborative story work that place narrative resources at the centre of human action (Figure 3). The arrangement of visual elements was inspired by a form of dance notation called labanotation.
The example (right) describes the journey of narrative resources that came into play in the course of conducting two, two-day Innovation Workshops. The first workshop took place in Delft, the Netherlands, with two design research colleagues from Northumbria University and a team of three design researchers from TU Delft who were investigating a challenging question:

Is it possible to design a boundary object?

No narrative resources came into play at the Delft workshop, and very little progress was made in answering the research question. However, a wiki website, as well as shared notes and recordings helped bridge the time and distance gap until the second workshop, which took place in Newcastle, England approximately 6-Weeks later. There, supported by a number of narrative resources designed to help designers tell good stories, a predominantly story-based approach to design was adopted to help answer the Delft Teams research question. Contextual information needed to help answer the question was drawn from two sources; storyboard transcriptions made from video recordings of the Delft workshop, and a role play activity guided by a narrative resources called "20-Questions". Once everyone was well primed with background and contextual information the two teams engaged in Storienteering; story-spinning with Dial-a-Plot, Event Cards and an Annotated Event Map (later renamed 'Visual Plot-line).

Bibliography

Baert, P. (2009) Neo-Pragmatism and Phenomenology: A Proposal. European Journal of Pragmatism and American Philosophy, 3 (2). pp. 24–40.
Carroll, J. M. (1995) Scenario-Based Design: Envisioning Work and Technology in System Development. 1st Edn. John Wiley & Sons.
Chatman, S. (1980) Story and discourse: narrative structure in fiction and film. Cornell University Press.
Cochran, L. R. (1990) Narrative as a paradigm for career research. In Young, R. A. & Borgan, W. A. (eds.) A methodological approaches to the study of career. New York: Praeger Publishers.
Don, A. (1990) Narrative and the interface. In; The Art of Human-Computer Interface Design. Reading, MA: Addison-Wesley.
Erickson, T. (1996) Design as storytelling. Interactions, 3. pp. 30–35.
Given, L. M. (2008) The SAGE Encyclopedia of Qualitative Research Methods. SAGE Publications.
Grimaldi, S., Fokkinga, S. & Ocnarescu, I. (2013) Narratives in design: a study of the types, applications and functions of narratives in design practice. In Proceedings of: the 6th International Conference on Designing Pleasurable Products and Interfaces. Apr 27- May 2. pp. 201–210.
Laurel, B. (1991) Computer as Theatre. New York: Addison-Wesley.
Moller, L. (2013) Creating Shared Understanding in Product Development Teams. PhD Thesis.
Muller, M. J. (1999) Catalogue of scenario-based methods and methodologies. Cambridge: IBM Watson Research Center.

workPlay: A design for an ideation and sketching tool

Research conducted in completion of a master of arts degree at the University of Sunderland, UK (2008).

Author:

Malcolm Jones

Supervisors:

• Manny Ling(primary).

• Neil Ewins (secondary).

Download the complete thesis here: Media:WorkPlay_Thesis.pdf

Overview

This research interrogates the expression and communication of user experiences in front-end product and service design. The need in front-end design for systems of communication that can express ideas quickly and clearly has arisen as a result of pressures to increase the success rate of new product and service development, to reduce time spent on research and development, to protect IP, and, particularly, to respond to an increased need to express and convey user experiences. workPlay is designed to be a broadly accessible graphical communications system that uses visual narrative to increase comprehension. As such, it uses familiar graphical elements found in comic strips - a simple but effective combination of words and pictures.

Aims

1/ To establish sound historical and theoretical contexts to support the development of a unique system of visual communication.

2/ To develop a comprehensive plan for a graphical system that enables a user who has little or no artistic training to convey concepts related to dynamic human activities and communicate them effectively to others.

3/ To provide an environment for holistic and creative thinking that readers with different learning styles can comprehend.

Methodology

Figure 1.

The research methodology encompasses;

• identification of the main fields of study and the areas within them that are relevant to the development of workPlay,

• qualification of the importance of each area of study to determine the level of effort to expend on research in that area,

• appreciation of the relationships between areas of study to better understand the influences that each has on the other, and

• analysis to determine important findings, and to determine areas of study that present new research opportunities.

The map (Figure 1) provides visual reference of the methodology’s architecture with indications of research activity and outcomes. It is divided into two registers, the upper register, the field of design(indicated by coloured triangles, circles and hexagons) shows the sequence of steps involved in the communication of an idea; the lower register, the field of technology(indicated by squares and a diamond) shows activities conducted in that sector that have an impact on the design process. The map clearly shows the areas of study considered to be most relevant to the development of workPlay; 05.Language, and 06.Message.

Literature Review

Communicating an idea
For the convenience of devising a research methodology the process involved in communicating an idea has been reduced to ten key steps (numbered 01 to 10). The ten steps are divided into three stages: the ‘author or pre-author’ stage, 01 to 03, represented by triangles, this stage encompasses the identification of a value proposition and the formulation of an idea; the ‘author’ stage, 04 to 06, represented by circles, this stage encompasses the communication process; and the ‘reader’ stage, 08 to 10, represented by hexagons, this stage encompasses the perception and acknowledgement of the idea by a 'reader'. Number 3, idea, and number 13, ideation, are areas of the fields of study that share common interests and values. They form a bridge between the two fields of study. Ideation is seen as an area of great importance to the study. It has been identified as the area most likely to provide new research opportunities, as indicated by it’s unique size and shape(diamond). The sizes of the shapes in the diagram indicate the relative volume of research conducted in each area.
Comics and comics theory
The sequential format adopted by workPlay is directly influenced by storyboards, a pre-production sketching system used in the advertising and movie businesses. The current form of storyboards, a sequence of scene sketches with textual notations, is widely attributed to Walt Disney who was influenced by the ability of early comic strips to communicate narrative quickly and simply. Research for this project included: the history of comics history, highlights of which are shown here; the graphical conventions they use, an analysis of comics theory and claims that comics should be considered a 'visual language', a subject which is addressed in the critical evaluation paper.
Comics history

Figure 2.

Figure 3.

Figure 4.

Comics have come a long way in the past hundred years. Increasingly they are being published online, either by the authors, their agents or reps. An indication of the trend towards ubiquitous authorship is the recent proliferation of online ComicMakers. A 13th century Catalan manuscript reminds us that the form of the comic as a work of sequential art has been around for some time (Figure 2). Thomas Rowlandson pioneered speech bubbles and accompanying notations within the imagery (Figure 3). This Harriet and Smith strip (Figure 4) appeared in Robin Annual, 1958, Illustrated by Paddy Spratley and written by Jane Gross. It shows the use of frame numbering and narrative text set beneath the imagery. No speech bubbles.
Philosophical Context

Figure 5. ‘When did this grand and consistent use of images disintegrate?’ (Gombrich 1961, p.29)

The philosophical context for the imagery used in workPlay might best be described as turbulent. Plato(c.429-347 B.C.)would have loved it. Leonardo DaVinci (1452-1519 AD) would have hated it (Figure 5). These two giants of western culture appear to have had very strong and opposing views about linear perspective.
Adhering to the principal of being just-barely-good-enough(JBGE), with the goal of simplicity, the style used in the workPlay imagery system is distinctly pictographic. It’s form rigidly follows its function(Greenough 1958, Sullivan). That it is pictographic might not be controversial, but that it is pictographic and based closely on a figurative system that has not been in use for two thousand years might be. Plato upheld a pictographic ideal, that of conveying information in a pure, unbiased form not distorted by a personal perspective. He exempted Egyptian art from his scathing comments about mimetic artists(Neiva). Later, the Christian church found their own justifications for keeping with the pictographic tradition, but the approach ultimately lost out to science and the edicts of daVinci(Gombrich). Since Leonardo’s time, western art has embraced linear perspective and moved methodically towards a more mimetic(mimicing reality) form of representation. 'For Plato image making must be understood as part of the process of cognition. In this process, we find a perceived thing, its name, its definition, its representation, and finally, on a higher plane, understanding and true knowledge.’ (Neiva 1999, p.78).
On Plato’s moral posture on images, ‘Images ought to be judged in terms of unambiguous ethical principals and never be left solely in the hands of their makers and consumers.’ (Neiva 1999, p.79). In defence of perspective and in opposition to pictures that have more than one scene or perspective, like those that used registers in the early Renaissance period and in Egyptian art, DaVinci has said that they were the ‘height of stupidity on the part of the masters’ (Gombrich 1961, p.16). Leonardo Da Vinci championed perspective. Through his numerous inventions, he also had a hand in furthering the development of the camera. Because of his prominence, many artists fell into line with his views on linear perspective, and by the ‘beginning of the sixteenth century, the old compartmentalised system was no longer acceptable’ (Gombrich 1961, p.34-35). Realism and the perspective view was in, and to this day many people judge images by a benchmark of what they perceive to be 'realistic'.
As a consequence the ‘mimetic ideal’ (Neiva 1999, p.82), supported in it’s relentless pursuit of perfection by ‘..the phenomenal advances in computer modelling and animation [that] allows us to create ever more realistic synthespians– human and animal models..’ (Cotton 2000, p.29). Fortunately there are distinct differences between the areas in which each of these approaches finds its place. Other than its use in holograms, computerised simulations and virtual reality where the duplication of reality is acknowledged as a duplication, the mimetic ideal is to be found wherever products are directly consumed, as in the entertainment sector, a sector dominated by movies and games. Here (as in advertising) because of the directness of the link to the consumer a form of courtship takes place. The consumer is seduced, placated, enticed to consume. Any device that will help in that cause is brought to bear. It might be said that imagery and images used in this way are consumed – taken-in like food. The pictographic approach, in the form of pictograms, symbols, icons, and logos, work in a space that is one step removed from direct consumption, and, their purpose is not to entertain but to inform. Here, people use imagery, they do not consume it. In this space of the lineage of Plato’s pictographic ideal, reside also diagrams and infoGraphics. This space might be considered more businesslike, where imagery is enlisted to help people understand, learn, benefit and attain knowledge.
Futurology
Any tools, technologies, techniques, or toys that let people improve how they play seriously with uncertainty is guaranteed to improve the quality of innovation.’ (Schrage 1999, p.2).
workPlay: a tool for anticipating possible futures? To predict the future is an ancient desire, but for some it is a compelling need. For companies and organisations currently struggling to keep up with an economic and social landscape that appears to shift like quicksand, getting a glimpse of what might be around the next corner, predicting the future has become a compelling need. Any tool or method that can provide some measure of insight into complex systems, events and issues has value in the information economy. One such method is Game Theory. WorkPlay does not crunch numbers, but it provides a platform for developing, mapping-out and communicating visual enactments of possible futures.
Games Theory

Figure 6.

Games Theory (Figure 6) has been around for almost three hundred years. It’s roots are in mathematics and economic forecasting. But, with the advent of computers it has become a powerful tool for analysing complex events and issues. It has been successfully applied to understand social, economic, military and evolutionary theories and strategies, and has recently been applied to AI and cybernetics research. As a prediction tool it is a parallel, although somewhat more robust, cousin of workPlay.
Epistemology

Figure 7. Ferdinand de Saussure

Figure 8.

The way different people think, perceive, and learn is a core consideration in the design of workPlay. Semiotics, the study of signs, comes into play (Figure 7), as does perception (Figure 8). Appreciating historical context and incorporating current theories in epistemology ensure that workPlay will be a relevant, defensible, and effective tool. Aspects of Howard Gardner’s theory of multiple intelligences and the TruColors personality system influenced the decision to incorporate a range of user options and inter activities on the publishing side. Rudolf Arnheim’s theory that all thinking is visual and other theories that support the value of imagery in communications have influenced the strong visual focus of workPlay.
Visualisation
‘In order to have actionable meaning, the fuzzy mental models in top-management minds must ultimately be externalised in representations the enterprise can grasp.’ (Schrage 1999, p.14). Companies and organisations are beginning to realise the value of visualising their products, processes and activities. The pace at which products and services are now developed demands it, and the increased capacity of computers and communications makes it possible. Spreadsheet software was developed to reduce tedium but it was soon recognised as a valuable tool for asking ‘what if’ questions, for making predictions, and for winning arguments. Tools that enable people to play with the possible, change how people interact and the types of activities they engage in. WorkPlay has the potential to be such a tool. Although the system may be viewed as a simple storytelling device, it is possible that some of its other qualities, such as cost, ease-of-use and availability(the same qualities responsible for the success of Lotus 1,2,3 and VisiThink) may lead to it’s application in other, perhaps unexpected, areas.
Visualisation in action
The U.S. Department of Homeland Security chartered the National Visualisation and Analytics Center (NVAC™) in 2004 with the goal of helping to counter future terrorist attacks in the U.S. and around the globe. A major objective for NVAC is to define a five-year research and development agenda for visual analytics to address the most pressing needs in R&D to facilitate advanced analytical insight. (http://nvac.pnl.gov/agenda.stm)
Authorship
There are two recent sociological trends related to authorship that have an immediate impact on design of workPlay and a potential impact on the climate into which it might be launched: ubiquitous authorship – a dynamic of post-modernism – is a popular movement by members of the general public to author all manner of ‘content’, a realm that was previously reserved for trained professionals; and, self-authoring - a response to time constraints and security issues related to the disclosure of intellectual property. Both are connected to shifts in economic, social and technological change.
Ubiquitous authorship

Figure 9. Ubiquitous authorship

Social indicators of the trend towards ubiquitous authorship include: so-called, reality TV; the indie music scene; online communities such as Facebook and U-Tube, and a plethora of Wiki’s and Blogs. In the visual arts; meaningful responses to major events, such as Hiroshima and 9/11, coming from a collective rather than a single spokesperson, as was the case with Picasso’s Guernica.
IP security
IP is largely intangible, extremely mobile and often highly valuable. For companies creating new products and services, working with outside sources during development can pose a serious risk of disclosure and loss of IP. Providing people with the capacity to develop their IP in-house was the impetus for developing early versions of workPlay.
Collaboration

Figure 10. Collaboration

Another dynamic spin-off of the information age is the growth in the numbers of people collaborating on work projects (Figure 10). Collaboration is one of the three primary ways in which humans interact (the other two are conversations and transactions). Technology has erased the time and cost considerations that previously defined the type and quality of collaboration that was practical and possible. Collaboration tools, or GroupWare, are a growth area of information technology(IT). They fall into three categories; communication tools, conferencing tools, and collaborative management tools. WorkPlay appears to fulfil the requirements of a conferencing tool, and perhaps some of those of a collaborative management tool.

Applications for workPlay collaborations

Application 1: A cross-department group might use WorkPlay to illustrate the features and benefits of a future product for use in a presentation to another group or to higher management. While two members of the group may work together to write the script, another member may assemble scenes, while yet another plans interactive features. Drafts are circulated and discussions are conducted via video conferencing or Synchronous conferencing.
Application 2: A school teacher may design a learning module that lets her pupils interact with a character in a historical story. The story is printed out for discussion in class. then posted online so the students can access it and interact with it at any time. The simple interactions allow students to choose different levels at which to read the story, so the teacher knows that all her pupils will be able to understand the story at some level. The inter activity encourages the students to explore higher levels of ‘reading’ in easy stages.

Studies

Figure 1.

Studies were conducted throughout the development phase to determine various aspects of design. The main studies include:

1. Egyptian pictographs.

2. Sequential narrative.

3. A comparison between Egyptian art and Star Trek Next Generation.

4. The scope of the figurative and graphical system.

5. Levels of graphical complexity, and

6. Design issues.

Study 1. Ancient Egyptian pictographs
The ancient Egyptian pictographic system ‘was not evolving towards something better; it was a fully fledged representational system – subtle, mature, and conveying precisely the information that the Egyptians wanted conveyed.’ (Robins 1994, p.21).
Design questions arise when considering the adoption of particular aspects of the ancient Egyptian figurative system as elements of a contemporary form of graphic communication (Figure 1):
• What elements make up the system?
• How do they convey information?
• What makes them successful and enduring?
• What enables them to be universally understood?

Design

The primary objective of Egyptian hieroglyphic language is to convey information. There may have been all manner of political and social reasons for creating such a language, but in terms of semiotics, Egyptian hieroglyphs are an ellaborate system of symbols designed to communicate concepts, ideas and stories. Using design principals that do not differ greatly from those used today, the ancient Egyptians developed and employed a combination of framing (preparing a surface, delineating the overall shape, marking borders) hieroglyphs (writing), cartouches (logos) and imagery (illustrations) to convey their message and enable it to be systematically interpreted and understood. The Egyptians made use of square grids, developed standard forms for writing and illustrating figures, props and scenery, and employed dynamic design principals, such as contrast, shape, direction, and colour.

Size

The use of size to give importance to certain elements. While a designer, illustrator or cinematographer may position an important element in the foreground of a design or scene to give it prominence, the Egyptians, who show signs in their work that they were aware of the illusion of perspective but never felt the need to fully employed it, simply made important elements bigger, and less important elements smaller. Children left to their own devices do the same thing, often making the heads on figures proportionally larger, and making people that they know well bigger than those they know less well. In fact, can this concept of size relationships be traced to the parent/child relationship? A parent is larger than a child.

Position

The social hierarchy or the position that people held in society is mirrored in their position in the design. Hence, royalty holds a central position, while lesser mortals cling to the margins.

Contour

The use of contour to provide a clear indication of form is a key element shared by art and design. The almost rigid application of this principal – to use the most recognisable contour for a given form – became a characteristic style of Egyptian figurative art. The head is seen only in profile, as are legs, feet and hands, while the chest and the eye is always drawn in front view. Other contour lines play dual roles; the line that forms the front view of the chest turns into a side view of the belly, and the line forming the opposite side of the chest turns into the profile of the buttocks. This midsection twist is characteristic of the Egyptian ‘style’, very cleverly mixing views to provide an ideal symbol of a complex form.

Study 2. Sequential narrative

Figure 2.

To begin to understand sequential narrative, a study was conducted of the Bayeux tapestry. The study considered some of the design principals at work in a piece of visual narrative that has stood the test of time, and to identify elements that might be incorporated into the workPlay figurative system.

Shape

The tapestry is very, very long and narrow (68.38M long x 46-53cm high). The most appropriate design for a work of linear narrative, which in general, it is. But not entirely, certain events are not in chronological sequence.

Spatial design

The length of the work is divided into three registers, a main centre panel that is used to depict scenes, and a margin that originally bordered the work on all sides that is ‘sometimes ornamental; sometimes they represent fables; sometimes they are used to portray a sub-plot and even possibly to foreshadow future events.’ (Wilson 1985, p.11). There are also occasions when they give way to larger scenes. Because the work is so long and one of the ends is missing, the upper and lower margins effectively become registers used to organise the scenic information.

Figurative stylisation

The depiction of figures is less rigid than the formal art of the Egyptians (some similarities with the art in the time of Arkhenatan, 1353 BC-1336 BC or 1351 BC–1334 BC, where the traditions were relaxed). The use of profiles is used a lot, but by no means routinely, strengthening the directional aspect of the narrative. There are examples of faces seen from the front or three-quarter view. Major figures, such as kings are often a little more naturally seen. Figure proportions are fairly fluid; sometimes hands are far too large or stretched out, heads can be unusually large, ratio of rider to horse can vary. This adds a sketchy, spontaneous quality that works for action scenes well.

Picture devices

Although most of the scenes use a ground line similar to the Egyptian system, it is not used consistently throughout, and it is often more than just a line. It changes to a series of ‘bumps’, or, where terrain is important, the ground line gives way to a contour line that rises into the scene suggesting a hill or rough ground. This follows the Egyptian style. Figures generally are anchored to the ground line, but may float above it or break through it. Elsewhere the ‘ground’ of the picture plane is used in a similar way to that of Chinese perspective. While foreground figures sit on the ground line, figures and objects appear raised behind them with no attempt to anchor them on a second register.

Narrative

Main characters appear several times throughout the story with no attempt to any likeness between their instances.

Colour

Five colours dominate the work; terra-cotta, blue-green, old gold, olive green, and blue. There is use of dark blue and black and some olive green. They are not used naturalistically. A horse may be blue with red ankles.

Pattern and rhythm

Colours and the blocking of certain elements appear to have been chosen to first delineate object from object, but also to create an attractive sense of interest and rhythm. Repeating figures, shields, spears, horses, etc.

Text

Text runs throughout the work, sometimes anchored to the upper register in a line or flowing down into the scene; sometimes in the upper margin, and important places or people are ‘tagged’ with text in the scenes.

Storyboards AND graphic scenarios

Figure 3.

Figure 4.

Similar in appearance to storyboards(Figure 3), graphic scenarios were developed as a visualisation tool for the technology sector. Text-based scenarios are commonly used to map software functionality and to test usability. Graphic scenarios are used to present concepts or to enhance dialog in the development of future products.

WorkPlay is a natural extension of graphic scenarios (Figure 4), by enabling the scenario author to illustrate their own stories it is more accessible, as a software application it has more functionality and connectivity with existing tools.

Study 3. comparison study: Egyptian pictographic art and StarTrek TNG

Figure 5.

In contrast to the original Star Trek series, which had an American frontier quality, StarTrek TNG appears to have some classical themes woven into it. This comparative study started out as a bit of a joke! But, turned out to reveal some grains of truth (Figure 5).
Goals of the study:

• To consider, as a fundamental starting point, what the system must achieve by way of the three unity’s that Aristotle sites as essential elements of storytelling: time, place, and action.

• To establish that the figure system can quickly and easily be adapted to a complex story of approx 42 frames.

• To work out a system for depicting both sequential and non-sequential narrative in the same story, since the story involves time shifts and events that happen in parallel.

• To provide a platform for testing and understanding the semiotics at work in such a system. To test a range of minimal graphic signifiers that identify characters and props, discover the threshold where recognition starts to falter, establish an optimal level of detail, and settle on an approach to the ‘pictographic method’ that is sufficient to convey to the reader what is happening and who everyone is.

• To ensure that characters can be sustained across a range of situations, events and scenes, i.e. If a character changes clothes will they still be recognisable? Since almost all the heads are identical, and heads are a primary signifier, clothing that hide key features, such as hats could cause recognition issues.

• To find an optimum balance of text(as little as possible), imagery and graphics.

Study 4. the scope of the graphics system

Figure 6.

This study was developed to test the scope of the workPlay system. The following aspects were included in the study: • application of colour, • optimum number of body parts, • comprehension of the pictographic system, • acceptability of the figurative system, and • text layout.

Method

The method used to complete the study consisted of the following steps:

• Source a good story.

• Create a thumbnail sketch storyboard.

• Create a library of properties and actors cast in a set pose (Figure 6).

• Assemble scenes using the graphics library.

• Assemble text and scenes.

• Output for the storyboard for presentation.

• Create a questionnaire.

• Present the storyboard to a small group of participants.

• Solicit answers to the questionnaire.

• Analyse and reflect on the responses.

• Modify the system where necessary.

Outcome

The study was conducted with students enrolled in the MA: Design Multimedia and Graphics course at Sunderland University (2007). Seven members of the group completed questionnaires. The responses suggested that the presentation material needed some refinements before it could be used in a wider context. Although the length of the story successfully validated the ability of the system to convey an epic story, it was considered to be too long for use in focus group studies.

Analysis

Comprehension of the story was fair to good. Identification of the actors appears to be poor. The actions in the story were clear to all but one respondent. Some of these responses are felt to be due to the length and complexity of the story, and the incomplete nature of the material.

Study 5. Graphical complexity

Figure 7.

Figure 8.

A chart designed to put levels of conceptual complexity into context (Figures 7 and 8). workPlay imagery falls into the ‘class symbol’ category.
Study 6. Design issues

Figure 9.

Issue 1. Actor identification

The figurative system uses a limited number of body types and body parts in order to simplify production, functionality, and usability (Figure 9). The design question that arises from this is; how to identify specific actors when they all share similar features?

Resolution: The system doesn’t use head shots or close-ups. Therefore, with full figures posed in the frame the facial features become indistinguishable anyway. Surprisingly, because of the minimalist approach to design, simple changes of hairstyles and clothing become every strong identifying elements. The published size of the work may call for a revision to this resolution.

Issue 2. Use of colour

Figure 10.

Colour is a powerful design element in visual systems. In this system, colour is used selectively to identify and differentiate. Questions that arise from this are; how, where, and when to use colour?

Resolution: The use of full, mimetic colour appears to be unnecessary and at odds with the fact that the imagery system does not pretend to emulate ‘reality’ or photographic qualities (Figure 10). During tests on large sets of actors, the application of an overall tint to identify a group proved successful. Showing divisions within large groups by using a secondary identifying tint appears to provide further clarity for the reader. As yet, it is inconclusive whether linking the tints to other elements in the system, such as locations(office, vehicle, etc.), enhances comprehension of the narrative.

The number of colours available for use in identification tints is not many. Six are easily distinguishable from each other. Further tests will prove whether more than six are necessary.

Issue 3: Frames and boxes

Traditional comics and storyboards use a ruled frame around the image, and often use boxes around narrative text and dialog. The design question that arises from this is: what effects do frames and boxes have on the design and are they necessary?

Resolution: There is an immediate, inescapable connection between the cropped image and photography. Before the advent of photography, drawings and paintings were never cropped by a frame, they were designed to fit within it. The frame defines a boundary between one element and another – image space and non-image space or image space and dialog space. These boundaries are convenient graphic devices that help to organise the design space, but, they are also small hurdles that break the rhythm of the roving eye. The absence of a frame supports the notion that the reader does not need a ‘window’ through which to look, but rather enters the space in an immersive way.

Bibliography

Cotton, B. (2000) The smart, global, networked, multi-user, multi-story machine. In; The Visual-Narrative Matrix. Coulter-Smith, G., Walker, J. A. And Owen, M. (Eds.) : Fine Art Research Centre, Southampton Institute.
de Bono, E. (1999) 6 Thinking Hats. Little Brown And Company.
Gardner, H. (1993) The Frames of Mind: The theory of multiple intelligences. Basic Books.
Gombrich, E. (1977) Art and Illusion. Phaidon Press Ltd.
Gombrich, E. (1999) The Uses of Images: Studies in the social function of art and visual communication. Phaidon Press Ltd.
Greenough, H. (1958) Form and Function: Remarks on Art, Design and Architecture. University of California Press.
Neiva, E. (1999) Redefining the Image: Mimesis, Convention, and Semiotics. Dissertation.
Robins, G. (1994) Proportion and style in ancient Egyptian art. University of Texas Press.
Schrage, M. (1999) Serious Play: How the world’s best companies simulate to innovate. Boston, MA: Harvard Business School Press.
Sullivan, L. H. (2010) The Tall Office Building Artistically Considered, GIA Reader, 21 (3).
Wilson, D. M. (1985) The Bayeux tapestry. Thames and Hudson.