Tim Wray

collections as landscapes : part 1 – empowering spatial, experiential interaction

Can we build digital products that induce positive, curious experiences? Is there a way to perhaps algorithmically curate collections? The possibilities of building engaging experiences from digitised collections are endless. New mediums provide tantalising ways of looking at works that weren’t previously possible. Via exploratory design, I’m interested in how we can build interfaces that reveal and unravel narratives within collections.

In a series of posts, I will outline insights revealed through my research and a summary of key academic literature leading to the design behind my up and coming app – A Place for Art. In this post, I will explain the concepts of the spatial organising framework of the container and the landscape. I hint at the necessary shift from the former to the latter as a mechanism for providing context for objects, and how landscapes – combined with engaging interaction designs and the notion of pliability – can used as a way of providing immersive experiences for museum collections.

a world of containers

Humans have a desire to group things together. We construct concepts – units of thought – as we instinctively form groupings via commonalities, analogies and shared associations. We reduce the world into cognitive fragments so that we can make sense of it. The ancient organising principle of the library: collocation – or grouping like things together – is based on this instinct.

One of the most enduring tools we’ve developed to achieve collocation is the notion of the container, a term coined by new media theorist Janet Murray. Often manifested as lists and tables, containers are nothing more than aggregations. They are some of the oldest artefacts of written culture and as such, remain as the most pervasive approaches towards information organisation. For example, the Library of Congress Classification system – with its 21 top-level categories – are containers that are supposed to account for everything in its own collection. These categories embody their own containers, or sub-categories, that place objects in ever-finer delineations.

As an enduring strategy for organising physical objects, containers are ubiquitous in modern user interfaces – menus, toolbars and browser tabs seek to group commands and data displays together. The way information is organised and presented on the Web inherits the containment concept – the various sections of news websites (Local, National, Weather, etc.), or the various divisions of a university website (often grouped by faculties and departments – organisational containers) provide a spatial organisational framework for retrieval and navigation.

Containers can also be used to as tools for aggregation and as a way of collating content from diverse, disparate sources. For example, Murray describes the Flipboard application for the iPad as a cluster of containers : articles from diverse news sources are formatted as a grid structure, providing glimpses of their content within the contained pieces. The pieces themselves are organised within a virtual container – the Flipboard application, which itself exists within a physical container, the iPad.

In recent years, the rise of mobile and tablet devices with their pseudo-standard form factors have provided ways for containers to present, isolate and aggegrate affordances and experiences. For example, the Windows Metro UI organises the display of information as a series of tiles : presented as an conglomeration of containers that flow laterally beyond the boundaries of the screen. Each of these containers serves as a window: from glimpses to your social, working and daily life to news snippets and events from around the world. In the sections following, I will describe the pervasiveness of containers within online collections : both as modes of information organisation, and as manifestations of interface designs.

Online collection websites naturally inherent the containment concept from their library forebears and their presentation is reflected in the dominant paradigm of containment on the Web. For example, objects are often grouped by category (e.g. The ‘Browse Categories‘ page of the Powerhouse’s Online Collection site presents us with list of categories – a container of containers);  sub-collections that are inherent within the organisation (e.g. Brooklyn Museum’s archive of ‘American Art’, ‘Asian Art’, ‘Contemporary Art’) or by artist (as is seen within the Museum of Contemporary Art’s online collections site). Collection websites are purported to provide opportunities for browsing and exploring : links and buttons entice us to ‘Explore Archives’ and ‘Browse Collections’, yet more often than not, we’re presented with a list of categories – a series of stuffy filing cabinets, the objects locked away in containers.

Some online collections provide a more compelling introduction their contents : thumbnail previews, and visual snapshots of each category provide the visitor with a compelling reason to explore, but for the most part, the fundamental and difficult question of “where do I begin” is answered by a list. This containment approach dictates the visitor experience : aside from the nostalgic search box, a typical visitor jump straight into and across containers, effectively slicing the collection into narrow silos.

While containers are convenient tools for organisation and retrieval, as human artefacts, they are also biased, prejudiced and unforgiving. Would it be possible for the “objects to speak for themselves” in how they were to be organised, rather than be spoken by its containment? This enlightenment is particularly pressing given that the potential for audiences to contribute their own ideas, thoughts and conjectures on the objects that contravenes the very nature of rigid containment, authoritative metadata, standard vocabularies and in general, structured organisation.

Containers are also inherently culturally biased - for example, The Library of Congress Classification is surely not objective : with two of its 21 top level categories devoted to American history, and another two devoted to military science. Rather, could we allow the organising framework and presentation of objects to reflect an exhibition: a mix-tape, an essay in n-dimensions?

rethinking collections

The desire to group things together, and the artificiality of containers as a way of doing so, is reiterated by Julia Flanders, director of the Women’s Writer project, as she discusses her views of “Rethinking Collections”. From her keynote speech at the inaugural conference on the Australasian Association for Digital Humanities, she again speaks of collocation and human nature:

The term “collection” speaks immediately of agency and emotion – it speaks of grouping – the need to overcome a natural entropy of the universe – a desire to bound things together. A collection is an aggregation of individual terms.

It was attended by a mass of questions and ideas circulating discussing how on the conscious (perhaps unconscious) modelling activities behind define a collection – i.e., the existence of a collection via selection and agency. She describes a fundamental set of dimensions that describe the lifecycle of a collection from its inception, construction aggregration, modelling, analysis and discovery – the first three being arguably human and conscious design elements intermediated by technological artefacts (that also embody conscious, human design elements). The overall question of the discussion was, “what are we modelling in the design of a collection?”

• Agency - the decisions that lead to the development of the collection : sponsorship, curation and readership.
• Design - the structure, encoding, representation and interface design of the collection.
• Boundaries and Contents - The selection of objects, the parameters that define that selection, and the degree that we are representing them.
• Variation - The degree of homogeneity or heterogeneity within the collection.
• Phenomena – explicit and implicit - the revealing of relationships that are apparent within the collection via interaction techniques (for e.g., navigating via predefined views and perspectives) or via commonalities and patterns that are revealed via analysis and data mining.
• Readerly knowledge - build in assumptions about its audience.

Another theme concentrated on the importance of the collection (or its ‘sub-collections’) as discrete cultural artefacts, and the need to model or represent their implicit relationships, as quoted by Dr. Axel Bruns:

Collections also tend to imply that their data and metadata will be modelled at some point: that there is an aim to gain a purchase on their internal semantics, to represent such semantic data in a meaningful, useful way – through a user interface of some form.

This significance of this philosophical discussion lends itself to the following question : are containers an appropriate way to organise and present collections? Containers are limited in that they only demonstrate, effectively, that an object belongs somewhere, in some categorie(s) or within the dreaded ‘misc’ and ‘etc.’ Containers also impose an arbitrary, artificially generated structure that may not be a true reflection of the muliplicity of relationships that the objects offer, or the multitude of expressions that are conveyed by their interpretations. Like a bygone age of computing in general, containers bely an undue emphasis on their storage, portability and retrieval.

from containers to landscapes

My argument is that the rich multiplicity of relationships among objects in a collection inherently exposes the limitations of the container, and perhaps are more liberating framework is necessary to enunciate Flander’s explicit and implicit phenomena. Again I refer to Janet Murray, using her landscape metaphor as a framework for information organisation : a framework that emphasises space, open-endedness and freedom.

Landscapes are perhaps most predominant in contemporary game design, with their florid 3D worlds and spatial expansiveness conveying a sense of disbelief as players immediately immerse themselves in the minds of the characters. They present navigable spaces that are in tune with the natural human path-finding instincts of our ancestors. The freedom to roam these landscapes is lends itself to the spatial expansiveness, open-endedness and freedom that physical exhibitions offer. As a framework for experiential interaction, landscapes are liberating in that they provide its inhabitants – explorers and wanders – to embody a creative mind, to explore, to spectate.

These allusions of the character and the landscape are well articulated in The Information Flaneur that introduces a literary figure – a poetic persona that embodies the urban flaneur – as a character who explores for pleasure, for curiousity, without direction or task-based goals. The discussion draws parallels between modern cities and information landscapes, and more broadly, speaks of the experiential nature of information seeking activities that seeks to go beyond the task-based notions of exploring and browsing:

We are particularly interested in his exploratory mindset. In order to experience the city, the flaneur does not methodically navigate streets, checking each edifice like a building inspector in search of code violations. Nor does the flaneur hastily interrogate each city-dweller, like a police officer in search of a thief. Because the flaneur does not accurately scrutinize everything that crosses his path, he is able to sense what city life is about. The flaneur is the embodiment of exploration and serendipity, while the police officer and building inspector personify traditional search and browsing.

Like the wanderer in Murray’s landscapes, a flaneur must have a sense of orientation – their position within the information space, or at least, options of where to go, how to get there, and a way to move forward (and how to return). Landscapes should encourage serendipity, the ability to find surprises, and the ability to create divergences within the information space.Visual momentum is a particular quality that refers to the visual smoothness of transitions between places and divergences. In 3D games, this momentum is achieved by the game’s character traversing across the vast virtual worlds, but how can digital artifacts, that present us with more abstract representations of landscapes, convey this sense of momentum?

pliability : visual momentum for navigating landscapes

One of the ways we can convey momentum is by designing pliable artefacts, defined by Löwgren as the degree that an interaction feels involving, malleable and tightly coupled to the eyes, the hand and the senses. Pliable artefacts denote a tight connection betweeen the actions and response – a sense that you’re truly ‘moving’ the landscape at your fingertips. These properties in turn facilitate exploration and serendipity in use. To demonstrate pliability, he uses examples of the smooth transition between zoom levels on the Google Maps interface – and the ability to manipulate those transitions through pinching gestures, as opposed to the static point and click, page-by-page interactions of the fixed map images seen in rival mapping software Eniro. Likewise, the iPhone’s address book shows an interface that is both pliable – the tight coupling between your fingers and a scrolling rolodex, that also conveys a sense of visual momentum : the way the rolodex continues to scroll after you lift your finger, but also in the way that it coveys the state changes between the address book list and the contact details via its use of animated segues.

Digital artifacts exhibit a unique design problem that its experiential qualities are notoriously difficult to sketch, and it is only through the constructing of prototypes and sensing their look and feel that insights into what ‘feels right’ can be gained. Much of the complexity of these artefacts, via its design or its “contents” – emerge only in sustained interaction.

Smooth pliable interactions are purported to increase pleasure and a sense of immersion with its contents. Nicolaas Earnshaw discusses the important concept of “frictionless” experiences, both in the pliability of the digital artifact and the way it can seemlessly integrates with the museum experience. Friction could be described as a barrier that could prevent the visitors from having positive, sustained experiences with the interface, and more deeply, with the underlying landscape of the collection itself. In terms of the urban flaneur, friction acts as a metaphorical roadblock or construction site that forces his or her attention away from the gaze of the cityscape and and onto his footsteps : skirting away from the potholes, following the detour signs and losing a sense of orientation. The same feeling could be said for an awkward navigation design, a non-responsive interface, a QR code that doesn’t scan, or the need to suddenly memorise a set of complex steps and URLs.

Although these abstractions of frictionless, pliable experience lend to an abstract and almost romantic set of design principles, examples of such interfaces do exist, or at least, exhibit some of these more intuitive qualities. For example, 500px’s iPad app lends itself to these notions of pliability beautifully. It immediately opens up with a seemingly endless stream of beautiful photographs – organised into grids of paged thumbnails – that can be flicked across the screen.

We perceive the app as a container, an unobtrusive invisible one that doesn’t get in the way of this alluring mosaic. The seductive quality of the app is demonstrated by the beauty of the photos within – 500px heavily emphasises content that is considered to be ‘beautiful’ by members of its user community. In essence, beauty is ranked, crowdsourced, and packaged within a container.

The designers of the app have made a very smart decision to create a pliable, frictionless, non-obstructive experience : the images within each off-screen horizontal page are cached and preloaded so that when the visitor swipes to the left or the right, the images slide in effortlessly and are not preceded by a blank ‘loading’ screen – thus preserving visual momentum. The container – which in this case, is the entire iPad’s display, serves as a window over a vast  landscape of photographs and allows their aesthetic qualities to define the user experience. The affective, emotional reaction of the photographs lends itself to the alluring simplicity of the app. Here we see no complex data structures and hyperbolic graphs – qualities that are often cited in ‘explorative interfaces’. This comes to show that effective, captivating experiences do not need to rely on needlessly complex mechanics and computational prowess to be considered ‘innovative.’

Some collection apps have embraced a highly spatial approach towards experience design. The idea of ‘roaming’ is taken to an almost literal level in MoMA’s AB EX NY app, where works are splayed across a virtual exhibition wall, stretching beyond the container of the screen. A rarely seen quality is the sense of scale of works as their proportions are rendered true to one another. It is surprising that a work’s imposition or intimacy can be rendered on a 10 inch screen.

Another pliable experience lies in the Design Museum’s app where again, it is the objects that define the aesthetics of the interface, utilising the entire viewport as a container. It presents us with a functionally pointless, but playful and alluring grid of sliders – each slider responds to a flick in a horizontal or vertical direction that allows us to discover new objects off-screen that in turn provide a certain element of fun and surprise.

The whole experience feels like your playing with a child’s toy, an inescapable puzzle game with the collection. Applying the many filters on the collection is a spectacle in and itself – tiles flip and spin, hiding and showing the objects from view as you narrow down to your favourite category, type, designer or colour.

Such faceted browsing and filtering features have been around for decades, yet very few applications provide such testament to expressive visual momentum and joyful pliable experiences, empowering the curious explorer to not only explore the landscape (to path find, to diverge, to leave breadcrumbs, to explore ‘related links’ etc.) but to manipulate it (to filter, to expand, to zoom). A shift towards towards a more aesthetic, cultural and humanist approach in human computer interaction and interaction design – or what Udsen and Jørgenson calls ‘the aesthetic turn‘ – entails a broader perspective goes radically beyond traditional notions of pure functionalism and exclusively use-oriented approaches. Both the 500px and the Design Museum app are examples of interaction designs that inform, challenge and excite. In line with the experiential nature of the museum visit and their role to engage audiences with collections, designers of museum apps should focus on ways to provide experiences that are emotional, engaging and seductive.

Another engaging experience, one that is more planted within the in-gallery visit itself, is MAD Museum’s in-gallery exhibit that allows you to interact with their online collection. Presented as a large touch screen near one of the waiting areas of the museum, visitors can pinch, swipe and zoom a large grid of objects – again, the screen’s container serves as purely as a medium for the ‘distant landscape’ of the rich, heterogeneous offerings of the collection. Oonagh’s review described the minimalism and emotional appeal and joy of this unexpected device:

No text, just objects – this was refreshing, it was nice just to click on things that looked nice. When you did the screen zoomed in, and collection information was displayed. The visual layout, and swipe and zoom features created a captivating experience. I know any time I watch CSI or any number of crime dramas I’m always impressed by the swishing and zooming of their large scale interactive tables and MAD Museum has certainly incorporated the joy of swishing, zooming, skim reading and visual search into this interactive.

The rich offerings of a collection, appropriated in a container where it’s interaction design evokes serendipity, naturally fits in with the surroundings of the museum, itself a (physical) container for the non-virtual explorer (i.e., you, the visitor). But even with the most pleasing and engaging interfaces in beautifully constructed physical containers, could we go beyond the restrictions of the data container - the dreaded ‘exclusive category’? Could we apply the landscape metaphor to very properties that reveal Flanders’ implicit and explicit collection phenomena? Could the principles of pliability and immersion apply to how the data itself?

from data containers to data landscapes

A more expansive, liberating way of organising things can be revealed via the use of categories that do not conform to a hierarchy – effectively containers that exhibit flexibility and polymorphism : properties of union, conjunction or multiple inheritance.

For example, Samuel Wilkinson’s Plumen 001 exists in a multitude of categories across dimensions (“it is one of many consumer products”, “it is one of many light bulbs”,  ”it was one of many products designed in the 21st century”) and interpretations and natural ambiguities (“does it belong in the design classics collection? the decorative arts? perhaps as part of a collection of simple, geometric works?”). An object’s interpretations, conjectures and iconography can also be classed as discrete categories – it exists among beautiful objects, it exists among functional objects, it exists among harmonious objects. A category can also be a conjunction of other categories (“it exists among beautiful geometric works of the 21st century … “) each exhibiting multiple inheritance (” … that also exist among other geometric works from other time periods, or other beautiful works from the 21st century”), each with their own almagations of other, notionally similar categories.

In the next part of this post, I will describe how computational techniques can be used to overcome rigid containment, providing a spatial organising framework for exploration. A shift from data containers to data landscapes. Connecting objects to one another. A way to construct narratives for positive, exploratory experiences.