Barbara Pfenningstorff

MSc Built Environment/ Virtual Environments

Bartlett School of Graduate Studies

Theoretical Analysis on Virtual Environments V09

May 2000

INHABITING INFORMATION -

THE ARCHITECTURE OF COGNITIVE AMPLIFICATION

Fig.1 Detail of First Page of the Ars Memoriae in Robert Fludd’s Utriusque Cosmi…Historia, Tomus Secundus Oppenheim, 1619

0. Contents

1. Introduction

2. Cognitive Theory

3. Cognitive Structures: Trees, Landscapes, Networks and Galaxies

4. Movement and Interaction

5. Information Buildings: Historical and Contemporary Examples

6. The Architecture of Cognitive Amplification

7. Conclusion

1. Introduction

As computers, media and telecommunication technologies continue to collect, manipulate and store an ever- increasing influx of data, they are installing a new dimension: the space of information. This virtual space is accessed only through the media of imaginative and technical representations. How well this information space is engaged depends on the manipulation and inhabitation of the representations which define the interface to cognitive processes.

Inevitably, information theory becomes information praxis: What are its possible logics, cartographies, entities and connections? And finally: How does it relate to architecture, the traditional art of space?

Information Visualization

This essay sets out to answer the above questions under consideration of the latest development in the field of Information Visualization (Card, Mackinlay, Shneiderman, 1999). Starting with the introduction of the Silicon Graphics workstation in the 1980ies, PCs are now coming to support real- time, dynamic, interactive visual representations. With this development, the field is in the process of passing out of the realm of academic research into the mainstream of user interface and application design.

Information visualization derives from several communities. Work in data graphics dates from about the time of James Playfair (1786), who seems to have been among the earliest to use abstract visual properties such as line and area to represent data visually. Edward Tufte (1983) published a theory of data graphics that emphasized maximising the density of useful information in a 2- dimensional plane. His theories became well known and influential to the development of information visualization as a discipline.

The Art of Memory

The ancient Greeks, to whom a trained memory was of vital importance- as to everyone before the invention of printing- invented an elaborate system of memorization, based on a technique of impressing places and images on the mind (Yates, 1966). These images, which related to things to be remembered, could later by recovered by virtually walking through the places. Inherited and recorded by the Romans, this art of memory passed into the European tradition, to be revived in occult form in the Renaissance.

Walter Benjamin outlined in his Theses on the Philosophy of History that ‘when one grasps the mental constellation which his own era has formed with a definite early one, one establishes a conception of the present as the ‘time of now’’.

This essay therefore, based on the emerging field of information visualization, will be supported by research into historical examples of cognitive spatialization in order to establish a temporal context for its techniques.

The essay will give a short introduction into the fundamentals of cognitive theory, which in combination with perceptual theory forms the basis for both internal cognition, as exercised in the art of memory and its external complementary form of information visualization. Based on the theory of thought, the organization of information into the patterns of cognitive structures will be explored. Paragraph 4 will focus on the notion of user movement and interaction, which are the core of the augmenting power of both arts. The later paragraphs will explore the relationship of the cognitive structures of paragraph 3 to various architectural structures of physically constructed buildings. The conclusion finally will establish this essay as the theoretical basis for the future development of an architecture of cognitive amplification in applied projects.

2. Cognitive Theory

Perception- Imagination- Thought

Aristotle’s theory of memory and reminiscence is based on the theory of knowledge which he expounds in his De anima .

The perceptions brought in by the five senses are first worked upon by the faculty of imagination, and it is the images so formed which become the material of the intellectual faculty. Imagination is the intermediary between perception and thought. Hence ‘ the thinking faculty thinks of its forms in mental pictures’. (De anima 431^b 2.)

The forming of the mental image he thinks of as a movement, like the movement of making a seal on wax with a signet ring. This metaphor compares the inner writing with writing on a waxed tablet suggested by the contemporary use of the waxed tablet for writing. Cicero refers to this metaphor as follows:

‘He inferred that persons desiring to train this faculty [of memory] must select places and form mental images in the places, so that the order of the places will preserve the order of the things, and the images of the things will denote the things themselves, and we shall employ the places and images respectively as a wax writing- tablet and the letters written on it.‘

Cicero, De Oratore, II, lxxxvi, 351-4.

Memory and Reminiscence

In his book De memoria et reminiscentia. Aristotle defines memory as belonging to the same part of the soul as the imagination; it is a collection of mental images from sense impressions but with a time element added, for the mental images of memory are not from perception of things present but of things past. Recollection is the recovery of knowledge from memory. By the principle of association, we find our way among a series of remembered things or events. We can start from any locus in the series and move either forwards or backwards from it.

Mnemotechnics

The Greeks invented an art of memory which, like their other arts, was passed on to Rome whence it descended in the European tradition. It was an art which belonged to the rhetoric tradition as a technique which enabled the orator to deliver long speeches from memory with unfailing accuracy. In the ages before printing such a trained memory was vitally important. The art consisted of mentally creating a series of places or loci. The commonest of mnemonic place systems used was the architectural type: The spaces of a building were to be remembered and rigorously reconstructed according to rules regarding right size and lighting. Within the space were then placed images of things or words to be remembered, ranging from striking figures of bloody goods to simple emblems like anchors or swords. The orator would later be able to walk through his imaginary building whilst he was making his speech, recovering from the memorized places the images he had placed on them. Orators therefore were able to recite their speeches backwards as well, by simply ‘walking’ in the opposite direction.

Contemporary architecture continued to be used as a cognitive structure for memory buildings until the late Renaissance.

Internal / External Cognition

The location for the practice of the art of memory is the imagination, which is the intermediary between perception and thought. The process of imagination can also be called internal cognition, as described by Aristotle in his De Anima. To understand the intuition behind information visualization on the other hand, it is important to acknowledge the role of the external world in thought and reasoning. This notion is sometimes called external cognition (Scaife and Rogers, 1996) to express the way in which internal and external representations and processing weave together in thought.

The field of Information Visualization amplifies the internal cognition by constructing a spatial external working memory, supported by the computer. The interweaving of interior mental action and external perception is the essence of the achievement of expanded intelligence. Without external aids memory, thought and reasoning are all constrained. Information Visualization is the development of external aids that enhance cognitive abilities.

Definition:

Information Visualization:

The use of computer- supported, interactive, visual representations of abstract data to amplify cognition

(Card, Mackinlay, Shneiderman, 1999).

Knowledge Crystallization

The major purpose of information visualization is the notion of knowledge crystallization which involves getting insight about data relative to some task. An example for such a task could be research for the purchase of a computer. Such a task is one in which a person gathers information for some purpose, makes sense of it by constructing a representational framework (schema), and then packages it in some communication or action. This process of abstraction or schematization and omission of information is a fundamental principle of how an information processing organism or machine reduces the otherwise unmanageable glut of information to an amount that can be processed.

Cognitive Amplification by Visualization

Human perception divides what is seen into areas of focus and periphery. As objects are exploited, their locations become visually indexed so that search time to relocate them is reduced. The dimensions of space or patterns on the space itself, such as lines joining nodes, may be assigned meanings. As a result, objects may form a spatial external working memory. Enlarging working memory can lead to dramatic improvements of cognitive functions.

3. Cognitive Structures: Trees, Landscapes, Networks and Galaxies

Mapping Data to Visual Form

Raw data come in many forms, from spreadsheets to the text of novels. The usual strategy is to transform this data into a set of relations that are more structured and thus easier to map to visual forms. Raw data are mapped into data tables, because they clearly depict the number of variables associated with a collection of data.

Data tables are transformed into visual structures, which combine spatial substrates, marks and graphical properties. Finally view transformations create views of visual structures by specifying graphical parameters such as position and scaling.

Fig. 2 Data Table

Spatial position is an excellent encoding for information. The following techniques have been developed to increase the amount of information that can be encoded with it:

.) Composition (Mackinlay, 1986b) is the orthogonal placement of axes, creating a 2D metric space ( Fig.3 ).

.) Alignment ( Mackinlay, 1986b ) is the repetition of an axis at a different position in space ( Fig.4 ).

.) Folding is the continuation of an axis in an orthogonal dimension ( Fig.5 ).

.) Recursion is the repeated subdivision of space ( Fig.6 ).

.) Overloading is the reuse of the same space for the same Data Table (Fig.7 ).

Fig. 3 Spotfire, Ahlberg, Williamson (1992)

Fig. 4 Positions on the New York Stock Exchange (1999)

Fig. 5 SeeSoft uses a folded axis when a software module is too large to fit in the height of the window, Steffen, Sumner (1992)

Fig. 6 Pad++ provides interactive zoom into a recursive space of directories and files, Bederson and Hollan

Fig. 7 Worlds within Worlds, Feiner, Steven, Beshers (1990)

Trees, Networks, Landscapes, Galaxies

Another type of visual structures are trees and networks. These allow relations among objects to be shown without the geometrical constraints implicit in mapping variables onto spatial axes.

Classifications are ubiquitous because they help organize information in a way that hierarchically differentiates objects and reduce the amount of information that users have to cope with at one time. Classification hierarchies for biological species go back to the Swedish botanist Linnaeus, whose Systema Naturae was published in 1735. Modern hierarchies can be complex: the U.S. Library of Congress subject headings, for instance, fill 24 Volumes.

Tree visual structures encode hierarchical data, typically by using connection or containment (fig. 8). Connection is used to create node- link diagrams, a well known technique for encoding relationships between cases. Trees typically start with levels that represent the generations of children nodes.

Networks become necessary when a tree structure is inadequate to capture the relationships among objects (fig. 9). Each node in a rooted tree structure has only one parent, and the root node has no parent. In a network however, each node can be linked to multiple nodes, and there is rarely a single root. Nodes and links can have multiple attributes so that more complex situations may be represented by networks of trees. A major application of network visualization is to hypertexts and the World Wide Web. Some projects attempt to visualize the entire Web, the result of a search or the set of pages visited by a user.

In information landscapes, two axes are used for input variables, forming a sort of topography (fig. 10). The third axis is used for an output variable. Color is used redundantly with the third axis To increase the precision with which the height can be perceived.

Starfield visualizations display document interrelatedness as three- dimensional scatterplots (fig. 11). The key for understanding these visualizations is the notion of ‘document similarity’. The more similar the clusters and documents are to one another in terms of their context and content, the closer and more proximate they are located within the space. By exploring this space, users can quickly gain an understanding of patterns and trends within the visualized data.

Fig. 8 Horizontal Cone Tree, Information Visualization Using 3D Interactive Animation, Robertson, Card, Mackinlay (1993)

Fig. 9 Network, Ben Fry, Asthetics and Computation Group, MIT Media Lab (2000)

Fig. 10 Themescapes, Wise, Pennock, Lantrip (1995)

Fig. 11 The Starlight Information Visualization System, Risch, Rex, Dowson (1997)

4. Movement and Interaction

Temporal Encoding, Animation, Space 3D + Time

Visual structures can also encode space temporally: Perception is very sensitive to changes of objects in position and color. Mapping time data into space allows comparisons between two points in time. For example if we map time and a function of time into space (e.g. time on the x- axis and accumulated rainfall on the y- axis), we can directly experience rates as visual linear slope, and we can experience changes in rates as curves. Tufte (1994) shows a more sophisticated variant in which miniature visualizations are arranged along an axis of time. This display then becomes a control for an animated sequence. Animation can be used to enhance the ability of the user to keep track of changes of view. If the user clicks on some structure causing it to enlarge, animation can effectively convey the change, whereas simply viewing the two end states would be confusing. Another use is to enhance a visual effect, like rotation to allow better reading of some complicated visual mapping.

View Transformations and Interaction

The above paragraph discussed the use of space for mapping data into different visual forms. The computer allows them to be created at the time of use, where rapid interaction fundamentally changes the process of understanding them. Interaction in information visualization implies controlling the parameters of the visualization transformations. One method for doing this is dynamic queries using sliders to control interactive filtering.

The following techniques describe the use of interaction to keep a view of the whole data available, while pursuing detailed analysis of a part of it. One of information visualizations major virtues is to be able to handle information on a very large scale. Scale however is the major usability problem of current interfaces. One very useful interaction technique for handling large information spaces is to coordinate two displays, one giving an overview, the other giving detail (fig. 13). Typically, by adjusting the point of view on the overview, the user can select a portion on the detail display, panning or zooming in to locate information of interest. A related technique is details- on- demand, where a detailed display pops up within a main overview display only when requested (fig. 12). An interesting variant to the overview and detail idea is the notion of focus and context. In this case essentially, the detail area is embedded in the overview. This produces a two- level focus and context display such as a bifocal lens. This permits ‘semantic zooming’, in which zooming for detail does not result in just a strict enlargement (fig. 14).

Fig. 12 The Hyperbolic Browser, Lamping, Rao (1995)

Fig. 13 Overview and Detail, with intermediate view, See Soft, Lucent Corporation (1993)

Fig. 14 Extending Distortion Viewing from 2D to 3D, Carpendale, Cowperthwaite et al. (1997)

Visualization Levels of Use

At the highest level information visualization helps users to access information outside their immediate environment, such as the internet (fig. 15). This information collectively is sometimes called the Infosphere. It is the place to find information needed for work. At the next level an intermediate level of application can be defined as the Information workspace (fig. 16). Information workspaces are oriented not around visualizations themselves, but around tasks. An information workspace might contain several visualizations related to a task. The invention of the desktop metaphor for PCs was an important advance in user interfaces, and information visualization could play a significant role in advancing the techniques of workspaces that can handle information on a larger scale. The information workspace is the place to hold work in progress and to remind the user of materials. At the third level are visual knowledge tools (fig. 16). These tools contain a visual presentation of some data set and a set of controls for interacting with that presentation. These tools constitute the substrate into which data are poured and manipulated. They are used for pattern detection and knowledge crystallization. At a fourth level are visually enhanced objects, coherent information objects enhanced by the addition of information visualization technique. An example might be the medical illustration of a body part or some other physical object, which is known in advance (fig. 17).

Fig. 15 Skitter, Cooperative Association for Internet Data Analysis, Burch, Cheswick (1999)

Fig. 16 The Starlight Information Visualization System, Risch, Rex, Dowson (1997)

Fig. 17 Voxel- Man, IDM University, Hamburg (1994)

5. Information Buildings: Historical and Contemporary Examples

This paragraph focuses on visualization examples which are supported by the use of architectural metaphors. It first investigates historical examples which were imagined internally by the art of memory. These are then followed by two contemporary examples, where information visualization meets architectural buildings.

The most powerful examples of internal cognition as exercised in the art of memory can be found in the Renaissance, when the art culminates in the hermetical, neoplatonist memory systems of Giulio Camillo, Giordano Bruno and Robert Fludd. All of these systems are based on the philosophies of the Egyptians, the Corpus Hermeticum, the philosophies of the Pythagoreans and the Hebrew tradition of the Cabala. Mind and memory are considered divine, having the power to grasp highest reality through magically activated imagination.

The Renaissance Memory Theatre of Giulio Camillo

The first example is Giulio Camillo’s adaptation of the Vitruvian theatre for his mnemonic purposes (fig. 18). The architectural counterpart is Palladio’s Teatro Olympico, which was also based on the Vitruvian theatre (fig. 20). Camillo’s wooden model is large enough to house two people. The auditorium rises in seven grades, which are lavishly decorated and would be too narrow for an audience to sit on (fig. 19). This does not matter as the function of the classical theatre is reversed and a solidary spectator stands where the stage would be and looks towards the auditorium (fig. 19). On each of the seven gangways are seven gates or doors, which in the original theatre were entrances for the spectators. On these doors, memory images are placed, which represent the expansion of the universe from first causes through the stages of creation. Under these images are drawers or boxes, containing speeches based on the works of Cicero, related to is the subjects recalled by the images. The seven grades of the theatre are divided into 3 realms: The inferior world, the heavens and the supercelestial world. Camillo compares the inferior world to a forest of trees, in which we would be lost, were there not the hill into the heavens from which we can see the whole forest from above. The theater served the two following purposes: to find things and words whenever needed and to give true wisdom, as the idea of memory is organically geared to the universe.

Fig. 18 Palladio’s reconstruction of the Roman Theatre (1567)

Fig. 19 The Memory Theatre Of Giulio Camillo based on L’Idea del Teatro, transcribed by Frances Yates

Fig. 20 The Teatro Olympico, Vicenza by Palladio

The Astral Memory of Giordano Bruno

In his book De umbris idearum, 1582, Giordano Bruno develops a cognitive system consisting of concentric wheels (fig. 24). The images on the inner wheel include images of the decans of the zodiac, images of the planets, the mansions of the moon and the houses of the horoscope. On the outer wheels are placed the contents of the inferior world and all arts and sciences known to man. These images form combinations as the wheels revolve. By arranging and manipulating the star- images one can change the stellar influences on the inferior world. This astral memory therefore gives not only knowledge, but powers.

In his book De imaginum, 1591, Bruno proposes a similar memory system based on architectural objects (fig. 22). He uses a sequence of memory rooms which are based on a geometry which is worked from above by celestial mechanisms. Everything that can be said, known and imagined is to be memorized through the images in his atria, fields and cubicles. Everything in the lower physical world, all plants, stones, metals, animals, every art, science, invention and all human activities are included. This system is encyclopedic like the one in De umbris idearum, in which all the contents of the world were included on the wheels surrounding the central wheel with its celestial images.

In accordance with the ancient Egyptian text, The Corpus Hermeticum, Bruno’s astral forces were instruments of the divine; beyond the operative stars there were yet higher divine forms. The highest of all forms was the One, the divine unity. The aim of the memory system was to achieve this unifying vision, the hermetic monad.

Fig. 21 Images illustrating the principles of the art of memory. Arte della memoria locale, del Riccio (1595)

Fig. 22 Memory System from Figuratio Aristotelici physici auditus, Bruno (1586)

Fig. 23 The Heaven, De imaginum compositione, Bruno (1591)

Fig. 24 Memory System based on Bruno’s De umbris idearum, transcribed by Frances Yates, Bruno (1582)

Robert Fludd

Robert Fludd is the last outpost of the Renaissance Hermetic tradition, in times when the modes of Hermetic and magical thinking were heavily under attack from the rising generation of 17^th century philosophers. His outlook is very much like the one found many years earlier in Camillo’s theatre. In Fludd’s Utriusque cosmi,.., he proposes a memory system based on two worlds, the macro and the microcosm. The macrocosm contains ‘ideas’ such as spirits, souls, angels, demons, effigies of stars (fig. 25). The microcosm contains corporeal images of men, animals and inanimate objects and real buildings. These buildings are called ‘theatres’ and precisely mean stages on which the imaginary comedies and tragedies are acted (fig. 26). Fludd’s work is richly illustrated with engravings to visually illustrate his philosophy. His illustration of the stage is based on the wooden public theatres of the English Renaissance, such as Shakespeare’s Globe theatre which was erected on the Bankside in London in 1599.

Fig. 25 The Zodiac from Ars Memoriae in Utriusque Cosmi,.., Fludd (1619)

Fig. 26 The Theatre from Ars Memoriae in Utriusque Cosmi,.., Fludd (1619)

This historical investigation is now followed by two contemporary examples, in which information structures are related to architectural buildings:

The Information Visualizer

This visualization evolves the Rooms multiple desktop metaphor into a workspace for information access, an information workspace.

The heart of the Information Visualizer architecture is a controlled resource scheduler, the Cognitive Coprocessor architecture, which serves as an animation loop and scheduler for application and interface agents (fig. 36). The basic building block in the Visualizer are Interactive Objects, which form the basis for coupling user interaction with the behavior of the application and offloading work to interface agents. Interactive objects include text, buttons, doors, sliders and thermometers for feedback indicators. Interactive objects are generalized to the point that every visible entity in the simulated scene, the walls, floor and ceiling of the 3D rooms are Interactive Objects. Finally, all application- specific artifacts placed in the rooms are Interactive Objects.

Room buttons for instance are used for movement, new interface building blocks and task assistance by agents. A button has an appearance (typically a bitmap) and a selection action which is executed when the button is pressed.

The Information Visualizer explores 3D visualizations of some of the classical data organizations:

.) Hierarchical, the cone tree (fig. 29)

.) Linear, the perspective wall (fig. 31)

.) Spatial: The spatial structure of a building can be used as a structural browser for people. Selecting an organization produces the names and pictures of its members and selects their offices. Clicking on offices retrieves their inhabitants. (fig. 30)

.) Continuous: the data landscape

.) Unstructured; the information grid

All these visualizations are contained in information rooms, which are linked by doors. Doors are interactive objects which allow users to move from one room to another. They support either manual control or scripted animation of opening.

An overview shows all the 3D workspaces at the same time, which is the information building in its entirety. In order to divide the surface of the screen efficiently, the technique of recursion is applied. Small reproductions of all the information spaces are aligned along a 2- dimensional grid. By clicking on these icons in the overview, users can access the actual workspaces.

The navigation techniques used are the walking metaphor and point of interest logarithmic flight for rapid, precise movement relative to objects of interest.

Associative retrieval based on linguistic searches can be used to highlight portions of an information visualization. Thus traditional associative searches can be combined with structural browsing.

Fig. 27 Information Visualizer Overview, Information Visualization using 3D Interactive Animation, Robertson, Card, Mackinlay (1993)

Fig. 28 Large Tree visualization, ibid.

Fig. 29 Horizontal Cone Tree visualization of a directory hierarchy, ibid.

Fig. 30 Visualization of partial floor plan of Xerox PARC, ibid.

Fig. 31 Perspective Wall visualization of files, ibid.

The New York Stock Exchange

At the New York Stock Exchange (NYSE) an average of $38 billion in trades occurs daily. During the last ten years, trading volume has quadrupled from 42 billion shares in 1989 to 170 billion in 1999. This increase in volume and velocity of trades has produced a vast amount of data, which have to be converted in information which can be readily used and managed. With Silicon Graphics imagery, the NYSE floor operations group has consolidated continuous data streams from the Exchange’s trading systems into the world’s first large- scale virtual environment for business applications.

The NYSE’s data processing branch, the Securities Automation Corp. (SIAC), hired Asymptote Architects of New York in an early stage of the project to design the virtual environment and the associated 4 by 6 foot physical workspace which houses the 3D model and facilitates operations control (fig. 35). Users had to be able to easily understand the spatial relationships and animations which link and convey the data information and navigate the environment quickly and efficiently. This 3 dimensional model is an architecturally accurate representation of the physical trading floor and monitors and interprets all the NYSE’s business and systems activity in real time (fig. 33). The operations staff can measure trading volume and velocity, monitor exceptions to established thresholds and draw correlations among stock and data elements in real time.

Fig. 32 Stock Data Visualization, The New York Stock Exchange (1999)

Fig. 33 3D Virtual Trading Floor, Overview, ibid.

Fig. 34 Information Wall Elevation, ibid.

Fig. 35 The GUI for the 3D system is centrally located between actual trading rooms, ibid.

6. The Architecture of Cognitive Amplification

Fig. 36 Cognitive Coprocessor Interaction architecture, Information Visualization using 3D Interactive Animation, Robertson, Card, Mackinlay (1993)

Resonances

We have seen in the memory buildings of the Renaissance, how a micro and a macrocosm were combined to comprehend the workings of the universe (Fludd, 1619). An array of buildings were constructed, inspired by auditoriums, stages or rooms which were connected to the zodiac and supercelestial realms, which in their terms influenced the inferior realm, the physical world. In Camillos’s theatre, images were aligned on the grades of his auditorium, which were connected to drawers of text underneath them. This idea can be interpreted as an early version of hyperlinks which are opened through icons, connected by the structure of a network such as the World Wide Web. The ‘stage’ desktop as used in interface design reminds us of the same metaphor in Fludd’s microcosm. Or we think of the brokers of the New York Stock Exchange who are, not unlike Bruno and his metaphorically magic memory, manipulating and tracking large amounts of money in real time, immersed in the virtual cosmos which is their trading floor.

Interface Design

In the practice of the Art of Memory, the structure of physical architecture was used to support images related to information to be remembered. This mental support was consciously enriched by notions of size, lighting, color, animation, allegory and particularity of imagined characters. This application shows the knowledge the Greeks developed to facilitate memorization. They understood the supportive role of spatial structure, color, light and animation for memory. These ideas are being applied in our time for the development of the computer- user interface. As Brenda Laurel has shown in her book The Art of User- Computer Interface Design (1998), animated icons, color and space, the use of the stage metaphor all support the understandability of the interface and ease of use of it as a tool for work and learning.

The Architectural Metaphor

As indicated in paragraph 1, this essay has shown the logistics, entities and structures through which large amounts of raw data a transformed into cognitively understandable patterns. The development of these patterns were shown through the viewpoint of historical interpretations of the problem and their relation to architectural structure.

The analogy between information and building is inherent in the definition of the word structure:

Structure

1. a whole constructed unit, esp. a building

2. a set of interconnecting parts of any complex thing; a framework (the structure of a sentence, a new wages structure)

3. v.tr. give structure to, organize, frame

The Oxford Dictionary and Thesaurus ( 1991)

It was shown that in the early stages of philosophy, cognitive patterns were equated to architectural structure as in the form of antique buildings. The array of column grids in antiquity, the line of buildings along a street were used as templates for mental patterns, onto which images were applied to support memorization. Only in the late middle ages, initiated by Raymond Lull, information started to emancipate itself from architecture, to develop its own structures, away from buildings in the shape of abstract wheels, trees and ladders.

From this time onwards, through the age of the enlightenment with the development of scientific methods by Descartes, Hume and Leibniz, the amount of classified data grew exponentially until the present day. The vast amounts of information of today, often changing at the speed of light and having to be experienced in real time, cannot possibly be understood without 3- dimensional, interactive representation. Architecture is in the process of making its way back into information visualization on various levels. First, and most closely to the human user, the metaphorical desktop is on its way of being projected into the third dimension, where it will not continue its existence in the present form. With the unfolding of 3- dimensional space in front of the user a new dimension is opened up in which new metaphors are needed for its control: the world, cosmos, landscape and building. And as in the physical world, so in the virtual world does architecture ground its users.

As shown in the example of the Information Visualizer, metaphorical rooms start to envelop and classify pure information structures such as trees and landscapes, where the user comes to interact with them. In this case architecture appears on two levels:

1. As an image of its own office building, showing its three floors, the practice areas within the office and the employees working in them. By clicking on a respective office on the screen, the employees who work in it can be contacted. In this respect we realize that architectural structure itself is used as a device to divide regions of activity into understandable patterns. In the case of practice areas within an office, spatial distance equals programmatic distance.

2. With its metaphorical rooms, the entire interface becomes architectural. Once again we see an information architecture, which as in the Renaissance is modeled following a physical example. The workspace rooms are aligned along a virtual corridor, interconnected by doors through which users can move. We can detect a virtual adaptation of the physical office itself. This representation becomes interesting at the moment of the ‘overview’, the point from which the user sees the metaphorical building in its entirety, when all the ‘rooms’ are lined up across the screen along a grid as on a façade (fig. 27). What can be seen here is the virtual version of the ‘outside’ of the information building. As we can see the entire volume of a physical building from its exterior, so also here do we see all the rooms at once, aligned in a grid as on a façade. Only, here, the building becomes flat, aligning its rooms along the surface of the screen, giving the user an overview of what becomes a cognitive façade.

As shown in the example of the New York Stock Exchange, physical architecture is used to carry screens of information which then are mirrored altogether in a virtual workspace. This workspace is used to reduce navigation time around the physical space and be able to track anomalies in data on one screen. Therefore the information contained on 2- dimensional screens across an entire physical space can be perceived compressed on one screen. In this case physical architecture and its virtual counterpart have entered a genuinely profitable symbiosis.

7. Conclusion

It was shown that raw data can be structured in various ways, depending on the nature of their contents: clusters as in starfields, trees, landscapes, networks or architectural buildings. The architectural metaphor, when related to its physical counterpart, provides context and base for the pure information structures. In large and complex networks, the above types can be combined: networks of trees, trees of networks. This notion casts a light on the exterior and interior of the metaphorical building. When its rooms contain galaxies of trees, for examples, its traditional role of containment is inversed: it contains a universe rather than being part of it. Or, if the nodes on the tree within the building contain other rooms of itself which contain other nodes of the tree, the building is part of a pure information structure. Information then becomes part of the architecture’s internal logic.

Further Development

This essay has analyzed information visualizations in all its forms. Their relationship to architectural structure was discovered. As was shown in this elaboration, the insight gained can be used for the actual design of consciously symbiotic design between the amplifying qualities of structured information and the virtually grounding qualities of metaphorical architecture.

Bibliography

Ahlberg, C., Williamson, C., Shneiderman, B., Dynamic Queries for Information Exploration: An Implementation and Evaluation. Proceedings of CHI’92, ACM Conference on Human Factors in Computing Systems, 619- 626, New York, 1992

Aristotle, De anima, translation by D. Hamlyn, Clarendon Press , Oxford , 1968

Aristotle, De memoria et remeniscentia, translation by T. Ross.,The University Press , Cambridge, 1906.

Bederson, B., Hollan, J., Pad++: A Zooming Graphical Interface for Exploring Alternate Interface Physics, Proceedings of UIST’94, ACM Symposium on User Interface Software and Technology, 17- 26, Marina del Rey, CA, 1994

Bruno, G., De umbris idearum…& non vulgares per memoriam operationes explicatis, apud Aegidium Gorbinum,

Paris, 1582

Bruno, G., De imaginum, signorum et idearum compositione, ad omnia inventionum, dispositionum et memoriae genera Frankfurt, 1591

Bruno, G., Figuratio Aristotelici physici auditus, Paris, 1586

Bruno, G., De imaginum compositione, Frankfort, 1586

Burch., H., Cheswick., B., Skitter, Cooperative Association for Internet Data Analysis (CAIDA), Lucent Bell Laboratories, 1999

Camillo, G., Tutte le Opere, Venice, 1552

Card, S., Mackinlay J., Shneiderman, B., Readings in Information Visualization; Using Vision to Think, Morgan Kaufmann Publishers, Inc., San Francisco, 1999

Carpendale, M., Cowperthwaite, D., Fracchia, F., Extending Distortion Viewing from 2D to 3D, IEEE Computer Graphics and Applications, July/ Aug., 42- 51, 1997

Cicero, Ad C. Herrennium de ratione dicendi, rhetorica ad Herennium / English translation by Harry Caplan, W. Heinemann, London 1954.

Cicero, M., De oratore / English translation by E.W. Sutton, London : W. Heinemann , London, 1942.

Eick, S., Steffen J., Sumner, E., Seesoft- A Tool for Visualizing Line Oriented Software Statistics, IEEE, Transactions on Software Engineering, 18 (11.Nov), 957- 968, 1992

Feiner, G., Steven, K., Beshers, G., Clifford K., Worlds within Worlds, Metaphors for Exploring n- Dimensional Virtual Worlds. Proceedings of UIST’90, ACM Symposium on User Interface Software and Technology, 76- 83, 1990

Fludd, R., Utriusque cosmi, maioris scilicet et minoris, metaphysica, physica atque technica historia, Tomus Secundus, Oppenheim, 1619

Lamping, J., Rao, R., The Hyperbolic Browser: A Focus and Context Technique for Visualizing Large Hierarchies. Journal of Visual Languages and Computing, 7(1), 33- 35, 1996

Laurel, B., The Art of User- Computer Interface Design, 1998The Art of human-computer interface design / edited by Brenda Laurel. Reading, Mass ; Wokingham : Addison-Wesley Pub. Co , c1990. xvi,523p, [16]p of plates : ill ; ports. (some col.) ; 26cm.

Linnaeus, B., Systema Naturae , Oslo, 1735

Moltenbrey, Karen, Taking Stock, The New York Stock Exchange Adds a New Dimension to Wall Street with a 3D Trading Floor, Computer Graphics World, October 1999

Robertson, G., Card, S., Mackinlay J., Information Visualization using 3D Interactive Animation, Communications of the ACM, 36 (4), 57- 71, 1993

Riccio del, A., Arte della memoria locale, Biblioteca Nazionale, Florence, 1595

Risch, J., Rex, D., Dowson, S. et al., The STARLIGHT Information Visualization System. Proceedings of IEEE International Conference on Information Visualization, 42- 49, London, 1997

Scaife, M., and Rogers, Y., (1996), External Cognition: How Do Graphical Representations Work?, Internal Journal of Human- Computer studies, 45 (2), 185- 213

The Oxford Dictionary and Thesaurus, Oxford University Press, Oxford, 1991

Tufte, E., The Display of Quantitive Information, Graphics Press, Cheshire, CT, 1983

Tufte, E., Envisioning Information, Graphics Press, Cheshire, CT, 1990

Wise, J. A. Thomas, Pennock J.J., Lantrip K., Visualizing the Non- Visual: Spacial Analysis and Interaction with Information from Text Documents, Proceedings of InfoVis’95, IEEE Symposium on Information Visualization, New York, 51- 58, color plate 140

Yates, F., The Art of Memory, Pimlico, London, 1966