Date: Tue, 24 Nov 92 22:25:59 -0500 From: ben@cs.UMD.EDU (Ben Shneiderman) To: bam@cs.cmu.edu, ben@cs.umd.edu, callahan@cerc.wvu.wvnet.edu, hopkins@bongo.garnet.cs.cmu.edu, weiser.pa@xerox.com Subject: Re: more pie menus! I couldn't resist sending you all this latest essay which is destined for IEEE Software...some readers expect it to generate some strong responses...Ben Beyond Intelligent Machines: Designing Predictable and Controllable User Interfaces Ben Shneiderman November 24, 1992 University of Maryland, College Park, MD 20742 Professor, Department of Computer Science, Head, Human-Computer Interaction Laboratory at the Center for Automation Research & Member, Institute for Systems Research Who's in control? An important shift is occurring from the old vision of computers as 'intelligent' to a new vision based on predictable and controllable user interfaces that depend on direct manipulation of objects and actions. Appropriate metaphors and terminology are important since they shape the thoughts of researchers, designers, managers, congress-people, journalists, etc. Most of us have learned the importance of gender neutral terminology and similarly I have been strongly opposed to suggesting that computers are 'intelligent' or 'smart' for several reasons: 1) Limits to Imagination I think we should have much greater ambition than to make a computer behave like an intelligent butler or other human agent. Computer supported cooperative work (CSCW), hypertext/hypermedia, multi-media, information visualization, and virtual realities are powerful technologies that enable human users to accomplish tasks that no human has ever done. If we describe computers in human terms then we run the risk of limiting our ambition and creativity in the design of future computer capabilities. 2) Predictability and Control are Desirable If machines are 'intelligent' or 'adaptive' then they may become less predictable and controllable. Our usability studies show that users want feelings of mastery, competence, and understanding that come from a predictable and controllable interface. Most users seek a sense of accomplishment at the end of the day, not the sense that this 'intelligent' machine magically did their job for them. 3) Human Responsibility I am concerned that if designers are successful in convincing the users that computers are intelligent, then the users will have a reduced sense of responsibility for failures. The tendency to blame the machine is already widespread and I think we will be on dangerous grounds if we encourage this trend. 4) Machines are not People AND People are not Machines I have a basic philosophical objection to the suggestion that machines are, or can ever be, intelligent. I know that many of my colleagues are quite happy to call machines intelligent and knowledgeable, but I prefer to treat and think about machines in very different ways from the way I treat and think about people. The lessons of history While some productive work has been done under the banner of `intelligent', often those who use this term reveal how little they know about what users want or need. The users's goal is not to interact with an 'intelligent' machine, but to create, communicate, explore, plan, draw, compose, design, or learn. Ample evidence exists of the misguided directions brought by 'intelligent' machines: - natural language interaction seems clumsy and slow compared to direct manipulation and information visualization methods that use rapid, high-resolution, color displays with pointing devices. Lotus HAL is gone, AI INTELLECT hangs on but is not catching on. There are some interesting directions for tools that support human work through natural language processing: aiding human translators, parsing texts, and generating reports from structured databases. - speech I/O in talking cars and vending machines is gone. Voice recognition is fine for handicapped users plus special situations, but doesn't seem to be viable in general office, home, or school settings. Our recent studies suggest that speech I/O has a greater interference with short term and working memory than hand-eye coordination for mouse menu selection. Voice store and forward, phone-based information retrieval, and voice annotation have great potential but these are not the 'intelligent' applications. - adaptive interfaces are unstable and unpredictable, often leading users to worry about what will change next. I see only modest chances for success in user modeling to recognize the level of expertise and revise the interface accordingly - can anyone point to successful studies or commercial products? By contrast, user controlled adaptation through control panels, cruise control for cars, and remote controls for TV are success stories. While algorithms to deal with dynamic issues in network or disk space management are needed, the task domain and user interface issues of the application program should generally be under direct user control. - Intelligent CAI (Computer Assisted Instruction) only prolonged the time (compared to traditional CAI) until the users felt they were the victims of the machine. Newer variations such as Intelligent Tutoring Systems are giving way to Interactive Learning Environments where students are in control and actively creating or exploring. - intelligent talking robots with five-fingered hands and human facial features (quaint fantasy that did well in Hollywood but not in Detroit or elsewhere) are mostly gone in favor of flexible manufacturing systems that enable supervisors to specify behavior with predictable results. It seems that some designers continue to ignore this historical pattern and still dream of creating 'intelligent' or 'smart' machines. It is an ancient and primitive fantasy, and its seems most new technologies must pass through this child-like animistic phase. Lewis Mumford identified this pattern (Technics and Civilization, 1934) when he wrote about the Obstacle of Animism: 'the most ineffective kind of machine is the realistic mechanical imitation of a man or another animal...for thousands of years animism has stood in the way of...development.' An alternate vision My point in this essay is not merely to counter a popular design philosophy, but to offer a new vision that is more in harmony with what users want. I believe that the future will be filled with powerful, but predictable and controllable computers that genuinely serve human needs (Designing the User Interface: Strategies for Effective Human-Computer Interaction, Second Edition, Addison-Wesley Publ. Co., Reading, MA, 1992). In this vision of predictable and controllable (PC) computing, the promising strategies are rapid, visual, animated, colorful, high resolution interfaces built on meaningful control panels, appropriate preference boxes, user-selectable toolbars, rapid menu selection, easy to create macros, and comprehensible shortcuts. These enable me to specify rapidly, accurately, and confidently how I want my email filtered, what documents I want retrieved and in what order, and how my documents will be formatted. Our Human-Computer Interaction Laboratory has applied these principles to information visualization methods that give users X-ray vision to see through their mountains of data. Treemaps enable users to see (and hear) 2-3000 nodes of hierarchically structured information by utilizing every pixel on the display. Each node is represented by a rectangle whose location preserves the logical tree structure and whose area is proportional to one of its attributes. Color represents a second attribute and sound a third (B. Johnson & D. Turo, Improving the Visualization of Hierarchies with Treemaps: Design Issues and Experimentation, Proc. IEEE Visualization '92). Treemaps have been applied to Macintosh directory browsing (Figure 1), in which area could be set to file size, color to application type, and sound to file age (our TreeViz application is available from the University of Maryland's Office of Technology Liaison, (301) 405-4210). When users first try TreeViz they usually discover duplicate or misplaced files, redundant and chaotic directories, and many useless files or applications. Other applications include: stock market portfolio management, sales data, voting patterns, sports (48 statistics on 459 NBA players, in 27 teams, in four leagues), etc. Dynamic queries allow rapid adjustment of query parameters and immediate display of updated result sets. These animations enable users to develop intuitions, discover patterns, spot trends, find exceptions, and see anomalies. The Dynamic HomeFinder prototype (Figure 2) allows users to adjust the cost, number of bedrooms, location, etc. and see points of light come and go on a map to indicate a matching home. Users execute up to 100 queries/second (rather than one query per 100 seconds) producing a revealing animated view of where high or low priced homes are found, and there are no syntax errors. Clicking on a point of light brings up a description or image (videotape available, or for an empirical comparison with a natural language system, see Williamson, C. and Shneiderman, B., The Dynamic HomeFinder: Evaluating dynamic queries in a real-estate information exploration system, 1992 ACM SIGIR Proceedings). Dynamic queries are very effective when a visual environment such as a map, calendar, or schematic diagram are available, but they can be easily applied with standard text file output (Figure 3). Dynamic queries exemplify the future of interaction; You don't need to describe your goals, negotiate with an intelligent agent, and wait for a response, you Just Do It! Furthermore, dynamically seeing the results enables you to explore and rapidly reformulate your goals in an engaging videogame-like manner. Open problems in information visualization include screen organization, widget design, algorithms for rapid search and display, use of color and sound, and strategies to accommodate human perceptual skills. We also see promise in expanding macro makers into the graphical environment with visual triggers based on controlled replay of desired actions - the general idea is Programming in the User Interface (PITUI) to Do-What-I-Did (DWID). I want to encourage the exploration of new metaphors and visions of how computers can empower people by presenting information, allowing rapid selection, supporting personally specified automation, and providing relevant feedback. Metaphors related to controlling tools or machines such as driving, steering, flying, directing, conducting, piloting, or operating seem more generative of effective and acceptable interfaces, than 'intelligent' machines. A scientific approach to user interface research Whether you agree with the design philosophy in this essay, and especially if you disagree, I hope that you will add to our scientific knowledge by conducting well-designed empirical studies of learning time, measuring performance time for appropriate tasks, recording error rates, evaluating human retention of interface features, and assessing subjective satisfaction. There's much work to be done to make computing accessible, effective, and enjoyable. Acknowledgements: This essay was prompted by the discussion between Mark Weiser and Bill Hefley, stimulated by lively email and personal discussions with Paul Resnick, Tom Malone, and Christopher Fry at MIT, and refined by comments from Catherine Plaisant, Rick Chimera, Brian Johnson, David Turo, Richard Huddleston, and Richard Potter at the Human-Computer Interaction Lab at Univ. of Maryland. I appreciate Bill Curtis's support for this vision. Thanks to all.