Intelligent Communication with Computers

AITopics > Interfaces


"The use of techniques from the field of autonomous agents provides a new complementary style of human-computer interaction, where the computer becomes an intelligent, active and personalized collaborator. Interface agents are computer programs that employ Artificial Intelligence methods to provide active assistance to a user of a particular computer application. The metaphor used is that of a personal assistant who is collaborating with the user in the same work environment. The assistant becomes gradually more effective as it learns the user's interests, habits and preferences."

computer monitor with assistant's hand

- FLUIDS (Intelligent User Interfaces - New Achievements in Research)

Even the most sophisticated and powerful system will be next to useless without an effective user interface. If struggling with the operation of a video player is not your idea of fun, you can well understand why a part of the artificial intelligence community is devoted to scientifically developing interfaces that are easy and pleasant to use, and that form an effective, and possibly seamless, link between the intelligent machine(s) and the human user(s).

Good Starting Places

AI Magazine cover:  Intelligent User Interfaces

Introduction to the Special Issue on Intelligent User Interfaces [Winter 2001]. By James Lester. AI Magazine 22(4): 13-14. "Recent years have witnessed significant progress in intelligent user interfaces. Emerging from the intersection of AI and human-computer interaction, research on intelligent user interfaces is experiencing a renaissance, both in the overall level of activity and in raw research achievements. Research on intelligent user interfaces exploits developments in a broad range of foundational AI work, ranging from knowledge representation and computational linguistics to planning and vision. Because intelligent user interfaces are designed to facilitate problem-solving activities where reasoning is shared between users and the machine, they are currently transitioning from the laboratory to applications in the workplace, home, and classroom."

Seven Aspects of Mixed-Initiative Reasoning: An Introduction to this Special Issue on Mixed-Initiative Assistants. By Gheorghe Tecuci, Mihai Boicu, and Michael T. Cox. AI Magazine 28(2): Summer 2007. "Mixed-initiative assistants are agents that interact seamlessly with humans to extend their problem-solving capabilities or provide new capabilities. Developing such agents requires the synergistic integration of many areas of AI, including knowledge representation, problem solving and planning, knowledge acquisition and learning, multiagent systems, discourse theory, and human-computer interaction. This paper introduces seven aspects of mixed-initiative reasoning (task, control, awareness, communication, personalization, architecture, and evaluation) and discusses them in the context of several state-of-the-art mixed-initiative assistants. The goal is to provide a framework for understanding and comparing existing mixed-initiative assistants and for developing general design principles and methods." Individual PDF files available:

HCI Bibliography. Provided by the HCI Bibliography Project. Over 58,000 records (as of spring 2010) including a definition of HCI, the history of HCI, recommended readings, conferences, and much more.

  • Also available from HCIBIB is the HCI Sites page: "a collection of resources on Human-Computer Interaction. It replaced Keith Instone's HCI Virtual Library (originally the HCI Launch Pad)."

With Kinect, Microsoft Aims for a Game Changer. NY Times, October 24, 2010. "Microsoft has one-upped Sony and Nintendo by eliminating game controllers and their often nightmarish bounty of buttons. Kinect peers out into a room, locks onto people and follows their motions. Players activate it with a wave of a hand, navigate menus with an arm swoosh and then run, jump, swing, duck, lunge, lean and dance to direct their on-screen avatars in each game. "

General Readings

Microsoft Predicts The Future With Vista's SuperFetch - SuperFetch, a feature within Vista, predicts which applications are used when, then pre-loads them so that they're instantly available. By Gregg Keizer. InformationWeek (January 19, 2007). "Microsoft Research contributed to the SuperFetch effort, a feature within Vista that predicts which applications are used when, then pre-loads them so that they're instantly available. 'As part of a long term set of projects, we want to teach the computer to learn from users to make the machine more proactive,' says Eric Horvitz, a principal researcher with Microsoft's R&D as well as the president-elect of the American Association for Artificial Intelligence. 'We want to use the system's idle time to make things punchier.' Horvitz and his colleagues developed the core algorithms that make up the predictive part of SuperFetch, the technology that plays Nostradamus for the operating system. ... Long-range, says Horvitz, he'd like to extend SuperFetch-like predicting to actions within individual applications"

  • Also watch this video: Eric Horvitz - Better communication productivity. A Microsoft Channel 9 Forum Video (July 7, 2006). "In this video, Robert and Charles drop in on Eric Horvitz at Microsoft Research. It’s always great to get an update on things he’s been up to. This time, we focus on technologies aimed at making it easier for people to communicate with each other, but we have a freewheeling discussion. We hear about research on the Bestcom project. Bestcom is short for 'best-means communication'. With Bestcom, smart computer agents try to figure out the context of people and decide how best to hook them up… based their own preferences and situations. Eric shows us a prototype named Bestcom-ET which has been used by many people inside Microsoft. This prototype led to the Communicator product. Eric also talks about Bayesphone, a really smart smartphone --and he also shows us some of the output of the Coordinate prototype. Coordinate does 'presence and availability forecasting'. Coordinate uses machine learning to make forecasts about things like when someone will next be in their office, when they will next read email, and next be free for some time to chat. Sounds eerie…but not to worry -- people have nice privacy controls to decide which people (or agents) they want to share these kinds of forecasts with. Fun stuff!"

Meet the Life Hackers. By Clive Thompson. The New York Times Magazine (October 16, 2005). " ... When you work next to other people, they can sense whether you're deeply immersed, panicking or relatively free and ready to talk - and they interrupt you accordingly. So why don't computers work this way? Instead of pinging us with e-mail and instant messages the second they arrive, our machines could store them up - to be delivered only at an optimum moment, when our brains are mostly relaxed. One afternoon I drove across the Microsoft campus to visit a man who is trying to achieve precisely that: a computer that can read your mind. His name is Eric Horvitz, and he is one of [Mary] Czerwinski's closest colleagues in the lab. For the last eight years, he has been building networks equipped with artificial intelligence (A.I.) that carefully observes a computer user's behavior and then tries to predict that sweet spot - the moment when the user will be mentally free and ready to be interrupted. Horvitz booted the system up to show me how it works. ... "

  • Also see/hear 'Interruption Science' - Costly Distractions at Work (radio broadcast). NPR's Day to Day (October 14, 2005) "Technology forces us to juggle competing demands on our attention over the course of our workdays. Alex Chadwick speaks with New York Times Magazine contributor Clive Thompson about 'interruption science,' the study of the effect of disruptions on job performance." Excerpt: "Clive Thompson: But another, perhaps more ambitious attempt is to work on artificial intelligence that observes you and knows when you're busy so that when an e-mail comes, it doesn't just automatically set off a ding saying 'Hey, there's e-mail.' It actually waits five or 10 minutes until you're finished with whatever you're doing and then gives you the message at the moment when you are, you know, unencumbered and ready to receive it.' [audio available]

A Head For Detail - Gordon Bell feeds every piece of his life into a surrogate brain, and soon the rest of us will be able to do the same. But does perfect memory make you smarter, or just drive you nuts? By Clive Thompson. (November 2006; Issue 110: Page 72). "Gordon Bell will never forget what I look like. He'll never forget what I sound like, either. Actually, he'll never forget a single detail about me. That's because when I first met the affable 72-year-old computer scientist at the offices of Microsoft Research Labs, in Redmond, Washington, he was carefully recording my every move. He had a tiny bug-eyed camera around his neck, and a small audio recorder at his elbow. As we chatted about various topics--Australian jazz musicians, his futuristic cell phone, the Seattle area's gorgeous weather--Bell's gear quietly logged my every gesture and all my blathering small talk, snapping a picture every 60 seconds. Back at his office, his computer had carefully archived every document related to me: all the email I'd sent him, copies of my articles he'd read, pages he'd surfed on my blog. ... For the past seven years, Bell has been conducting an audacious experiment in 'lifelogging'--creating a near-total digital record of his experience. His custom-designed software, 'MyLifeBits,' saves everything it can get its hands on. ... You could trace the notion of perfect recall back to 1945, when presidential science adviser Vannevar Bush published a provocative essay in The Atlantic Monthly entitled 'As We May Think.' ... Consider for a second how, precisely, we think. We use our memories all the time, of course, often by "active" remembering--scrolling through our minds to locate a tidbit. But much mental labor is passive. We think about something in the background, subconsciously letting a problem brew. Then we suddenly hit upon an interesting combination of things, a new way of thinking about a problem: the elusive, all-important epiphany. What if our computers had their own intelligence, and could do that background work for us? ... One of Bell's Microsoft allies is also investigating whether artificial intelligence could be used to find hidden patterns in memory. One day last summer, I visited Eric Horvitz, an expert in machine intelligence at the Research Labs, to see his 'Lifebrowser.' The Lifebrowser's goal is to automatically identify the most significant events in your life, so that when you scroll back through your history, it shows you only the most important highlights."

AI's Next Brain Wave. New research in artificial intelligence could lay the groundwork for computer systems that learn from their users and the world around them. Part four in The Future Of Software series. By Aaron Ricadela. InformationWeek (April 25, 2005). "Computer scientists at the Palo Alto Research Center also are trying to bring user interfaces to life by replacing raw information with material that selects itself based on what the computer thinks the user wants to know. PARC's user-interface group published a paper in January describing new software called ScentHighlights that helps users skim information by extracting key sentences from an electronic book, relevant to keywords a user types in or clicks on in the text. The system, based on a PARC theory called information scent, is part of an emerging class of user interfaces that react to what gets a user's attention, says Stuart Card, manager for user-interface research and a senior research fellow at PARC."

  • Cars that Think. PBS television broadcast of Scientific American Frontiers (January 26, 2005). "The fully automatic car may be down the road a ways, but cars that do your thinking for you are just around the corner -- they watch out for hazards, they listen to you, they read your lips, they even know when you're distracted."
  • Intelligent software aims to give users peace of mind. Microsoft Notebook feature by Todd Bishop. Seattle Post-Intelligencer (March 7, 2005).

CMU's Brad Myers. Technology Research News Editor Eric Smalley carried out an email conversation with Carnegie Mellon University professor Brad Myers. (August 22, 2005). "Myers: Another area that I think is going to take off is intelligent interfaces, where the system actively tries to be helpful and learns from the user."

The Ghost in Your Machine - Computers may soon monitor your work, notice when fatigue sets in, and fix mistakes. BusinessWeek Online Reporter Olga Kharif interviews Chris Forsythe (August 25, 2003). "At their most benign, smart computers seem like executive secretaries for those of us who can't afford one -- offering tremendous advances in productivity. Yet some fear that the concept suggests an ominous encroachment out of a sci-fi movie. Cognitive psychologist Chris Forsythe, who leads the Sandia team, insists that the machines are designed to augment -- not replace -- human activity. 'We don't want to take the human out of the loop,' he says. ... Q: How would you characterize the current state of human-machine interaction? A: The biggest problem is that if you're the user, for the most part the technology doesn't know anything about you. The onus is on the user to learn and understand how the technology works. What we would like to do is reverse that equation so that it becomes the responsibility of the computer to learn about the user. The computer would have to learn what the user knows, what the user doesn't know, how the user performs everyday, common functions. It would also recognize when the user makes a mistake or doesn't understand something."

Agents that Reduce Work and Information Overload. By Pattie Maes (1994). A crisp summary of the CACM July 1994 (Vol. 37, No. 7, 31-40) article. An introduction to "interface agents [that] are computer programs that employ Artificial Intelligence techniques to provide active assistance to a user with computer-based tasks."

Designing for Human-Agent Interaction. By Michael Lewis. AI Magazine 19(2): Summer 1998, 1998, 67-78. "Interacting with a computer requires adopting some metaphor to guide our actions and expectations. Most human-computer interfaces can be classified according to two dominant metaphors: (1) agent and (2) environment. Interactions based on an agent metaphor treat the computer as an intermediary that responds to user requests. In the environment metaphor, a model of the task domain is presented for the user to interact with directly. The term agent has come to refer to the automation of aspects of human-computer interaction (HCI), such as anticipating commands or autonomously performing actions. Norman's 1984 model of HCI is introduced as reference to organize and evaluate research in human-agent interaction (HAI). A wide variety of heterogeneous research involving HAI is shown to reflect automation of one of the stages of action or evaluation within Norman's model. Improvements in HAI are expected to result from a more heterogeneous use of methods that target multiple stages simultaneously."

Squirrel helps with mobile calls. By Luke Alexander. BBC News (August 26, 2005). "There are few things more intrusive than a mobile phone ringtone. ... MIT research student Stefan Marti may have the answer: ditch your mobile phone, and get a squirrel. Specifically, an animatronic desktop squirrel which deals with your calls for you. The squirrel answers phone calls, works out if you are busy or asleep, evaluates how important the incoming call is and takes messages. When it wants to alert its owner to a call, it waves and moves about rather than making a sound. And, it is ridiculously cute. ... The key principle behind the Autonomous Interactive Intermediary (AII), or 'cellular squirrel', is that machines should display what psychologists call social or emotional intelligence. In other words, a computer should be able to communicate information in a way which is responsive to the social situations around it."

An Integrated Environment for Knowledge Acquisition. By Jim Blythe, Jihie Kim, Surya Ramachandran, Yolanda Gil. Information Sciences Institute, University of Southern California. Best Paper, International Conference on Intelligent User Interfaces, 2001. Abstract: "This paper describes an integrated acquisition interface that includes several techniques previously developed to support users in various ways as they add new knowledge to an intelligent system. As a result of this integration, the individual techniques can take better advantage of the context in which they are invoked and provide stronger guidance to users. We describe the current implementation using examples from a travel planning domain, and demonstrate how users can add complex knowledge to the system."

Smart Graphics: Papers from the 2000 Spring Symposium, ed. Andreas Butz, Antonio Krüger, and Patrick Olivier. Technical Report SS-00-04. American Association for Artificial Intelligence, Menlo Park, California. "Advances in the field of computer graphics have made visual media a major ingredient of the modern interface and it is certain that graphics will play an increasingly important role in the way people communicate and interact with computers in the future. Smart Graphics is the interdisciplinary approach to the design, generation, presentation and interaction with 2D and 3D graphical interfaces in a manner that is sensitive to technological, computational and cognitive constraints. As an enterprise it relies on the synthesis of insights from graphic design, cognitive science, human-computer interaction, graphics and artificial intelligence, and the symposium aims to broker a multidisciplinary dialogue between these communities."

Generating Text from Compressed Input: An Intelligent Interface for People with Severe Motor Impairments. By Patrick W. Demasco and Kathleen F. McCoy. Appears in Communications of the ACM, May 1992, Vol. 35, No.5 (1992) and made available by one of the authors.

The Future of Computing. By Michael L. Dertouzos. Scientific American (August 1999). "To date, computer vendors have abused the phrase ease of use.' When they call a system user-friendly, it is tantamount to dressing a chimp in scrubs and earnestly parading it around as a surgeon. When I say "ease of use," I do not mean incorporating more colors and floating animals into our systems. I mean true ease of use, even if the interaction is only via text." <full text available for free only to subscribers>

high-fiving with computer

The Love Machine - Building computers that care. By David Diamond. Wired Magazine (December 2003). "'I want computers to have emotions only to help them survive in the world, not as a way of responding to me,' says Don Norman, a professor of computer science at Northwestern and a computer interface expert. 'I'd rather have a machine that knows its place. Otherwise, you feel like it's a used-car salesman.'"

. CHI 2001 Conference on Human Factors in Computing Systems (Seattle; April 2, 2001). "What I wanted to do today is share some of the projects that Microsoft is involved in that we hope will create new frontiers for human/computer interfaces. I also thought I could take some of my favorite Microsoft error messages and get your feedback, this is one I just got recently. I think it's fair to say that Microsoft has done good things in computer/human interface, and also done its fair share of bad things as well. And so we look forward to participating in the conference, and seeing what new things we can do. ... Today, the user is very much involved in moving the information around. Even as you go from Web site to Web site, the fact that you have to have different user names, different passwords, the fact that you have to resupply the same information again and again, the fact that that information, once you supply it, you have no idea how it's retained and how it's used. That represents a very significant problem."

Intelligent Human-Computer Interfaces. Lynette Hirschman, Guest Editor. The Edge (The MITRE Advanced Technology Newsletter, December 1999) Volume 3 Number 4. "This issue is devoted to intelligent human-computer interfaces -- interfaces that raise the computer to the human level, rather than requiring that the human adapt to the computer. An interface may be intelligent because it can communicate using human language, or because it performs intelligent functions, or because it adapts to a specific task and user. In all these cases, an intelligent interface makes interaction with the computer easier, more intuitive, and more flexible. The area of intelligent interfaces is just one facet of the broad area of human-computer interaction. Other important areas include graphical user interface design, usability, human factors, visualization, immersive environments, and intelligent tutoring systems."

  • The five articles in this issue are: Spoken Language Interfaces Gaining Acceptance as Technology Matures; Command Post of the Future Interactive Visualization to Increase Speed and Quality of Decision Making; DARPA Communicator Program Tackles Conversational Interface Challenges; IMPs Enhance Virtual Collaboration Environments; X-Technologies Ease Customization of Web Interfaces.'''
  • Visit FAME and CHIL project information.

When E-Mail Points the Way Down the Rabbit Hole. Essay by Kirk Johnson. The New York Times (September 2, 2004; no fee reg. req'd.). "The very basis of the spam wars is a search for better analysis of the way human beings think. ... Some theorists, like Professor [Sherry] Turkle at M.I.T., say the first real flash points of spam and human identity might come when our ever more sophisticated anti-spam programs start to understand us a little too well. ... 'As spam becomes more and more sophisticated, most people think your filter will be developed by a smart agent observing you carefully, so the question becomes, what kinds of information do people want their software agent to know?' Professor Turkle said."

Reflections on Challenges to the Goal of Invisible Computing. By Arun Kumar, Department of Philosophy of Technololgy,Institute for Philosophy, Dresden University of Technology, Germany. ACM Ubiquity (May 17 - 24, 2005; Volume 6, Issue 17). "We know that computers are complex beasts in their own right, but for all of their internal complexity computers are just as complicated in their embedding in the outside world, even though the complexity of this embedding is largely invisible to the people who design computers, and to people who make a living promoting their use. And it is possible that computers [fn] might have the power to change us even when we engage with them unconsciously, as when we relate to a tool through the performance of a skill like driving or typing."

Robots get friendly - Robots are acting more like people. Will our attachments eventually become too strong? By Gregory M. Lamb. The Christian Science Monitor (February 5, 2003). "Studies have shown that expectations are higher for such virtual people than, say, a faceless search engine like Google."

See Me, Hear Me . . . New computer interfaces may respond to gestures and speech. By Sharon Machlis. Computerworld (August 26, 2002). "Researchers working on new generations of interfaces are taking a pragmatic approach to how we'll interact with machines, adding support for text, speech, gaze, gesture and more, depending on circumstances and what seems to make sense. Ideally, tomorrow's computers will be better at anticipating what users want, without needing typed commands. This will involve 'context-aware' interfaces, according to Ted Selker, head of the context-aware computing research group at MIT's Media Lab. That might mean a Web application will be able to sense both your mouse and eye movements to determine whether you've visited a site before and what items most interest you -- and then dynamically generate a page based on those interests. By 2006, Selker says, 'the computer will know more about why you're doing what you're doing and what it can do to help you.' Speech-recognition software is already making inroads in telephone-based customer service applications."

A Case-Based Approach to Adaptive Information Filtering for the WWW. By Mauro Marinilli, Alessandro Micarelli and Filippo Sciarrone; Dipartimento di Informatica e Automazione Universita di Roma. "The system is based on a user modeling component, designed for building and maintaining long term models of individual Internet users. Presently the system acts as an intelligent interface for the Web search engines."

Emotionware. By Lynellen D.S. Perry (1996). ACM Crossroads Student Magazine. "The capability of displaying emotion seems to be a critical component of creating intelligent agents with whom humans can comfortably relate and communicate. The emotional aspect distinguishes a dead machine from an agent who is believable, alive, and trustworthy."

Persistent Assistants: Living and Working with AI: Papers from the 2005 Spring Symposium, ed. Daniel Shapiro, Pauline Berry, John Gersh,and Nathan Schurr. Technical Report SS-05-05. American Association for Artificial Intelligence, Menlo Park, California.

Conversations control computers. By Eric Smalley. Technology Research News (January 12/19, 2005). "Because information from spoken conversations is fleeting, people tend to record schedules and assignments as they discuss them. Entering notes into a computer, however, can be tedious -- especially when the act interrupts a conversation. Researchers from the Georgia Institute of Technology are aiming to decrease day-to-day data entry and to augment users' memories with a method that allows handheld computers to harvest keywords from conversations and make use of relevant information without interrupting the personal interactions. ... The researchers' system protects privacy by only using speech from the user's side of the conversation, said [Kent] Lyons."

Citrine: Providing Intelligent Copy-and-Paste. By Jeffrey Stylos, Brad A. Myers and Andrew Faulring. ACM Symposium on User Interface Software and Technology, UIST'04; pages 185-188. Abstract: "We present Citrine, a system that extends the widespread copy-and-paste interaction technique with intelligent transformations, making it useful in more situations. Citrine uses text parsing to find the structure in copied text and allows users to paste the structured information, which might have many pieces, in a single paste operation. ..."

  • Also from the Citrine project site: "the Citrine Phone Dialer, allows the dialing of copied phone numbers. Its novelty --- and usefulness --- comes from automatically learning and applying dialing rules."

What is an Intelligent Interface? Notes from an introduction seminar, March 1997. By Annika Waern, groupleader of, and researcher in, the Human - Computer Interaction and Language Engineering (HUMLE) Group at the Sweedish Institute of Computer Science. "The main application areas for intelligent interfaces are thus such where the knowledge about how to solve a task partially resides with the computer system. Since the user does not know exactly what should be done, he or she cannot manipulate the computer as a tool, but must ask the system to do something for him or her. This request may be incomplete, vague or even incorrect given the user's real needs. Some typical application areas that can be characterised this way are Intelligent tutoring, intelligent help and information filtering."

Reflective Interfaces: a video of Dan Weld's talk at CSE Colloquia - 2005, The University of Washington Computer Science & Engineering Colloquium Series, available from the ResearchChannel ("a non-profit organization founded in 1996 by a consortium of leading research universities, institutions and corporate research centers dedicated to creating a widely accessible voice for research through video and Internet channels").

The Role of Intelligent Systems in the National Information Infrastructure. An American Association for Artificial Intelligence Policy Report. Edited by Daniel S. Weld, University of Washington. Excerpt from (2.1) Intelligent Interfaces: "The gap between current tools and the NII's human-computer communication demands leads to a crucial challenge: providing intelligent interfaces to resources so that people can use the NII without difficulty. ... To be called 'intelligent,' it must satisfy several interrelated criteria: Integrated ... Expressive ... Goal oriented ... Cooperative ... Customized.

Bringing Design to Software. Edited by Terry Winograd. (Addison-Wesley. 1996) "The book contains essays contributed by prominent software and design professionals, interviews with experts, and profiles of successful projects and products. These elements are woven together to illuminate what design is, to identify the common core of practices in every design field, and to show how software builders can apply these common practices to produce software that is more effective, more appropriate, and more satisfying for users." Many sections of the book are available online, including Winograd's Introduction.

Curricula for Human-Computer Interaction. From the ACM Special Interest Group on Computer-Human Interaction, Curriculum Development Group. Don't miss Chapter 2 which covers topics such as: Definition of HCI, Historical Roots, and The Nature of Human-Computer Interaction.

List of IUI Conference Topics

Intelligent User Interfaces (IUI) is the premier conference for reporting on the study of user interfaces with intelligent devices.

IUI 2011 will be held in Palo Alto, California, USA, February 13-16 2011.

  • Intelligent interaction with devices

• Intelligent interactions with handheld devices such as mobile phones, music and video players, GPS, cameras, electronic books, etc. • Sensor- and actuator systems for user interfaces • Location- and context aware information systems • Tangible interaction with smart artifacts • Ubiquitous displays environments • Smart environments Novel, intelligent interaction systems • Modeling and prediction of user behavior • Affective, social and aesthetic interfaces • Natural user interfaces including tangible computing and body computing • User-adaptivity in interactive systems • Personalization and recommender systems • Planning and plan recognition

  • IUI design

• Knowledge-based approaches to user interface design and generation • Proactive and agent-based paradigms for user interaction • Example-based and demonstration-based interfaces • Smart use of sensing technologies for IUI Design User studies • User studies concerning intelligent interfaces • Evaluation methods and evaluations of implemented intelligent user interfaces • Smart technologies for remote usability testing and experience sampling

  • Processing of human-generated input

• Recognition and interpretation of user input (face, body, speech, physiology, text) • Analysis of psychological user states, such as attention and affect • Analysis of conversational cues, such as grounding and turn taking • Intelligent sensing platforms • Synchronization and fusion of Multimodal Input

  • Generation presentation of system output

• Smart visualization tools • Intelligent authoring systems • Context-aware systems for the generation and presentation of situation-specific output (e.g., on mobile phones, portable tablet devices, wall-size displays, multi-touch screens, meeting accessibility criteria) • Synthesis of multimodal behavior for virtual characters and social robots Intelligent help assistants for complex tasks • Support for collaboration in multiuser environments • Intelligent information and knowledge management

  • Novel trends, innovative solutions and applications

• Novel sensing technologies for games and entertainment • Automotive user interfaces • Novel interaction methods for touch phones and multi touch surfaces

Related Resources

Adaptive Systems & Interaction Group (ASI) at Microsoft Research. Areas of Focus include User Modeling and Intelligent User Interfaces ("We pursue methods for enhancing human-computer interaction via architectures that mesh models of a users' context-sensitive interests, needs, and goals with expressive event systems that sense activity and related information."), Information Access, Filtering, and Management ("We are pursuing principles and applications of technologies that allow information retrieval, filtering, and management. In this realm, we have refined collaborative filtering (CF) algorithms methods for recommending content or services to a user based on the analysis of the behavior of a large number of users."), and Conversational Systems.

COLLAGEN (COLLaborative AGENt) is Java middleware for building collaborative agents. From MERL (the North American arm of the central R&D organization of Mitsubishi Electric Company). "A collaborative agent is a software program that helps users solve problems, especially in complex or unfamiliar domains, by correcting errors, suggesting what to do next, and taking care of low-level details. A collaborative agent can be added to an existing graphical user interface, such as a software simulator, or integrated into the design of a new hardware device, such as a personal video recorder. COLLAGEN is currently being used to build prototype systems for a range of applications, including power plant operator training, car navigation, and spoken-language web form filling."

COMIC. COnversational Multimodal Interaction with Computers. Max Planck Institute for Psycholinguistics, coordinating partner.

Conversational Humanoid. From the Gesture & Narrative Language research group at the MIT Media Lab. "We are developing autonomous agents that are capable of having a real-time face-to-face conversation with a human. These agents are human in form and communicate using both verbal and non-verbal modalities. We believe that such agents provide a new form of human-computer interface which users can interact with naturally, without training, since they already know how to engage in face-to-face conversation with other people."

Emotionally Intelligent Interfaces. From the Rainbow Research Group at the University of Cambridge Computer Laboratory. "Facial displays are an important channel for the expression of emotions and are often thought of as projections or 'read out' of a person’s mental state. Our research objective is to develop emotionally intelligent interfaces that make use of this readily available input modality to adapt and respond to its user."

"FLUIDS [Future Lines of User Interface Decision Support] is a project in the Telematics Engineering sector of the Telematics Applications Programme managed by the European Commission (DG XIII). The project aims to provide a software environment for building intelligent interfaces for decision support systems by using reusable building blocks (adaptable to the requirements of different systems and users) and a methodology for designing and developing interface models." Be sure to read their introduction to Intelligent User Interfaces and then follow the links to New Achievements in Research, Latest Commercial Developments, Research and Market Trends in Europe, Research and Market Trends in Japan and USA, and Related Information Sources.

HCI Research at Stanford. Explore the amazing diversity of contexts in which the science of HCI is being applied.

Human Computer Interaction Institute. Carnegie Mellon University. Among the many, and diverse, research projects that you'll find on this page are "Demonstrational Interfaces ... the user gives an example of how the system should operate, and the system automatically generalizes from the example to produce a parameterized procedure." and "GM/CMU Project: Driver-Vehicle Interface - The goal is to improve security and driver-vehicle interfaces, by building a car that can analyze the driver's intention and watch the driver's physical and mental status for any impairments or information overload. The application combines a smart car environment and monitoring of driver state, with a wide range of input-output modalities."

Human Media Lab. Queen's University. "HML researchers at Queen's University's School of Computing are addressing the problem of the barrage of messages people receive from large numbers of digital appliances. Their Attentive User Interface (AUI) is a new paradigm for interacting with groups of computers that moves beyond the traditional desktop interface. Current computers are generally designed to act in isolation, without considering what the user is doing before producing distracting interruptions. As a result, today’s user has trouble keeping up with volumes of e-mail, instant messages, phone calls and appointment notifications. 'Today’s digital lifestyle has the unfortunate side effect of bombarding people with messages from many devices all the time, regardless of whether they’re willing, or able to respond,' says HML director Dr. Roel Vertegaal. 'Like spam [unsolicited e-mail], this problem needs to be addressed.' The HML team is designing devices that determine the level of user attention and the importance of each message relative to what the user is doing. Then the computer decides whether to 'take a turn' to deliver the message." - from Researchers invent computers that “pay attention” to users. Queen's News Centre (April 1, 2003).

International Conferences on Intelligent User Interfaces (IUI). "Since 1997, the annual conferences on Intelligent User Interfaces (IUI) have been the principal international forums for the presentation and discussion of outstanding research and applications involving intelligent user interfaces, a field that intersects Human Computer Interaction and Artificial Intelligence. In 2007, we will be celebrating the 10th anniversary of the first IUI conference."

International Symposium on Wearable Computers. Topics addressed at the 6th International Symposium on Wearable Computers (October 2002) included: "Human Interface, including hands-free user speech recognition, sensory augmentation, human-centered robotics, user modeling, user evaluations, and health issues."

Relational Agents. From Timothy Bickmore, Assistant Professor College of Computer and Information Science, Northeastern University. "Relational Agents are computational artifacts designed to build and maintain long-term, social-emotional relationships with their users. Central to the notion of relationship is that it is a persistent construct, spanning multiple interactions, thus Relational Agents are explicitly designed to remember past history and manage future expectations in their interactions with users."

  • Also see 'Laura' makes digital health coaching personal. By Catherine Elton. The Boston Globe (, May 21, 2007. "As a computer scientist, one wouldn't expect Timothy Bickmore to concern himself with making sure that schizophrenics take their medications or the elderly get enough exercise. But that's where Laura comes in ..."

STIMULATE. Rutgers University. "[R]esearchers in a project called STIMULATE are developing systems that mimic other forms of communication that humans use to interact with each other, including eye contact, touch and voice. The experimental hardware and software may find uses in medicine, the military and other fields that could benefit from more natural forms of human-computer interaction across distributed networks."

Other References Offline

Ehrlich, K., and A. Henderson. (Inter)facing the Millennium: Where Are We (Going)? Interactions (January-February 2000) pages 19 - 30. A timely and exciting collection of thoughts and viewpoints about HCI - past, present and future.

Mittal, Vibhu O., Holly A Yanko, John Aronis, and Richard Simpson, Editors. 1998. Assistive Technology and Artificial Intelligence: Applications in Robotics, User Interfaces and Natural Language Processing. Berlin: Springer-Verlag. Lecture Notes in Artificial Intelligence #1458. <Preface by B.Buchanan and chapter summaries downloadable for free, individual chapters free for subscribers only.>

Vertegaal, Roel, Editor. March 2003. Attentive user interfaces, a Special Issues of the Communications of the ACM (Volume 46 , Issue 3). The table of contents and abstracts are available to non-subscribers.

Tags: Interfaces
AAAI   Recent Changes   Edit   History   Print   Contact Us
Page last modified on May 10, 2012, at 03:42 PM