Cover Story

UI Sensors Mimic 5 Human Senses
Incorporating "five-sense sensors" makes electronic devices much easier to use; devices of the future may be able to recognize human emotions - a feat which would alter the way in which we relate to them.

The appearance of the iPhone, Wii and similar equipment has finally freed us from the belief that the mouse is the ultimate interface," said Prof Jun Rekimoto, Interfaculty Initiative in Information Studies, University of Tokyo, also director of Interaction Laboratory at Sony Computer Science Laboratories Inc (Sony CSL) of Japan, who has been researching the "multi-touch" user interface since the late 1990s at Sony CSL. It closely resembles the user interface (UI) of the iPhone, released by Apple Inc of the US in 2007, and is years ahead of the Apple design in some ways. When he began his work, recalled Rekimoto, "We felt that practical multi-touch was still years in the future," because the mouse ruled.
Diversification in UIs

Prof Rekimoto believes that there will be a rapid diversification in UIs in the near future thanks to the iPhone and other systems. For example, he explained, equipment UIs will come to mimic the feelings existing between people, letting us exchange feelings through methods like a handshake (Fig 1). "Once a person tries out one of these humanistic UIs, they really dislike using pushbuttons or mice," he added. He is not surprised at the popularity of the iPhone, the Wii or other equipment, but feels rather it was only to be expected.

"The mouse was invented in 1964, and was modeled after a hard, rectangular rock. Computers have no feelings, and it was used to input logical commands, so designers felt there was no need for human warmth," he explained. "Our hands are flexible, though, and a single handshake can communicate quite a lot through force and other interactions. Exchanging feelings with users will be very important in electronic equipment in the future, and it is even possible that UIs may appear mimicking human shapes or the softness of human skin."
The attainment of user interfaces that we can use intuitively, naturally, will depend on sensors. Especially important are sensors covering the five human senses of sight, hearing, smell, taste and touch (Table 1). This is because making UIs more human-like will probably demand that equipment comes with sensors similar to human sensory organs. The iPhone uses a touch sensor. The "smile shutter" function that is beginning to appear in cameras and mobile phones from Sony Corp and other manufacturers, automatically snapping a photo when a smile is detected, extracts smile information from the image. The Wii Fit Balance Wii Board used with the Wii, from Nintendo Co Ltd of Japan, is operated by game users standing on top of it, and makes use of a pressure sensor, as well as tactile information.
Relating to Machines

The five-sense sensors mounted in equipment such as this are still elementary, and continuing evolution could entirely change the relationship between human beings and electronic equipment. Hiro Yamasaki, Professor Emeritus of the University of Tokyo, who has been involved in sensor research for many years, predicts that the development of five-sense sensors will transform this relationship.

Source:
http://techon.nikkeibp.co.jp /article/HONSHI/20080428/151131/