We present a novel headband-type wearable device called FaceLooks, used for measuring the time of the face-to-face state with an identity of the partner, using an infrared emitter and receiver. It can also be used for behavioral healthcare applications, such as for children with developmental disorders who exhibit difficulties with the behavior, by providing awareness through the visual feedback from the partner’s device. Two laboratory experiments showed the device’s detection range and response time, tested with a pair of dummy heads. Another laboratory experiment was done with human participants with gaze trackers and showed the device’s substantial agreement with a human observer. We then conducted two field studies involving children with intellectual disabilities and/or autism spectrum disorders. The first study showed that the devices could be used in the school setting, observing the children did not remove the devices. The second study showed that the duration of children’s face-to-face behavior could be increased under a visual feedback condition. The device shows its potential to be used in therapy and experimental fields because of its wearability and its ability to quantify and shape face-to-face behavior.
We present EnhancedTouch, a novel bracelet-type wearable device for facilitating human-human physical touch. In particular, we aim to support children with autism spectrum disorder (ASD), who often exhibit particular communication patterns, such as lack of physical touch. EnhancedTouch is a unique device that can measure human-human touch events and provide visual feedback to augment touch interaction. We employ personal area network (PAN) technology for communication with partner devices via modulated electrical current flowing through the users’ hands. Our user study shows that the visual feedback provided by the developed bracelet motivates children with ASD to touch one another. Moreover, EnhancedTouch offers a function to record the time and duration of a touch event as well as the identity of the touched person. This allows us to identify and evaluate intervention based on physical touch in a quantitative manner.
Multiple Texture Button
We present Multiple Texture Button, which provides various haptic click sensations. A physical button that is mounted in a mouse, a keyboard, or a shutter of a camera, provides both force and tactile feedback when a user presses it. It is certainly that users can recognize the press and input through the sensation of “click” vibrations. However, one physical button only provides one type of the vibration. We developed Multiple Texture Button that presents a variety of sensations by combining the actual physical buttons and vibration. A video game designer can modulate the click sensation of the button in real time according to the context of the content using Multiple Texture Button.
We present a system called HapTONE that modulates the touch sensations of a keyboard. HapTONE provides the player with different vibrotactile sensations, not only after pressing a keyboard but also during the pressing operation itself. This is done by using optical distance sensors and vibrators placed in the keyboard. This system reproduces the touch sensation of a keyboard, stringed, wind, percussion or non-musical instrument. HapTONE is designed as a musical entertainment system for players themselves.
We present VacuumTouch, a novel haptic interface architecture for touch screens that provides attractive force feedback to the user’s finger. VacuumTouch consists of an air pump and solenoid air valves that connect to the surface of the touch screen and suck the air above the surface where the user’s finger makes contact. VacuumTouch does not require the user to hold or attach additional devices to provide the attractive force, which allows for easy interaction with the surface. This paper introduces the implementation of the VacuumTouch architecture and some applications for enhancement of the graphical user interface, namely a suction button, a suction slider, and a suction dial. The quantitative evaluation was conducted with the suction dial and showed that the attractive force provided by VacuumTouch improved the performance of the dial menu interface and its potential effects.
Ear Cleaning with Auditory Feedback
We present auditory feedback of earpick interaction to provide an additional cue. Ear cleaning is familiar for Asian people not only for medical care but also for relaxation and communication. However, cleaning the other person’s ear has a risk of injuring because it is difficult to grasp movement and position of the earpick. Since the visual and auditory cues are limited, users must rely only on somatosensory cues. On the other hand, an endoscope to see inside the ear is expensive. Our developed device is composed of microphone and earphone to catch and replay the sound associated with ear cleaning in real-time. We envision that this technique would improve safety and efficiency of ear cleaning.
HACHIStack and HaCHIStick
We present a novel photo touch sensing architecture, HACHIStack (Haptic Audio Computer-Human Interaction Stack). With the touch screen using two optical sensing layers above the surface, the developed touch sensor have three unique capabilities: high-speed sampling, velocity acquisition, and contact time prediction. This work quantitatively examines these capabilities through two laboratory experiments and confirms that the capabilities of HACHIStack are sufficient for multimodal interaction, in particular, touch-based interaction with haptic enhancement. We introduce a virtual piano instrument that allows players to perform weak and strong strokes by changing the tapping velocity. In addition, combining with a stick device that presents low-latency and high-resolution vibration, the system offers chromatic percussions (xylophone and glockenspiel) with haptic feedback and no-delay haptic feedback with the sensation of tapping on various simulated materials (e.g., rubber, wood, and aluminum).
Virtual Alteration of Body Material
Characters with body materials that are different from that of humans, such as metal robots or rubber people, frequently appear in movies and comics. While the abilities of their synthetic bodies can be easily observed from their actions, their somatic sensations are more difficult to appreciate. In this work, we simulate the alteration of the material of the human body by means of vibrotactile feedback. The feedback represents the properties of the materials and is periodically applied to the elbow joint in synchrony with the elbow angle. This simulated sensation of having a different body material gives us the feeling of those characters.
We developed a virtual robotization system that provides a robot-like feeling to the human body not only by using a visual display and sound effects but also by rendering a robot’s haptic vibration to the user’s arm. The vibrotactile stimulus was modeled by measuring vibration generated by a robot arm and using linear predictive coding. We experimentally confirmed that the subjective robot-like feeling was significantly increased by combining the robot-vibration feedback with a robot-joint animation and creaking sound effects.
Augmentation of Toothbrush
Brushing teeth is a daily habit to maintain oral hygiene, including the maintenance of oral cleanliness and prevention of caries and periodontal disease. However, tooth brushing is often not carried out correctly or forgotten because the task is boring. We present a novel approach to augment the tooth brushing experience by modulating the brushing sounds to make tooth brushing entertaining in an intuitive manner. In the experiment, we demonstrate that increasing the sound gain and manipulating the frequency can control the overall impression of brushing by giving a sense of comfort and accomplishment.
Pouring-Liquid Feeling “Glug-Glug” Feedback
It is known that visual, auditory, and tactile modalities affect the experiences of eating and drinking. One such example is the “glug” sound and vibration from a bottle when pouring liquid. We measured and modeled the wave of the vibration by summation of two decaying sinusoidal waves with different frequencies; we examined the validity of this model by subjective evaluation. Furthermore, to enrich expression of various types of liquid, we included two new properties of liquid: the viscosity and the residual amount of liquid, both based on measured data.
Augmented Obstacle Sensation
Inspired by obstacle sense-blind people localizing only with hearing, we present a novel method to augment existence. Obstacle sense is caused mainly by coloration by reflected sound and the attenuation by shielding. By focusing on the attenuation, we found an effective sense can be created by decreasing high-frequency component and increasing low-frequency component simultaneously. Experiments were conducted to evaluate our approach.
In sports, dancing and playing music, it is important to achieve correct body movement as it greatly affects performance. However, matching one’s movement with ideal movement is fundamentally difficult, because we do not have a detailed perception of our own body movement. In this study, we present “rotary switch feeling” feedback as a new haptic cue.
Tick-Tick Accelerator Pedal
An accelerator pedal is controlled by the foot, but precise adjustment requires much training because the driver must rely on only somatosensory cues. In this study, we present periodic tactile feedback for the accelerator pedal to provide an additional tactile cue. We conducted an experiment using a driving simulator. The results show that the feedback makes the control of acceleration easier and facilitates safer driving.
Haptic sensations can be induced without requiring haptic devices through pseudo-haptic feedback by the influence of another sensory modality, such as vision. In this research, we present two novel approaches that combine pseudo-haptic feedback with visual and tactile vibrations in order to augment the overall haptic sensation. The first technique enhances pseudo-haptic textures with a stripe pattern that provides a vibrotactile stimulus. The second technique modulates the perceived material stiffness of a virtual object using vibratory models for visual and tactile feedback, introducing novel pseudo-haptic effect based on visual vibrations.
Virtual Chromatic Percussion
Musical video games that allow users to play expensive musical instruments in a virtual environment constitute one of the most popular genres in the field of video games. Recent developments in motion input technology have enabled users to play the instruments intuitively and immersively. However, output technology, in particular haptic feedback, is not as advanced as input technology. We present a system for playing virtual chromatic percussion, where the haptic feedback changes according to the instrument, as well as the acoustic feedback. In this research, we employed visual and tactile vibrations to enrich the haptic sensation of the content. We also present an application that provides a virtual environment for playing two chromatic percussion instruments, namely the xylophone and glockenspiel.
Today in general traffic field, visual signs and audio cues are used for pedestrian control. In this study, we present a new visual navigation method for pedestrians using “Vection Field”, where the optical flow is presented on the floor. The optical flow is presented by using lenticular lenses. The lenticular lens, that is the passive optical element, generates the visual stimulus based on the pedestrian’s movement without electrical power supply. We designed a basic visual stimulus and evaluated principle of our method for the directional navigation. The results revealed that the optical-flow composed of stripes and random-dot pattern both displace pedestrian’s pathway significantly.
Soap bubble, whose transparent sphere has colored pattern caused by the interference effect of visible light, is beautiful and has enchanted people through the ages. However, it is difficult to appreciate it for a long time because it is fragile. We present a pseudo-soap bubble dubbed “Bubble Candle”, which lasts almost permanently by using rotating grating sheets instead of soapy water. It offers the interaction that allows a viewer to control the colored pattern and the shape of the bubble.
Haptic Illusion of Elasticity
Thus far, numerous precedents of haptic illusions perceived in static condition have been reported, such as the “Aristotle’s illusion”. We found a new illusion as if an elastic object bounced on a palm to the accompaniment of a forearm movement if we gave a steadily vibratory tactile stimulation to the palm while moving the forearm vertically. This is a combined illusion of skin sensation by steadily vibratory tactile stimulation to the palm and proprioceptive sensation by vertical forearm motion. In this study, we investigate the mechanism of this new haptic illusion focusing on the temporal change of tactile suppression during periodical forearm motion.