Sunday, July 20, 2008

Electronic artificial skins

Cati Vaucelle at Architectradure posted a informative article on the electronic artificial skin.

Link: How to design a reconfigurable artificial sensate skin?

More links to visit

  • V. J. Lumelsky et al., "Sensitive Skin," IEEE Sensor Journal, vol. 1, no. 1, pp. 41-51, Jun. 2001.
    • Lazy summary
      • Requirements to sensitive skin devices
        • Skin materials
        • Sensing devices
        • Signal/data processing
      • Research opportunities
        • New measurment and control theory to use really large amount data from senstive skins
        • Motion planning based on sensitive skin data
        • Bioengineering (see the previous article Links: July 19, 2008)
        • New human-computer interface
  • Flexible tactile sensors could help robots work better, ScienceDaily, May 20, 2005 (See also related stories).

More information

  • Definition of sensitive skin: Sensitive skin is a large array of sensors embedded in a flexible, stretchable, and/or foldable substrate that might cover a moving machine
  • Synonyms: electronic skin, sensitive skin, e-textile, e-skin

Saturday, July 19, 2008

Statistics and survey results on touch screens

Prediction of worldwide touch screen panel market

Touch panel market trend

Source: Displaybank via KISDI touch panel report

Touch panel market Which touch panel technologies are suppliers pursuing?

Source: Touch screen market to reach $3.3 billion by 2015 (cellular-news)


Japanese survey results on touch screen mobile devices

Which touch panel mobile would you want to use?

Link: Touch panels: screen and soft keyboard sources of worry via

Multi Touch Resources: July 19, 2008

Labs, Projects, People

Harvard Biorobotics Laboratory
Tactile Imaging is a way of non-invasively imaging palpable features, especially in soft tissue such as that of the breast.
Wayne Westerman: Founder of FingerWorks, which has been acquired by Apple. He has also invented many multi-touch patents.

References, Resources

Introduction to multi-touch interaction: a PowerPoint lecture material on multi-touch interaction
The haptic community web site: a central online resource for researchers in the haptic community
Best practices of touch screen interface design: A guideline to design touch user interfaces


XSENSOR: XSENSOR measures pressure through its advanced pressure imaging systems, also known as pressure mapping, to offer innovative solutions. The figure below shows a pressure image when a person lies on the bed.
Live pressure imaging sample

An effort at simulating a multi-touch screen with single-touch hardware

Another research work to make a mulit-touch system with a standar touch screen. Some people prefer such an approach due to cost benefit.

Link: Multi-touch simulation on single-touch hardware

See also: Apple iPhone multi-touch patents and multi-touch input with standard touch screens.

Friday, July 18, 2008

Cut and paste tactile sensor

Japanese researchers introduced an interesting concept: cut-and-paste tactile sensors. The figure below illustrates the concept (I think they tried to depict an robot arm, but failed).

Application example of cut and past tactile sensors

Following is the excerpt from their research paper:

The term “cut-and-paste tactile sensors” refers to the installation of the sheets by literally cutting and pasting them on a surface. Moreover, because these sheets are flexible enough to be folded and cut, it is possible to select the location of each tactile sensor element rather freely. An important merit is that only one type of tactile sensor sheet is required for covering a whole system.

Installed tactile sensor sheet consisting of 120 sensing elements.

As shown in the above figure, installation of the tactile sensors is quite simple. Just connect sensor modules and put them on any curved surface.

Link: Y. Ohmura, Y. Kuniyoshi, and A. Nagakubo, "Conformable and Scalable Tactile Sensor Skin for Curved Surfaces," in Proc. the 2006 IEEE Int. Conf. Robotics and Automation, pp. 1348-1353, Orlando, Florida, May 2006.

Thursday, July 17, 2008

Apple's iPhone multi-touch patents - updated July, 2008

Last week, several iPhone and multi-touch patents are published through USPTO.

User Interface

FIG. 9 illustrates a screen display of an interactive gesture learning application in the form of a game according to some embodiments of the present invention.

Although multi-touch increases input freedom, users have to remember many gestures to fully use their multi-touch devices. This patent application presents methods to make people naturally familiar with multi-touch gestures. One example is shown below, where a user learns multi-touch gestures by games such as Tetris, Final Fantasy, etc.



Improvement of usability by removing accidental activation

A plot of empirically determined data illustrating patch minor radii's ability to discriminate between palm contacts and other touch-surface contacts (e.g., fingertips and thumbs).

These patent applications details methods to identify objects on or above a touch screen. Such information is very useful to prevent accidental activation due to inadvertent contact.



A perspective view of an exemplary sensor panel with rows and columns on opposite sides of substrate according to one embodiment of this invention.

In these patents, you can find how Apple engineers mechanically configure the touch panel and other sensors into iPhone.


Sensor calibration

Like other sensors, capacitive touch sensors also suffer from various noises. To enhance the performance of the sensor system, it is crucial to calibrate sensors correctly. US 2008/0158176 and US 2008/0162996 present methods to calibrate sensors automatically.


New multi-touch screens and pads

An exemplary circuit capable of measuring pulse travel times to allow for equalization of the spatial dependency of the pulse travel times according to one embodiment of this invention.

In the patent application US2008/0158198, the author tries to build a single-layered multi-touch screen while iPhone utilizes a two-layered touch panel. Its sensing mechnism is unique. Let me quote the author's summary.

A multi-touch sensor panel can be constructed on a single surface of a touch substrate to reduce manufacturing costs and minimize light loss in transparent embodiments. The panel can be formed as a plurality of distributed RC lines arranged in an array of rows and columns. Each distributed RC line can include alternating connected transistors and metal pads formed on a single surface of a sensor panel substrate, with the drain and source terminals of the transistors connected to adjacent metal pads.

The proposed device is a single-layered capacitive touch screen and cannot capture two dimensional touch images. That is, it also suffers from ghost-touch phenomenon like other conventional touch screens. Although the patent application details disambiguation process, I don't think that it is a true multi-touch screen.

Ghost Touch

However, once the rows and columns containing finger contacts are known, the un-equalized left-to-right, right-to-left, top-to-bottom and bottom-to-top pulse travel time data can be used to determine the relative positions of the fingers within the rows and columns and un-ambiguously determine the positions of all the finger contacts. In particular, for each row indicating a possible contact, the pulse travel times for right-to-left and left-to-right are compared against other rows for which there were possible contacts, one by one, to establish the relative positions of the contacts across the rows. After all rows have been processed, the same process is then repeated for each column showing a possible contact. For each column indicating a possible contact, the pulse travel times for top-to-bottom and bottom-to-top are compared against other columns for which there were possible contacts, one by one, to establish the relative positions of the contacts across the columns.


In this patent, two types of sensors - IR sensors and capacitive touch sensors - are combined together to result in a new proximity detecting touch screens/pads. Following is a quotation from the patent application:

Future touch panels can detect multiple touches and near touches (within the near-field detection capabilities of their touch sensors) occurring at about the same time, and identify and track their locations.

In addition, the authors details various configuration of the hybrid touch sensors:

One or more proximity sensors can be used in conjunction with a multi-touch panel. In some embodiments, an exemplary multi-touch panel can include a proximity sensor located at every touch sensor or pixel. In other embodiments, a proximity sensor can be selectively deployed at certain pixels where the detection of touch or hover may be more critical, or in a spread pattern in broad hover-sensitive areas. In still other embodiments, some rows in the multi-touch panel could be proximity sensor rows, with others being touch sensor rows.


Monday, July 14, 2008

How does a touch screen work has an article defining major componts of touch screens - touch sensor, controller, and software driver.

Link: How does a touch screen works

Friday, July 11, 2008

Touch Screens and Multi Touch Resources - July 10th, 2008

Tactile display applications - a state of the art survey: Today, tactile displays are just research topics of academic field. However, it is worth while to browse the technology to get some insight of the future of touch interfaces.
E-fiel evaluation module: Outdated, but put here for reference purpose. The article deals with the evaluation board of Freescale MC33794 touch sensor IC.
Mosaic Industries Panel-Touch Controller: The product seems to be useful if you want to embed touch controllability to DIY or low-cost electronics project.
Multitouch interfaces is starting to spread among new devices: Written by Walt Mossberg who is a member of All Things Digital tech. site. Well, the article is not so impressive for me.
All Things Digital: Although one of their article is not so impressive, the site seems to get deeper than other casual technology blogs.
Writing drivers for common touch-screen interface hardware: Very practical step-by-step guide to complete your resistive touch screen project.
Pointui: As proven by the success of iPhone, touch technologies cannot survive without excellent user interfaces. The company lists three user interfaces for Windows Mobile PDA and Smartphones. At this time, only Home is downloadable.

Wednesday, July 9, 2008

Non-wearable data glove

This is a little old research work but a good application of touch sensing. Prof. Pai, who is working with Department of Computer Science, Rutgers, presented a ball-shaped 3D input device called Tango.

Grasp approximation results

It can sense a user's grasping pressure with capacitive pressure sensors. The sensors cover the whole surface of the Tango. By processing the measurement signals, it reproduces the 3D shape of the user's hand.

Although the technology is quite different from that used in most data gloves, the result is same - shape and orientation of the hand. This is why I regard the Tango as a non-wearable data glove.

Further readings are listed below:

[1] D. K. Pai, E. W. VanDerLoo, S. Sadhukhan, and P. G. Kry, "The Tango: A Tangible Tangoreceptive Whole-Hand Human Interface," World Haptics Symposium, vol. 1, 2005. (also available from Prof. Pai's web site)
[2] P. G. Kry and D. K. Pai, "Grasp Recognition and Manipulation with the Tango."
[3] F. Montbonnot, "Grasp Recognition and Manipulation with the Tango," Algorithmic Foundations of Robotics VI, 2005.

Tuesday, July 8, 2008

Multi-touch input with standard touch screens

If you want buld a multi-touch system but have only a conventional touch screen, consult following references:

Capacitive touch screens with projective matrix array configuration

Resistive touch screens

Surface capacitive and surface acoustic wave touch screens

Monday, July 7, 2008

How to make a cheap multitouch pad

This is a great DIY video offered by Seth Sandler who is a member of NUI group. NUI group has open source interface projects such as TouchLib, TouchEvent, and OpenTouch.

Shape shifter concept mobile phone

Shape shifter

Shape shifter is a mobile phone that physically changes its frontpanel as user touches. You can find more pictures at Ideo introduced the concept of physically changing touchpad in the Welcome to 2010 report.





 Shape shifter

A multi-touch three dimensional touch-sensitive tablet

A model of a selected sensor in the sensor matrix This paper has been long forgotten from my memory among clutters of my hard disk. The paper is quite old but worthy of browsing, including its citations. Besides, the figure shown in the left clearly illustrates a schematic diagram of capacitive type multi-touch screen adopted in iPhone. Compare this drawing with Fig. 12 of Apple's patent multipoint touchscreen (US2006/0097991 A1).

DOI Link: S. Lee, W. Buxton, and K. C. Smith, "A multi-touch three dimensional touch-sensitive tablet," in CHI '85: Proceedings of the SIGCHI conference on Human factors in computing. New York, NY, USA, 1985, pp. 21--25.

Sunday, July 6, 2008

Touch Screens and Multi Touch Resources for July 6, 2008

Catching up...
digitalDash: digitalDash presents an intersting interface "Reconfigurable Tactile Display (RTD)". It can be regarded as a control panel that can change its front cover dynamically. It is no wonder that RTD cannot dynamically change its knobs and switches. However, as technology advances, I think it is possible we have a truly reconfigurable tactile display as seen below, where the image is from Yanko Design. Finally, the related research field of the RTD is tactile display and you can find many interesting research examples at Someya Lab, Tokyo University.
All in One Haptic Phone
Ki-Uk Kyung's Haptics Homepage: Dr. K.-U. Kyung, who is a research scientist at MIT touch lab, has joined many interesting haptics projects.
F-origin: F-origin originally its business with motion and haptics UI. Now, the company is also armed with a new touch technology called zTouch. zTouch seems to use pressure sensors to locate a user's finger.
PVI: Prime View Internation (PVI) is working on TFT products as well as e-paper solutions. The company is collaborating with F-origin to develop e-paper touch solutions.
atracsys: atracsys offers interactive tabletop solutions using vision-based tracking technologies. They sells interactive systems (beMerlin and atracTable) and tracking systems.
reactable: People at Universitat Pompeu Fabra presented a tabletop interface called reactable.
Display monthly and optronics: They are Japanese technical magazines specialized in display technologies. If you are fluent in Japanese, see an article entitled "入出力デバイスとして注目を集めるタッチ機能付きLCD"
Organic User Interfaces: ACM covers a new UI initiative called organic user interfaces. The concept is intended to provide a uniform framework encompassing various interface ideas including touch, tangible, and flexible interfaces. Interesting point is that the term organic user interface was orginally suggested by J. D. Mackinklay, PARC in the paper entitled "An Organic User Interface for Searching Citation Links". However, the paper considers some GUI issues. The Future of Things provides a good introductory article on organic user interfaces (Link: Flexible Computers Conform to Any Shape).
TWEND (Twisting and Bending as new Interaction Gesture in Mobile Devices): HCI people think that touching is not enough to interact with machines. Media Computing Group at RWTH Aachen University suggests a bending interface for mobile interface in CHI 2008 paper. Note that Sony and Microsoft also proposed similar interfaces. Sony's work is called Gummi and was presented in CHI 2004. Microsoft showed off their own twisting interfaces at UIST 2008 with a paper entitled "I Sense A Disturbance iin the Force: Mobile Device Interaction with Force Sensing".
Multi-touch systems that I have knwon and loved: Written by Bill Buxton. A must-read article if you are working in this field.
Being Human, HCI 2020 Report: I found the article via Touch Usability. It's the collection of thoughts from many HCI researchers and practitioners.
Touch computing hits its stride: Now, business people are also interested in touch interfaces.
A Look Back at CeBIT2008 Displays: Touch technologies are tightly coupled with displays. With the article and related links, you can find good insight on the next step of touch technologies.

Saturday, July 5, 2008

SiMa Systems, Inc.

SiMa Systems, Inc.SiMa Systems offers Multi-Touch and Multi-Force (MT/MF) capabilities with the x-wire touch sensor, which can be applied to touch screens and touch pads. The x-wire sensor is an advanced version of conventional resistive touch screens, allowing both finger and stylus as input method. Other features are dynamic home row, high resolution, auto calibration, and manufacturing using standard materials and processes. From the SiMa System's blog, one can find that the company provides sensor technologies as well as user interface (UI) solutions, which differentiates SiMa Systems from others.

Friday, July 4, 2008

Next touch interface: possible candidates

Although Microsoft and Apple have initiative in touch interface, there are many people who are working on next touch interface.

N-trig DuoSense provides dual mode pen and multi-touch sensing with the combination of capacitive and electrostatic sensing techniques.


Microsoft ThinSight utilizes infrared LED and sensor to detect multi-touch. Unlike MS Surface, its sensor system is tightly coupled with the backpanel of the LCD display.


Microsoft seems to be passionated by touch interface. With LucidTouch, you can control a device by touching its rear side. Unlike other rear side touch, you can still see your fingers although they are moving on the backside of the device.


Mitsubishi should be appreciated as a pioneer of surface computing. DiamondTouch is a table top interface system supporting multi-user collaboration.


Philips have been joined table top interaction with Entertaible, which takes use of infrared sensing technology. 


Sony's SmartSkin is a multi-touch table top interface. The interesting point is that its sensing technology is very similar with that used in iPhone - mutual capacitive sensing.


Touch KO, now acquired by Wacom showed off touchless touch. The so-called Spatial Capacitive Touch lets users control comptuer interfaces by hand movement from up to  6 inches away.


Stantum Multi Touch utilizes resistive touch sensing while most other competitors prefer capacitive or vision-based approaches.


Tactiva Tactapad reconstructs hand motion by using a pressure sensor and camera.


Touch interface is not just for flat, rectangular display.

Yes, you can cotnrol object shown in 3D display by touch. IO2 Technology Helio Display and Eon Reality TouchLight clearly show how you can control 3D virtual objects with hand movement in the air.


Thursday, July 3, 2008

KERI working on transparent electrodes for display screens

▲flexible transparent film manufactured by one-component solution KERI (Korea Electrotechnology Research Institute) announced that they developed a new technique to make transparent electrodes.  In the announcement, KERI reported they developd special conductive liquid that consists of carbon nanotubes (CNT), binders, stabilization compounds, etc. A KERI researcher Geon-Woong Lee said that manufacturers do not need to depend on conventional indium tin oxide (ITO)-based process to get transparent electrodes. Traditional process suffers from high pressure and heat process. Transparent electrodes  are widely used for LCD and touch screen manufacturing processes.

Wednesday, July 2, 2008

Multi-touch patents for July 2, 2008

Sony files haptic touchscreen patent

EE Times Europe reports that Sony patents a "tactile pixel" touchscreen paten. This patent application (US20080150911, Hand-held device with touchscreen and digital tactile pixels) was filed January 21, 2008 and was just published June 26, 2008. The patent describes an array of tactile pixels (104) configured on the perimeter of the touchscreen (102).

FIG. 5 is a three dimensional diagram illustrating use of a portion of a tactile array to indicate an end of a scrolling in a hand-held device according to an embodiment of the present invention.

Based on the configuration, the tactile pixels are turned on an off according to users' touch gestures. In another embodiment, the patent says that a user's tilting action also produces tactile feedback (See figure below)


More interestingly, it also proposes a touchscreen with embedded tactile pixels. However, authors do not illustrate detailed implementation method.


The technology described in the patent is categorized as a tactile display with pin-array type tactile feedback. For example, a KAIST researcher Ki-Uk Kyung proposed a tactile feedback mouse with similar technology.

KAIST tactile feedback mouse


Force, Pressure, and Touch -
Force pressure touch technology: FSR sensor, electronics, firmware and software
Design Service Low Cost Pressure Mapping
Related Posts with Thumbnails