Publications Professional Activities Bio-sketch Press Lectures/Talks Contact Info

Internship Opportunity at Google Research: I regularly look for research interns (summer or otherwise). The candidates should be very good at developing software and systems. Previous research experience is not required, but a career interest in research, technology and user interface innovation is a must. Send me an email if you or someone you know should be interested.

Welcome to Shumin Zhai's research page
Shumin Zhai, photo, 2007
Shumin Zhai is a Human-Computer Interaction research scientist interested in both foundational issues of user interfaces and practical innovations. He joined Google in Mountain View California in January 2011. Previously he worked at the IBM Almaden Research Center in San Jose California for 15 years. He originated and led the SHARK/ShapeWriter project that pioneered the touch screen gesture typing paradigm. He is active in the HCI academic community and is currently the Editor-in-Chief of ACM Transactions on Computer-Human Interaction. He has been a visiting professor and lectured at universities in the US, Europe and China. He is a Fellow of the ACM and a Member of the CHI Academy.

Select Recent Publications (see the more complete publications)

Foundational Issues in Touch-Surface Stroke Gesture Design — An Integrative Review
S. Zhai, P. O. Kristensson, C. Appert, T. H. Andersen, and X. Cao
Foundations and Trends in Human–Computer Interaction Vol. 5, No. 2, 97–205, 2012

The word-gesture keyboard: reimagining keyboard interaction
Shumin Zhai, Per Ola Kristensson
Communications of the ACM, 2012

Bimanual gesture keyboard, Xiaojun Bi, Ciprian Chelba, Tom Ouyang, Kurt Partridge, and Shumin Zhai,. In Proceedings of the 25th annual ACM symposium on User interface software and technology (UIST '12). ACM, 137-146.

Multilingual Touchscreen Keyboard Design and Optimization
Xiaojun Bi, Barton A. Smith, Shumin Zhai.
Human-Computer Interaction, Volume 27, Issue 4, 352-382. 2012

Research Highlights (see publications)

Word-Gesture Keyboard

Human-Computer/Information interaction is rapidly moving from a desktop model to an multi-device and information cloud model. Text is an indispensable form of information. With this background, the ShapeWriter project (with Per Ola Kristensson and ShapeWriter Inc) pioneered gesture typing on smart keyboard as a new paradigm of information input on touch screens.  

Project Home Page|“A new way of writing" by Mike Langberg of Mercury News| A brief history of ShapeWriter | "Top 11 iPhone Applications" by Time.com | ADC 50 - Android Developer Challenge Winner | ShapeWriter Inc | Text Entry Epiphany” - the first comprehensive ShapeWriter review in 2004 | iPhone ShapeWriter user reviews | Key papers on ShapeWriter| Press Coverage |

FonePal

FonePal is a multi channel, multi modal and multi device solution to the “Touchtone Hell” IVR problem. See a video demo and papers.

Eye-tracking augmented user interfaces

The eye, being simultaneously the mind's window to the world and a world's window to the mind, has many potential applications in human-computer interaction. Two applications I have been involved in are MAGIC Pointing (with Ihde and Morimoto) and iTourist (with P. Qvarfordt). I have argued that whenever possible the eye-gaze should be used as a contextual and implicit, rather than a direct and explicit modality of computer input. See an CACM article on eye-tracking, and my keynote abstract and slides at the 2008 ACM Eye-tracking Research and Applications Symposium.

Laws of action

In any research field establishing robust laws and regularities are fundamental to the field's development. This is considerably harder in HCI given the complexity of human performance, behavior and experience. Fortunately some perceptual-motor actions in HCI can be modeled by “Laws of Action". Contributions I have helped to make in this area include 1. Clarifying a fundamental logical error of using a compound throughput (TP) metric to characterize computer input with Fitts' law (MT = a + b ID) and suggesting that a and b separately represent the non-informational and informational aspects of pointing; 2. (With Kong and Ren) Understanding actual and nominal pointing precision in Fitts’ law tasks.3. (With J. Accot) A more rigorous understanding and formulation of 2D Fitts' law, or more precisely models of pointing with simultaneous amplitude and directional constraint. 4 (With J. Accot) The regularities in crossing actions (“more than dotting the i's). 5 (With J. Accot) The steering law, its connection to Fitts' law and its strength against movement scaling. 6 (With X. Cao) The CLC model of gesture stroke complexity.

ScrollPointTM Mouse

A common challenge to research is the lack of opportunity to make a broad and direct impact on technologies used by real users. I was fortunate to work with (and lead) a team of engineers from IBM and IBM vendors (and their vendors) to bring the ScrollPoint mouse from research to market (received a CES award and millions of users). In addition to the central function, every “peripheral” factors can also make or break a product, including cost, design (both deep and surface), packaging and install, and retrofitting its software to operating systems that are not designed to support new user interfaces. HCI researchers and practitioners with broad skills are uniquely suited to drive a user experience centered system engineering process in product development. For example, in order to balance between cost and quality, a deep user experience and human performance understanding can be applied to decisions on sensor quality, processor speed, A/D conversion resolution, and the shape and form appeal of a mouse. I hope one day to find the time to write “The tale of a mouse”.

6 DOF input control

Researchers sometimes wonder whether academic research is really useful to the larger world. This project on multiple degrees of freedom input control was rather academic and abstract when it was done (advised by P. Milgram and B. Buxton at the University of Toronto), but it has found surprising amount of practical (and academic) applications. Researchers, engineers, designers and industry executives personally tell me how they use the concepts, understanding, and methods developed in this research project in their design, development, and research of new 6DOF controllers. Key contributions of this research include the taxonomy of controller resistance (isometric, elastic, and isotonic), the relationship between transfer function and controller property (e.g. rate control's compatibility with self-centering devices), the effect of different muscle groups in 6DOF manipulation, effective evaluation tasks (6DOF docking and 6DOF tracking), and methods of quantifying coordination in multiple degrees of freedom input control. The work was driven by the need of designing telerobotics and virtual reality interfaces, but the recent wild success of the Nintendo Wii has liberated multiple DOF controllers from these specialized fields to ordinary households. Read more on manipulation and navigation in 3D interfaces.

Professional Activities

Editor-in-Chief, ACM Transactions on Computer-Human Interaction

Other Conference Committees, Editorial Boards, Adjunct Appointments

Research Collaborators

A list of my colleagues, Postdoctoral fellows, PhD students, PhD committees, Academic visitors, Hosts of academic visits, and Graduate interns.

Contact Info

last name at acm.org for academic and ACM TOCHI journal communications

first name dot last name at gmail.com for personal emails

(408)476-6038 ( Mobile )