How many times does a user look at her ambient/public display?

As more and more ambient and situated displays inhabit our everyday-life, there is an increasing demand to support the design of these devices with appropriate evaluation methods. A key question here is the analysis of usage patterns over time. The logging of interactions (e.g., touch input) is an integral part to the evaluation method repertoire for interactive systems. However, how can we capture interactions without explicit user input, and if observation or video recording are not appropriate means?

To this end, Trackaware was created. It operates visually by face-detection and  depicts an unobtrusive and lightweight method for the tracking of user awareness and hence user interaction. Trackaware is based on the assumption or simple heuristic that, if the camera of the phone ‘sees’ (detects) a face or multiple faces, then attention is paid to the phone or to its direct surroundings. It is possible to display information on the phone’s (or tablet’s) screen, while at the same time the device’s front facing camera is employed for detecting faces. As a consequence, information can be displayed and awareness can be quantified simultaneously by using one single device. This is a great advantage (no additional hardware and power supply needed, no synchronization, etc.) and makes this system a lightweight solution to conduct unobtrusive field studies.

In summary, Trackaware:

• Is a software for mobile phones/tablets
• It uses the (front-facing) camera to detect whether someone is looking at the device
• For a later evaluation, this information is logged into a database
• For real-time monitoring of studies, this information is sent to an online server, for example, this one
• The software is light-weight and inobstrusive. It runs on 'cheap' and conventional mobile devices. Images don't get recorded, just timestamps
• The mobile device's screen etc. can be used for displaying the content of an application, which use is to be evaluated by means of Trackaware
• Thus, if the to-be-evaluated application is desgined for a mobile phone or tablet, no additional hardware is needed.
• If a big surface is to be monitored, for instance, a large public display or an analogue paper poster, phones with Trackaware installed can be mounted on top of this surface (see one of the images below)

The evaluation of Trackaware (video)

Currently, we are working on the evaluation of the proposed awareness tracking method (Trackaware). Below you can find a video and some screenshots, which were created during the course of quantifying Trackaware's capabilities. The video shows how a Kinect is employed for tracking a participant's positions. Green markers depict coordinates where the participant's face could be detected. Red markers, on the other hand, are locations where the participant has 'travelled' to, but his face was not visible for Trackaware.

Related Links and Images

• "Granny Displays"
• Güldenpfennig, F., Ganhör, R., & Fitzpatrick, G. (2015). Will it catch their attention? Evaluating situated and peripheral displays in a personal context". In Interaction Design and Architecture(s) Journal. PDF.