The COGAIN Technical Committee on eye data quality (scroll down for members list) aims to define standard measures of eye data quality in research reports and across systems. Experts from eye tracker design and eye movement research are working together with the wider eye movements community to reach consensus on measures of accuracy and precision, temporal and spatial resolution and the robustness of eye trackers under variant environmental and individual characteristics. This work is very important to the development of eye tracking and eye movement research for a number of reasons, including:

1. We need to know something about the quality of data recorded for every research report using eye movement measures in order to know if the results are comparable to previous research, to replicate results, and to know if the conclusions drawn are valid. There is currently no formal consensus on how to measure and report data quality.

2. People interested in buying an eye tracker need to know if it can produce data capable of the measures they need, for the people they wish to study and for the environment they wish to record in, or for the single user who wants to use a gaze controlled interface.

3. Designers of gaze controlled interfaces need to know what range of accuracy and precision they can expect from a system in order to design an interface which will operate smoothly on it.

4. Manufacturers want to report, compare and optimise the quality of data their systems produce, and know that other manufacturers are using the same measures. System specifications based on standardised measures are the best way to do this.

Eye data quality standardisation helps to improve eye tracking in general. Everyone benefits.

The current work of the Technical Committee is organised into the following workpackages:

Standardisation of eye data terminology
The terms used to describe data quality are often inadequately defined, or used inconsistently. Agreeing definitions and standardising terminology, both conceptually and mathematically, will allow for consensus on the definition of measurement techniques.

Designing a standardised set of artifical eyes
Artificial eyes are important tools for measuring system error or precision. Since real eyes vary according to colour, shape, and pupil size, artificial eyes should represent a range of eye types and work equally well on diverse systems. An important aspect of the design of artificial eyes is that they must work for both bright and dark pupil methods, we need to measure system temporal performance by simulating a change in gaze position, and robotic eyes are considered to measure the quality of the data with regard to eye dynamics.

Designing a standard experimental protocol for the collection of data for data quality measurement
This workpackage defines the actual experimental procedures to investigate data quality, which should be capable of running on all systems. There is a high degree of collaboration with manufacturers in this work. The aim is to produce open source tools for the calculation of data quality for researchers using a simple routine which can be performed prior to an experiment, and for highly controlled experiments by expert operators in order to compare eye tracking systems.

Selecting samples to include in data quality measurement
This workpackage focuses on how to decide which samples to include for data quality measures, in relation to stimulus / target onset and fixation. We need to define common criteria which can be applied to all systems.

Physiological measures of eye data quality
This workpackage focuses on the eye in motion, and uses knowledge from eye physiology to assess the quality of eye data. Methods considered include dual recording of eye movements with coil based systems and DPI systems. Other methods including identifying thresholds for maximum velocity, comparing main sequence, peak velocity and skew of saccades, among other measures.

Implementing commercial eye trackers to the experimental software
This work involves manufacturers working with the committee and subcommittee to get all available eyetrackers integrated with the experimental software. As of October 2012 the following systems have been or are close to full implementation: SMI systems, SR Research systems, DPI systems, LC Technologies systems. Several other manufacturers have provided systems or committed time to the work in the future, including EyeTech digital systems, tobii systems, and others. This is very concrete work which should accelerate now as more manufacturers willing to spend some days working with us come forward, and as example implementations are complete. The experimental software will be entirely python based. Interfacing with various eyetrackers has been made possible by the donation of large amounts of time from Sol Simpson, formerly of SR Research, currently of iSolver Software (https://sites.google.com/a/isolver-software.com/evolution/).

Funding
We are actively looking for funding for this research. Funding is donated via COGAIN, which is a not-for-profit organisation. We also look for funding from research agencies and governmental bodies. Funding covers travel of committee members for meetings and workshops, equipment and publication. No members of the committee are paid for their work. SMI and SR Research have provided some initial funding for travel of committee members, we are currently asking other manufacturers to join them in supporting this work.

The members of the Technical Committee are:

Andrew Clarke, Charite Medical School, Berlin, Germany
Arantxa Villanueva, Public University of Navarre, Spain
Dan Witzner Hansen, IT University of Copenhagen, Denmark
Dixon Cleveland, LC Technologies, USA
Fiona Mulvey, Lund University, Sweden (Secretary)
Jan Hoffman, SensoMotoric Instruments, Germany
Jeff Pelz, Rochester Institute of Technology, USA
Josh Borah, Applied Science Laboratories, USA
Kara Latorella, NASA Langley, USA
Kenneth Holmqvist, Lund University, Sweden (Chair)
Markus Joos, Interactive Minds Dresden, Germany
Marcus Nyström, Lund University, Sweden
Thomas Haslwanter, Uni. of Applied Sciences, Linz, Austria

ETaccuracyTC@cogain.org

The Technical Committee is supported in their work by a large subcommittee from a broad skill base across eye tracking research and applications:

Andrew Duchowski, Clemson University, USA
Barney Hawes, Sensory Software International Ltd., UK
Bo Hu, Rochester Institute of Technology, USA
Bonita Sharif, Youngstown State University, Ohio, USA
Carlos Morimoto, Universidade de São Paulo, Brazil
Chip Clarke, Assistive Technology Works, Inc., USA
Detlev Droege, University of Koblenz, Germany
Dong Wang, Rochester Institute of Technology, USA
Edwige Pissaloux, Pierre and Marie Curie University, Paris, France
Erik Wastlund, Karlstad University, Sweden
Frank Marchak, Veridical Research and Design corporation, USA
Gintautas Daunys, Sauliai University, Lithuania
Haakon Lund, Royal School of Library and Information Science, Copenhagen, Denmark
Haijun Kang, Kansas State University, USA
Hendrik Koesling, University of Bielefeld, Germany
Henrik Eskilsson, tobii technologies, Sweden
Jacob Fiset, mirametrix, Quebec, Canada
Jayson Turner, Lancaster University, UK
Javier San Agustin, IT University of Copenhagen, Denmark
Joakim Troenge, tobii technologies, Sweden
John Paulin Hansen, IT University of Copenhagen, Denmark
Jonas Andersson, SmartEye, Sweden
Karl Frederick Arrington, Arrington Research, Arizona, USA
Laura Rossi, University of Torino, Italy
Lester C. Loschky, Kansas State University, USA
Linnea Larsson, Lund University, Sweden
Lisa West Åkerblom, tobii technologies, Sweden
Magnus Sjolin, SmartEye, Sweden
Margret Buchholz, DART, Sweden
Marina Green Jarvinen, CTC, Tikoteekki, Finland
Martin Raubal, Swiss Federal Institute of Technology, Zurich, Switzerland
Oleg Spakov, University of Tampere, Finland
Patrick Worfolk, Synaptics, California, USA
Peter Blixt, tobii technologies, Sweden
Pieter Blignaut, University of the Free State, South Africa
Pieter Unema, Consultant, Salt Lake City, USA
Reynold Bailey, Rochester Institute of Technology, New York, USA
Ricardo Matos, tobii technologies, Sweden
Robert Allison, York University, Toronto, Canada
Rudolf Groner, Journal of Eye Movement Research, Bern, Switzerland
Sandra Marshall, San Diego State University, USA
Scott MacKenzie, York University, Canada
Sol Simpson, iSolver software (formerly of SR Research), Canada
Thies Pfeiffer, University of Bielefeld, Germany
Zoi Kapoula, Universite Paris Descartes, France

If you are working in the area of eye tracking research or applications and would like to become a sub committee member, please contact Dr. Fiona Mulvey, fiona.mulvey@humlab.lu.se