Microsoft has integrated TRUE as part of their user research to measure “user initiated events” (UIEs) to better understand user behavior. UIEs are not new, and have been used by behaviorist psychologist such as B.F. Skinner to understand animal behavior. In his implementation, called the “Skinner Box” he was able to have many advantages to other measuring tools lacked.
- It did not require an experimenter.
- It improved reliability by reducing experimenter interaction with the animal.
- It could accurately record behavior over a long period of time
Solutions such as Skinner’s where rather simple, however with the combination of larger amounts of data, and more complex systems there has been a growing need to come up with a method of dealing with the massive quantities of data and displaying it in a manner that is interpretable.
TRUE is expands on all the strengths of UIEs while integrating existing User Centered Design methodologies such as usability testing, ethnographic studies, and surveys.
- User interacts with the systems that automatically record user events into log files and uploads them to a server.
- TRUE looks at streams of data through the use meta information such as the timestamp of each event. Understanding the sequence of events could lead to better recommendations.
- Event Sets which are comprised of key information describing various contextual information is packaged with each event.
- Attitudinal data is also included in each event set which alone consist of behavioral data. This is collected from a short survey that is presented to the gamer after each session. By understanding the participants attitude, an event that reads “Player Failed”, could look more like “Player Failed, but had fun”. Information like this in valuable, since failing does not necessarily mean the user is having a bad user experience.
- Data visualization is where the power of TRUE really shines. By presenting data in an interpretable method and allowing transformation, the root cause of problems are easier to identify.
- Create series of graphs
- Use a meaningful hierarchical scheme: (possibly the game genre, then specific game, then UIEs)
- Supports drilling down the data
- Supports cross-linking
- Ethnographic video is synchronized to timestamps of event stream
As you could see TRUE is an instrumentation system that combines a large variety of methodologies that enable Microsoft to:
- detect issues and understand root causes in the same way usability testing does
- to support design iteration
- to incorporate attitudinal behavior in the manner of surveys
- to understand the naturalistic use of our products as is found with ethnographic methods.
- enable unobtrusive naturalistic observations of usage over time
- see things such as extended usage patterns, data from large numbers of participants, and data that is difficult to observe but impacts user experience
Kim, J. H., Gunn, D., Schuh, B, Phillips, B, Pagulayan, R. and Wixon, D. Tracking Real-Time User Experience (TRUE): A comprehensive instrumentation solution for complex systems. In Proc. CHI 2008, 443-451.
Popularity: 10% [?]