Session:「Methods for exploration」

Rethinking Thinking Aloud: A Comparison of Three Think-Aloud Protocols

論文URL: http://dl.acm.org/citation.cfm?doid=3173574.3173618

論文アブストラクト: This paper presents the results of a study that compared three think-aloud methods: concurrent think-aloud, retrospective think-aloud, and a hybrid method. The three methods were compared through an evaluation of a library website, which involved four points of comparison: task performance, participants' experiences, usability problems discovered, and the cost of employing the methods. The results revealed that the concurrent method outperformed both the retrospective and the hybrid methods in facilitating successful usability testing. It detected higher numbers of usability problems than the retrospective method, and produced output comparable to that of the hybrid method. The method received average to positive ratings from its users, and no reactivity was observed. Lastly, this method required much less time on the evaluator's part than did the other two methods, which involved double the testing and analysis time.

日本語のまとめ:

ユーザビリティテストで用いられる3つの思考発話法のうちどれが優れているか図書館ウェブサイトの評価を通して評価した。その結果リアルタイム思考発話法(concurrent think-aloud)が最も優れていることが判明した。

“Is More Better?”: Impact of Multiple Photos on Perception of Persona Profiles

論文URL: http://dl.acm.org/citation.cfm?doid=3173574.3173891

論文アブストラクト: In this research, we investigate if and how more photos than a single headshot can heighten the level of information provided by persona profiles. We conduct eye-tracking experiments and qualitative interviews with variations in the photos: a single headshot, a headshot and images of the persona in different contexts, and a headshot with pictures of different people representing key persona attributes. The results show that more contextual photos significantly improve the information end users derive from a persona profile; however, showing images of different people creates confusion and lowers the informativeness. Moreover, we discover that choice of pictures results in various interpretations of the persona that are biased by the end users' experiences and preconceptions. The results imply that persona creators should consider the design power of photos when creating persona profiles.

日本語のまとめ:

ペルソナのプロフィールに付加する画像について、ペルソナの異なる状況における写真はエンドユーザーにとっ てプロフィールからの情報を改善するが、他の人物との写真を示すことは混乱をもたらし情報性を低下させる。写真はエンドユーザの経験と先入観によっ て様々な解釈をもたらすのでプロフィール作成者は写真を考慮する必要がある。

Semi-Automated Coding for Qualitative Research: A User-Centered Inquiry and Initial Prototypes

論文URL: http://dl.acm.org/citation.cfm?doid=3173574.3173922

論文アブストラクト: Qualitative researchers perform an important and painstaking data annotation process known as coding. However, much of the process can be tedious and repetitive, becoming prohibitive for large datasets. Could coding be partially automated, and should it be? To answer this question, we interviewed researchers and observed them code interview transcripts. We found that across disciplines, researchers follow several coding practices well-suited to automation. Further, researchers desire automation after having developed a codebook and coded a subset of data, particularly in extending their coding to unseen data. Researchers also require any assistive tool to be transparent about its recommendations. Based on our findings, we built prototypes to partially automate coding using simple natural language processing techniques. Our top-performing system generates coding that matches human coders on inter-rater reliability measures. We discuss implications for interface and algorithm design, meta-issues around automating qualitative research, and suggestions for future work.

日本語のまとめ:

定性的研究を行う研究者は「コーディング」と呼ばれるデータアノテーションを行う。我々は自動化に適したいくつかのコーディング手法に従うことを発見した。我々はコーディングを自動化するプロトタイプを作成し、このシステムは人間と同程度のコーディングを行う。

A Bermuda Triangle?

論文URL: http://dl.acm.org/citation.cfm?doid=3173574.3174035

論文アブストラクト: User experience (UX) evaluation is a growing field with diverse approaches. To understand the development since previous meta-review efforts, we conducted a state-of-the-art review of UX evaluation techniques with special attention to the triangulation between methods. We systematically selected and analyzed 100 papers from recent years and while we found an increase of relevant UX studies, we also saw a remaining overlap with pure usability evaluations. Positive trends include an increasing percentage of field rather than lab studies and a tendency to combine several methods in UX studies. Triangulation was applied in more than two thirds of the studies, and the most common method combination was questionnaires and interviews. Based on our analysis, we derive common patterns for triangulation in UX evaluation efforts. A critical discussion about existing approaches should help to obtain stronger results, especially when evaluating new technologies.

日本語のまとめ:

UX評価について近年から100件の論文を体系的に選択して分析したところ、複数の調査方法を使用してのUX評価は3分の2以上の研究に適用され、最も一般的な方法の組み合わせはアンケートとインタビューである。