A Comparative Evaluation of
Animation & Small Multiples for
Trend Visualization on Mobile Phones

Matthew Brehmer * · Bongshin Lee · Petra Isenberg · Eun Kyoung Choe

mb@mattbrehmer.ca | @mattbrehmer · @bongshin · @dr_pi · @slowalpaca

mattbrehmer.ca / talks / mobiletrendvis (slides) | tinyurl.com / mobiletrendvis (pre-print)

IEEE VIS 2019 · Oct 23 2019
* Work conducted while affiliated with Microsoft Research, Now with Tableau Research

Animation vs. Small Multiples on Mobile Phones

Crowdsourced Experiment: Animation vs. Small Multiples , between subjects

Task: identify 1-3 targets from a set of 16 items.

The Rise of the Data GIF

Sources (L → R): The Washington Post | Cornell Lab of Ornithology | Pew Research Center.

Curated collections of Data GIFs:

lenagroeger.com / datagifs | Lena Groeger (ProPublica, NICAR Tutorial 2017)
pinterest.com / jsvine / datagifs | Jeremy Singer-Vine (Buzzfeed)

Animation on Mobile, Small Multiples on Desktop?

Data viz solutions: small multiples on desktop, GIFs on yer phone!
Brian Boyer (NPR News Graphics, 2015).

Question: Is there evidence to justify this allocation of designs to different device profiles?

Prior Work: Animation vs. Trails vs. Small Multiples

Effectiveness of Animation in Trend Visualization.
G. Robertson, R. Fernandez, D. Fisher, B. Lee, and J. Stasko. IEEE TVCG (Proc InfoVis 2008).
Inspired by Hans Rosling's 2006-07 TED Talks | Test-of-Time Award recipient at InfoVis 2018.

Prior Work: Animation vs. Trails vs. Small Multiples

Effectiveness of Animation in Trend Visualization. IEEE TVCG (Proc InfoVis 2008).

Lab Experiment:
2 Contexts (Analysis vs. Presentation) | 3 Design conditions | 2 Dataset sizes (large, small) | 24 Tasks

Notable results: Analysis performance higher with multiples; participants preferred animation.

Question: Will we arrive at similar results with mobile phones?

A Crowdsourced Experiment on Mobile Phones

A crowdsourced visualization evaluation study performed exclusively on phones. Crowdsourced Experiment:
1 Context | 2 conditions: vs. | 1 Dataset size (16 items) | 9 Test Tasks (+ Training & QA Tasks)

Tasks *: Targets & Distractors

Tasks adapted from Robertson et al (IEEE TVCG 2008).
* These are simplified representations of our stimuli.

5-Stage Task Format

Show instruction & axes → 5s delay → Start → Select 1 - 3 countries → Done

96 Participants

Small Multiples (N = 45), Animation (N = 51)

Animation or Small Multiples: Overall Results

Detailed results in paper: tinyurl.com / mobiletrendvis

= Faster

= More accurate

= Multiples

= Animation

Animation participants were slightly more confident.

Trajectory Reversals: Animation & SMs at Parity

Small Multiples | Animation | (no performance differences in these two tasks). These tasks involve targets having trajectories that reverse course,
unlike the targets in the other 7 tasks.

Vertical or Long Trajectories: A Case for SMs

Small Multiples (more accurate in these two tasks) | Animation These tasks involve comparing ∆Ys: a ∆Y with no ∆X (top) or the 2 largest ∆Y values (bottom).

Direction of Motion Outliers: A Case for Animation

Small Multiples | Animation (more accurate in these two tasks) These tasks involve assessing both ∆Y and ∆X: target trajectories quite different from distractors.

Conclusions: Animation or Small Multiples?

Surprisingly, Small Multiples are still viable on small displays.

Follow-up work should examine distance and angle comparisons.

Animation and Small Multiples are comparably accurate in > 50% of tasks.

Follow-up work should examine outlier trajectories and reversals.

aka.ms / multiples | (mobile only) experimental app.

github.com / Microsoft / MobileTrendVis | open source app and analysis.

tinyurl.com / mobiletrendvis (pre-print) | mattbrehmer.ca / talks / mobiletrendvis (slides)

mb@mattbrehmer.ca | @mattbrehmer · @bongshin · @dr_pi · @slowalpaca