I thought that simple, non-distracting animations that brought in slide elements one at a time as I verbally introduced them was helpful. I thought that it helped members of my audience focus on the slide element that I was talking about.
Seems I might be wrong.
Research carried out by Stephen Mahar, Ulku Yaylacicegi and Thomas Janicki found that students who were shown an animated PowerPoint slideshow learnt less than those that saw a non-animated slideshow.
When I first read of the research on Science Daily Could PowerPoint presentations be stifling learning? I thought the researchers might have indulged in unnecessary and distracting animation. The rather dramatic introduction to the Science Daily article did little to disabuse me of my assumption:
Bold and brassy titles slide into view, tasty slices of pie chart fill the screen one by one, and a hail of arrows spikes the points the lecturer hopes to highlight.
The PowerPoint custom animation they used
I asked Dr Stephen Mahar, one of the authors of the paper, to send me the PowerPoint files he used. He generously sent me samples of the Camtasia files (Camtasia is screencasting software that lets you record narration as you click through PowerPoint slides). I don’t yet have his permission to post them on the blog (I’ll put them up if I do get permission) but meanwhile here is my description of the Camtasia files. Update: I’ve been given permission to publish the screencasts. They’re here: Custom animation screencasts.
Note: If you’re not interested in the details, skip this bit. But if you’re like I was and want to dismiss the results of the research because you think it was “bad” animation you need to read this.
The slideshows were designed to teach internet security principles. The samples Dr Mahar sent me were three minutes long and were composed of three slides:
- One screenshot of a webpage
- One bullet-point slide with four bullets
- One bullet-point slide with five bullets – three with small images.
The slides and the narration of the animated version and the non-animated version were identical. The only difference was the animation. The screenshot slide had four animations:
- A simple entry of a labelled arrow to the “https” in the address bar
- A simple entry of a labelled arrow to the padlock in the address bar
- A simple entry of a circle around the words “view certificate”
- A zoom entry of the Certificate dialog box.
The zoom entry of the dialog box was a bit over the top, but all the other animations were pretty much what I would do, if I were having to present that information.
In the case of the bullet-point slides, each bullet-point was animated with a simple entry. I wouldn’t use bullet-points, but if I were forced to use bullet-points – this is how I would animate them.
The results
Ninety-three students were divided into two groups. They were demographically similar and got similar results on a test prior to the experiment. One group saw the animated version and the other saw the non-animated version.
Why would this be?
The authors give two reasons why animated slides may not help learning:
- Animation increases the load on working memory (this is called cognitive load)
- Animation decreases the time that students are exposed to the information.
Two limitations discussed by the authors
1. There was one question where students who saw the animated version got slightly more correct answers (58.93% to 56.76% for the non-animated version). The researchers speculate that this could be because the students were familiar with the concept tested by this question. Previous research has shown that where custom animation is used to deliver a familiar topic, the animation has a positive impact on student learning.
2. The authors also draw a distinction between teaching a conceptual topic (they classify internet security as a conceptual topic) and a problem-solving technique. As teaching a technique involves teaching one step at a time, with the steps building on each other, they speculate that in this situation the benefits of animation could outweigh the drawbacks.
My thoughts – the importance of cognitive load
1. Out of the sample that I was sent, two out of the three minutes of the screencast were based on bullet-point slides. If the proportion of bullet-point slides is similar in the full-length screencast (the length of the full screencast was 17: 30 minutes), the results may point to a problem with animating bullet-point slides. Trying to read bullet-point slides while the presenter talks increases cognitive load. Had the slides been more visual, the results might have been different.
2. The narration in the screencast was continuous. Students had to pay attention to an animation and the narration at the same time. This would definitely increase cognitive load. If the narrator had stayed silent during the animation, it would have reduced cognitive load and might have lead to different results.
This research has caused me to rethink my assumptions about simple animation. I’ll pay more attention to the amount of time my audience is exposed to different slide elements and ensure I stay silent as I introduce each animation. But I’m not yet convinced that all animation is bad. What are your thoughts?
Fascinating, and more than a little alarming Olivia. We can but hope that this is arising because of a large use of bullets. I’ll look forward to seeing more on this in the weeks to come.
Over the years, we have tried various methods of highlighting elements on screen to draw the audience’s attention. Most of the time now, we dim all but the element we wish to highlight. This can be done in a few seconds in PhotoShop for large visuals; PPT 2003 requires overlaying semi-transparent boxes and it can get quite fiddly lining them up around the featured element. It’s easier in PPT 2007 with cut-outs, but we don’t get to use that as much as so few clients have migrated to it.
And all that fiddling might be to no avail? Gaaah!
Hi Rowan
I can relate. It’s disconcerting when research might come along and say that what you’ve been doing is wrong. But I don’t think that this research does that. We need to wait for more. Olivia
Hi Olivia,
Thanks for this – I have ordered the journal article through inter-library loans and waiting for it is making me crazy! So it was nice to get an early preview, if you like 🙂
Like you, I’m a big fan of cognitive load theory, though I think I’ll need to reserve judgement until I read the full article. But I think you’re right to be cautious: this is one paper and a very specific context. Does it mean that all custom animation is bad? Probably not. But perhaps there are some circumstances in which it is less desirable to use custom animation; as you say, if students need to read bullet-points and listen to the lecturer, that’s going to increase the cognitive load of the overall task, and affect learning. Replacing text with graphics (where possible) could quite possibly make a difference.
It’s all good though – more grist for the mill. I’m hoping to get a paper accepted for publication soon (closely-related topic) so this is just fascinating to me, to see what other people are doing right now. (Fascinating and heart-stopping: every thing like this I read, I panic that someone else has got there before me!)
Great post, thank you.
Hi Chris
Look forward to reading your paper – how exciting! Olivia
I remember a professor “teaching” me how to use PowerPoint in graduate school. I made a sample ppt for a lesson on Upton Sinclair’s book The Jungle. My state standards were aligned, my objectives clear, my lesson engaging. But when the professor returned the project he had taken off because the clipart of a cow wasn’t animated and didn’t included a mooing sound file.
Did the lack of these things really make my presentation less effective?
Hi John
That would be sooo irritating. It does alarm me that people with such little understanding of effective learning principles can be in a position to judge others. Olivia
I would be interested to see the slides as well!
How did they test “learnt more”?
Hi Jan
I’ve published the slides plus more information on the results in my next post http://www.speakingaboutpresenting.com/design/powerpoint-custom-animation-experiment/. Olivia
Yes I agree not all animation is bad. However we sure need to be careful about how and to what extent animations should be used.
Hi Olivia
I so admire that you asked for a copy of the slides. I would love to see them if you get permission, and would also be interested to see the full article.
I’ve just been reading Bad Science by Dr Ben Goldacre this week. Ben is a British doctor whose website badscience.net debunks… well, bad science – studies that are badly put together, and the media reporting that goes with them.
I’m not a scientist, so I found his book very intriguing.
And it obviously had an effect on me.
Now when I read about this study I go:
~ Were the subjects properly randomised?
~ Was the study properly blinded?
~ Were the controls sufficent?
These are things that make significant difference, apparently.
I am very excited that there is evidence-based research going on in our field as we mainly base our thoughts on hunches, anecdotes and non-representative observation (guilty).
I agree with your comments and the comments of… commenters above that we have to be careful to explore how much this study can be generalised.
Having said that, snaps and props to you for bringing this study to the notice of the presentation community.
Thanks Andrew
Hoping to read that book on my long plane journey coming up. And I’m hoping you’re going to an analysis of the research paper…
Olivia