Tablet exams: study guides and review sessions

I don't really like to hold exam review sessions. But students ask for them. And they ask for "study guides" (for students who might be reading this, here's a hint: the lecture slides, the syllabus, your textbook, and your notes are the study guide!) This post explores new approaches using tablet computers to potentially improve the efficacy of

I have three predominant concerns about review sessions:

1. I'm not sure they necessarily help students who attend; they definitely don't help students who are unable to attend. My experience suggests that many of the students who will show up to a review session are the motivated, high-achieving students; these students may only make incremental gains by attending an office hour.
2. Related, it is next to impossible to pick a time (and find an empty classroom) that works with everybody's schedule. So, it seems almost inherently unfair to hold review sessions, as they will certainly exclude attendance by some students.
3. They never seem to be very efficient. This might be an outcome of a combination of my implementation of the review session. The format I use is that I show up and students ask me questions about content. I don't bring any prepared agenda. So, in this sense, the review session is merely an extended office hour in a space large enough to accommodate a large number of students. I often schedule exams on Mondays, to give students the weekend to study, but this also means that students typically (if my experience is still typical of college students) don't start studying until the weekend just before the exam. Because I strongly discourage last-minute (i.e. Monday earlier in the day than the exam) question-asking, I will schedule a review session for the previous Thursday or Friday - unfortunately, this is before most of the students have probably started reviewing the material and formulating questions to ask. So, many "A" students tend to show up just to hear questions from other students and my responses. Much of the time is spent with me waiting for students to actively leaf through their notes and find topics that they want to review.

New review session approach
I just held an exam review session for my genetics students; I used my tablet to make a change this time, and I'm excited to follow the outcomes over the next week. I recorded the review session. This isn't a major leap, as I'm already recording pre-lectures and in-class lectures and posting them on YouTube for students to access. However, the reason I'm hopeful that this new practice will improve student success is because I'm addressing some of my concerns above.

Benefit to students
Students who are unable to attend a review session scheduled at a time other than our normal class meeting time are now able to access the exact same content as those who were in attendance. I'm giving students another resource to help them help themselves succeed, and it takes next to no additional effort on my part.

Best practices

• After I exported my screencast video of the review session, I spent ten minutes going through the video and noting the time when I began responding to each question, along with the topic of that question. When I posted the URL to the video on our learning management system, I also provided this guide to the video, so that students don't have to watch over an hour of video to find out whether there are parts of the video that address a specific question they may have
• Although nobody used this opportunity, I had suggested that students should e-mail or tweet me questions that I could discuss during the live review session

On Study Guides & Answer Keys

My students seemed particularly concerned before my first exam about whether I could provide an example exam for them to see the types of questions I write. Because I had only taught this course once before, I declined to provide my meager stockpile of exam questions to the class.

After the first exam, as part of a quiz (in Socrative), I asked each student to write an exam question (and answer) that they thought represented the type of question I might ask on the upcoming second exam. This was a useful exercise for two reasons: it made students consciously assess what content they thought was exam material, and it also gave me a stockpile of potential exam questions! To motivate the students to take this task of question-writing seriously, I told them that I would choose one of their questions and use it on the exam. From the rest of the questions, I chose several and published those to the class as a "study guide" (more like a practice exam) that covered the highlights, as chosen by the class, of the exam material.

So, I distributed a PDF of this study guide, and a student asked me, almost immediately, whether I was going to post a screencast key. What a great idea! So, instead of filling out a printout of the study guide by hand, scanning it, and posting it as a PDF, I used my established process of annotating a PDF using ExplainEverything and then exporting the annotation as a movie. The reason I'm particularly keen about this approach (and glad that a student suggested it!) is that, at least in genetics, it is difficult for many students to get a handle on the order of steps (the process) necessary to solve a multi-step problem. Because the normal answer keys they see are static (a printout posted on a bulletin board, or a digital text document or PDF), it can be difficult to understand the logic behind how the key-writer arrived at the answer. I hope that being able to see (on-demand, probably the weekend right before the exam) example problems being solved as screencast videos with audio voiceover will help students improve their understanding of the key concepts I'm trying to help them learn.

Explain Everything and blended learning

In today's post, I'll give my impressions on how tablets are extremely useful for helping faculty "flip the classroom" and to give students more tools for succeeding in class.

As soon as I heard about the app Explain Everything, which is my most-used app, I knew that one of the main ways I would use my tablet was to create a blended learning course (aka a "flipped classroom"). Explain Everything, which costs next to nothing to purchase, is a dual-purpose (or more!) app: I present information to students with it (like one might use PowerPoint), but it also records audio and written annotation!

Purpose 1: Explain Everything Before Class

I'm now using Explain Everything to voice-over (and annotate) pre-lecture content. Before every chapter of my textbook, I record a brief (5 to 20 minute) screencast lecture to try to convey the basic, factual information that the students will need to engage in what we do in class: work through problems and examples and answer student questions. To ensure compliance watching these videos in advance, I have a low-point quiz on every pre-lecture video.

Pre-Class Workflow for Screencasting

1. Create slides (little text - maybe just a title) containing graphics (photographs, primary data, etc.) that I wouldn't want to reproduce by hand. I use PowerPoint for this
2. Export slides as a PDF file
3. Copy the PDF file on my computer to my Google Drive
4. Open the multi-page PDF file in Explain Everything (which converts each page of the PDF to its own slide)
5. Press the record button and start talking and making on-screen annotations using my finger and the tablet touchscreen (only audio and touchscreen-based activity are recorded - not video from the tablet camera - so the students just see what's displayed on the tablet screen and not the instructor's face)
6. When I'm done navigating through the presentation, I export the presentation (audio plus all of my live annotations) as a movie back to my Google Drive account
7. I upload the movie to YouTube (although I could export directly from Explain Everything to YouTube, I like keeping a copy of the presentation as a movie file on my laptop)
8. I'll describe the YouTube side of the workflow in a later post, but briefly: students know where to look on YouTube to find the videos I upload (I provide a link to my YouTube channel on our learning management system website).

Purpose 2: Explain Everything: Presenting In Class

I also use Explain Everything for my in-class presentations, which, keeping true to the blended learning approach, usually comprise activities/exercises I've devised for doing in class. Practicing applying the knowledge the students gained before coming to class (by reading the textbook and watching the pre-class video) is a really valuable use of the face time I have with students.

In-Class Workflow for Presenting

1. I create slides in PowerPoint and export them as a PDF file, or I create a worksheet/activity in Adobe InDesign (page layout program) and export that as a PDF
2. Import the PDFs to Explain Everything for presentation. Available tools for Explain Everything include basic shapes, a pen with variable width and color, an eraser, a text tool, zoom tool, and more advanced tools (layering, locking objects, etc. and a LASER pointer tool). Another fabulous thing about Explain Everything is that each slide can be drawn on (and, of course, those annotations are displayed by projector to the class)
3. I draw on my tablet (or use the LASER pointer tool to draw attention to particular parts of the slide), and students see that in real-time. This is writing on a digital white board, on top of graphics that I provide! And, of course, because I always upload my lecture PDF to our learning management system before class, all of the students follow along on their tablets (many of my students are taking their own hand-written annotations on their copies of the lecture slides).

I should explicitly mention that I'm not having students themselves use Explain Everything. Because the app is not free (though only a couple dollars and change, the last time I checked), I'm not requiring students to purchase any apps for this class.

In my flipped classroom, when I write here about designing exercises/activities, those are things that the students are doing on their tablets. This is most frequently by distributing to the students a PDF of the exercise, asking them to annotate/draw on the PDF (the topic of yet another future post), and then return the annotated PDF to me by e-mail attachment - in cases where I want to collect evidence of their work. Some of my colleagues are having the students purchase Explain Everything and then use them to produce and deliver presentations on content, which is certainly another great use of this app.

Purpose 3: Recording during class

What comes next is the very best part: I always record my in-class presentation! The audio picks up my voice, and I try to remember to re-state student questions so that they are audible on the recording. (One minor drawback is that recording only occurs while in Explain Everything, so when I switch to a different app during class, that part doesn't get recorded). As with the advance screencast lecture capture, the audio gets recorded along with all of the on-screen annotation with the Explain Everything tools (pen, LASER pointer, etc.) After class, I export and upload every class session as a movie to YouTube, just like the screencast lectures.

In a future post, I'll delve into evidence I have that students are taking advantage of these resources (the pre-class screencast lecture content; the in-class recording)!

Example

Here's the first pre-lecture video I ever recorded (this was for a graduate course I taught in a previous term, as practice for tablet teaching this term):

https://www.youtube.com/watch?v=s0AwWBA3Vfo

I'm not pointing you toward my current-term videos because I'm trying to limit views of that content only to current students. For reasons mentioned earlier: I'm gathering data by tracking viewing habits, and while I would be honored to have you watch those videos as well, I'm afraid you'd skew my data. So, maybe I'll more broadly advertise my current tablet course videos after the term ends!

Happy teaching to you all!

Musings on tablet-based exams

A few days ago, my syllabus indicated that an exam was to be given in my two sections of genetics (one traditional lecture, one tablet-based). The educator in me wanted to give different versions of the exam, so that the tablet students could have an exam that harnessed the full potential of having tablets in class. However, the scientist in me was too curious to see whether there might be a difference in learning by this point in the course, so I gave both sections the same version of this exam.

Distributing the exam

In the traditional section, I provided the exam in paper and students turned it in to me at the end of fifty minutes. In the tablet section, I uploaded a PDF of the exam into our course management system, and at the top of the hour, it became available to the registered students. Having already spent time doing PDF annotation in previous class periods with my students, I was confident that they wouldn't encounter any technological issues completing an exam in this fashion.

However, just in case, I had paper copies of the exam on hand. You know: in case the internet went down. It could happen. I had comforted by students with this information in advance, so they knew paper copies would be available; five of my thirty students opted to take the exam on paper instead of by PDF annotation. Unfortunately, this nullified my earlier claim that tablet courses would be "green" because we wouldn't need to bring paper into the classroom. I am happy to report, though, that for lower-importance assessments (in-class activities and low-point quizzes), I have been using an entirely digital workflow and loving it!

Collecting the exam

At the end of the hour, the link to the exam PDF turned off (although, as noted below, this precaution probably isn't necessary as every student, even ones with excused absences who would be taking make-up exams days later, had access to the exam and the ability to download it during the hour). Students e-mailed their annotated PDFs to me as attachments; my inbox filled up with PDFs. I told the students I would stick around after class for a few minutes if they wanted to verify that I received their e-mail attachments before shutting down their tablets.

Grading

As soon as I got back to my office after class, I downloaded each PDF and renamed each file with the last name and first initial of each student. This was partly for record-keeping purposes and mostly because many of the generic filenames given to the PDF by the tablet-based e-mail attachment process were identical. Last, I noticed that some students did not fill in their name at the top of the first page - they might have assumed that because it would be arriving from their e-mail account, they didn't need to. As soon as I downloaded the exam, I opened it to ensure that the PDF wasn't completely void of student entries; I then replied to the student's e-mail to tell them that I had received their exam. This is to alleviate student fears about whether the e-mail attachment and submission process worked.

The following class meeting, one student inquired about how I intended to pass the exams back. I admitted that I hadn't yet considered that aspect of the workflow. By now, I have. Here's what wound up happening and why I am definitely a convert to administering digital exams.

I rarely provide individual feedback on exams, because I provide an answer key and expect students to take the initiative to pursue with me any questions they have about their performance. So, having digital files instead of paper didn't help or hinder me at this point. I opted to use PDF annotation to provide the score each student received on each question. I added each question's score as a "sticky note" annotation and saved the PDF. At this point, I have not yet returned the exams (those students with excused absences have yet to take the make-up exam). However, I have already replied (again) to each of the student submission e-mails and attached the graded version of their exam - these were saved as draft e-mails. Right before the class period where I'll digitally "pass the exams back," I'll go through my Drafts box and send all of the e-mails.

As far as whether the tablet class fared better or worse than their traditional lecture counterparts: those data still need to be crunched.

Now for my Pros and Cons of digital exams.

Pros

• It is better for the environment than using print exams
• They're lighter to carry than a stack of print exams
• You can edit the exam up until you administer it (as opposed to having to have a print copy to photocopy at some point prior to the start of the exam)
• Students don't need to bring a calculator, or a pen, or a bubble sheet and pencil
• Most importantly, perhaps: exam responses are neater. Students have the ability to type out written responses, which makes it much easier to read; it is also easier for students to edit their written responses as they write, and they never run out of room. I didn't have to read any tiny writing meandering down the right margin, past all of the other questions on the page and then flipping over to the back of the page
• A big plus for the instructor: you have a digital copy of the student's work as it appeared when they turned it in. There have been many past circumstances where I've wished (after I returned them to students) that I had copies of student exams. That wish has been granted!

Cons

• There have been rare instances of some types of student PDF annotations not "taking" when PDFs are attached to e-mails. Only a couple of students had had issues with this during normal class periods; I've only had one student complain (days after the exam - I'm skeptical) about this.
• Tablet students accessing a PDF of the exam can distribute the exam to anybody (especially students in other sections of the same course that might not have taken the exam yet)
• Speaking of the potential for cheating…what about cheating?

Cheating

I fully realize that opportunities for cheating are rampant when students have access to the internet (Wikipedia. Their class notes. Chat with other students in the class!) And I have little power to detect it. So I gave them The Talk. We're all adults, and my perspective is that they get out of the class what they put into it. It is their responsibility to take advantage of the opportunity they have to demonstrate that they've learned the material. They're cheating themselves if they decide to cheat. Plus, because I don't grade on a curve, the only person that a cheater is impacting by cheating is her/himself.

Future Refinement

What will I do in the future to improve my tablet-based exams?

• Possibly using Qualtrics (web-based survey design tool) to write and administer exams, with the potential benefit that it isn't a PDF and can't be copied/saved/distributed to others easily, but with the drawback that question types are limited to the types of things you usually see in surveys (matching, multiple-choice, written response, ranking, etc.) When annotation becomes a reality, I'll definitely be on board.
• To address the potential for cheating, I certainly plan not to let the scientist in me continue to run the show. Instead of administering the traditional exam to the tablet class, my goal is to write tablet exams that obviate the potential for cheating. After all, I'm teaching an upper-division genetics class, and one of my goals for the course is to give students authentic experiences in genetics analysis. As I've told them, geneticists don't (all) work by themselves and certainly not without access to resources (manuscripts, colleagues, the internet, Wikipedia!) So why should I forbid access to notes, or Wikipedia, or fellow students, during an exam? What is required is development of an exam that is cheat-proof: an exam where there is not a single answer to the question; an exam where I could reasonably expect each student to provide a unique response regardless of whether a secret chat session is occurring in the background. Heck, letting students "cheat" in these ways might actually improve student learning and retention! I can proudly boast that I haven't cheated in years, because it is only where grades are involved that cheating normally exists; otherwise, it is called something like "making efficient use of resources." Let's develop exams that would make Woodrow Wilson proud of his famous quote, "I not only use all the brains that I have, but all I can borrow!" Let's make exams that help students synthesize and integrate information and demonstrate the depth of their understanding, rather than exams that can be cheated: ones that test memory and recall. I'll keep you informed of progress along this front as it is made!