Google Classroom for anonymous student feedback

As I've mentioned, one of the reasons I feel that having computers in the classroom is exceptionally useful is for collecting student feedback in real time during class (e.g. using Socrative, Twitter). However, both of these tools have a shortcoming: they tend only to excel at collecting typed responses (True/False, multiple-choice, written response). With my discipline being one that also involves diagrams and visual representations, I have been dreaming of a way to rapidly collect drawn responses from students during class as a method of formative assessment: to help me understand what concepts need more practice time.

Past Failure

I've written previously on how I've started to use Google Classroom to assign and distribute exercises to students in my tablet course on genetics. My first use failed at what I was trying to accomplish: to send one exercise requiring students to draw annotations and then return them to me anonymously. My plan was to then open a few representative images and display them on the projector, in class on the same day, to engage in some group critique and analysis. However, I discovered that setting an Assignment using Google Classroom, in which I provide a PDF and have Google Classroom make a copy of that PDF and deliver it to each student registered in my Google Class, doesn't work anonymously. When the PDF containing the image I want students to manipulate is copied to each student, Google Classroom adds student names to each filename.

Goal

This is not ideal for my goal, because I'd like to be able to open my Google Classroom folder on my computer, browse through the previews to find student PDF submissions representing useful points to discuss, and then open them for projection to the class. However, if I do this in front of the class, everybody sees the student names in the filenames, and so I've lost the anonymity. Sure, I could turn off the projector (or switch inputs) while I look through student submissions, but this seems like a cumbersome process. Fortunately, a relatively easy solution is at hand: don't add the PDF activity to the Google Classroom Assignment!

1. Set up a new Assignment, but do not attach the file here
2. Perhaps distribute that file using another method
3. Ask each student to attach his/her final annotated version to the Google Assignment when they Turn it In

Present Success

For example, tomorrow in class we start our section on Pedigree Analysis. To ensure that students have read the part of the textbook on the format of a pedigree (and to make the topic relevant), I will ask each student to draw the pedigree of his/her immediate family and send it to me. Now, clearly, this is not a situation in which I want any identifying information from a student to be delivered to me with their pedigree drawing (I don't want to break HIPAA or other rules, of course). So, this is what I am doing:

I used the "+" button in the lower-right to add an Assignment, and here is the window with the assignment details. I'm not providing any document (there are no attachments, which I would add with the paperclip button). Instead, I'm just asking students to use whatever drawing program they have installed on their tablets to draw the pedigree and attach it to this assignment. Then, students attach their drawings to this Assignment and submit it via Google Classroom. When I open my Google Drive folder for this Classroom Assignment, I'll see something like this:

where the file names contain no identifying information. I can easily browse through this folder (while projected to the class) and open student responses anonymously to look for points to praise and points to critique.

If I want to know which student submitted each file, I can still do that, by looking at each student's record on the Google Classroom website, but I do not need to project this view during class simply to look at the attached submissions.

Conclusion

Google Classroom is now an integral part of the suite of apps I use (including todaysmeet.com, Twitter, and Socrative) to foster an active-learning and engaging medium-enrollment course (75 at present) in which students hopefully feel like they can obtain personalized feedback on their understanding and also engage in peer evaluation.

Flipped Classroom: Switched On (v2)

Take blended learning, add computers, and you get my next-generation (v2) flipped classroom. This means an enhanced ability to:

• interact more efficiently with students in large courses
• engage students in authentic practices in the discipline (assuming your discipline, like mine, relies heavily on the internet these days)
• practice information literacy skills
• collaborate with peers

Today, I share how the typical structure of my DISCOVERe (tablet-based instruction) courses has evolved into an experience that integrates the above points into a class where the tablets are not the only things that are switched on - so are the students!

0. Before students come to class
A reading assignment from a textbook is due to be completed by the start of class; I often also post links to videos I've recorded (or I've found) that introduce the topic for the day. This is the "lecture" - where factual information and approaches to solving problems are conveyed to the students.

1. (5 m.) While you're waiting…

While students are assembling in the classroom (and during the first five minutes of class), I have a projected slide that contains a problem to solve or a question to think about, as well as (often) the link to a Socrative quiz that the students should take. This entry quiz is always based on the pre-class video/reading assignment, to ensure compliance. I use the on-the-fly Socrative results to determine what concepts I need to spend more time in class discussing with them. Having never use this approach previously, I'm routinely surprised at which relatively complex concepts the students seem to already understand, and which relatively simple (to me) concepts the majority of the class struggles with!

I like "While you're waiting…" questions, because it gives students who desire more practice with material an optional opportunity to do so. As the term has progressed, I've seen more and more students actually taking the time to work on these practice questions as they wait for class to start.

2. (10 m.) Anonymous review of student exercise submissions (from previous class)
In the latter half of the previous class, I introduced a topic and set the students a rather difficult (at the time) problem to solve - one that they should be able to get at least part way into before encountering some issues that they haven't had to deal with before. The students work solo, and then in small groups, and then send me their work (digitally) by the end of that previous course period. This  approach does two things for me:

1. It provides students context for the reading (and/or video-watching) assignment due for the following class (i.e. today's class). This is the typical blended learning approach: ask the students to attempt something with little essential background; they fail at it. Then, when they go encounter the reading assignment relevant to that problem, they are (hopefully) a bit more invested and have that mental "hook" to identify where they went wrong during their initial attempt. This, ultimately, leads to better-formulated questions in the following class meeting (today) when we discuss that topic.
2. This gives me time to look over the work and assess, before today's class, where the class stands in terms of common themes: where they succeeded and where they had issues.

During this portion of today's class, I show some anonymous examples of student submissions and ask the students to perform some peer evaluation: "Does anybody see something in this approach that you like? That you would change?" I also use this time to point out, with praise, patterns in thinking that I've observed that show strong understanding of difficult concepts, even if the answer ultimately was not obtained (or was wrong).

Part of the reason I like group peer evaluation is that it helps me "grade" the student answers to the questions in class, so that all students get some hopefully-pertinent feedback on how to accomplish the exercise, even if it is not their work that is being specifically critiqued. This also potentially means that the instructor does not have to formally grade and return all of the student submissions - the students gain the benefit of having worked on the problem, and then they benefit from time in class to discuss any issues they encountered.

After we work through the solution to the question or problem and address questions, we move on to:

3. (10 m.) More practice; Q&A
in which I provide another, often just marginally related problem, for students to work on solo and then in small groups to practice the concepts just discussed. We also usually discuss the answer to the "While you're waiting…" question from the start of class at this point.

Other types of activities we do in class together involve learning to access and use online resources. For example, last Friday, as I was discussing types of mutations, we discussed how mutations in BRCA1 predispose to the development of breast cancer. A really fabulous question from a student was, "If the BRCA1 gene is involved in DNA repair in all of our cells, why don't BRCA1 mutations predispose to other types of cancer - like skin cancer, for example - as well?" So, today in class, I spent some time online showing students how I would go about learning how to answer that question. We went to the Online Mendelian Inheritance in Man website (omim.org) to learn about what's know about the genetic basis of breast cancer, and found some primary literature references indicating, for example, that male carriers of BRCA1 mutations might have an increased risk of developing prostate cancer. This lead to a great (but brief) discussion to the extent that we don't know enough about the BRCA1 gene, and all of its potential effects, yet! (Tangentially, this also raises the important point that gene names are not always accurate, nor do they always (ever?) reflect the true complexity of biology).

4. (5 m.) New topic
Having now completed the instruction on the previous topic, it is time to introduce the next topic (which builds on the previous topic), often with a story or scenario that sets up a new topic. I try to do this by being provocative, or by posing a puzzle that has no obvious immediate solution.

5. (15 m.) Try, and fail
Now with five minutes worth of "mini-lecture" by me on the new topic, students attempt to solve a problem alone, and then in small groups. Then, they electronically submit their work to me (usually as a screen shot of hand-written work on their tablets). For example:

Again, the idea behind this approach is to have students partly invested in figuring out how to solve this problem before they first encounter the relevant material (on their own, through the reading/video assignment for next class). The work they submit is what we will spend the first portion of our next class meeting anonymously evaluating and discussing whether the reading/video assignments led to any new insights about how to address the question.

6. (5 m.) Exit Survey
At the end of most classes, I provide the opportunity for students to reflect on the class period. This is often a Socrative quiz that asks questions like, "What did you learn today that you didn't know before?" or "What topic should we spend more time discussing next class?" or "How does what you learned today apply to your life?"

Summary (and a word of caution)

It takes quite a bit of mental effort to keep up this sort of course design, where, essentially, the first half of each class meeting is about material initially encountered during the previous class, and the second half of each class period is spent introducing students to concepts to be continued in the following class! Staying organized, and planning activities and pre-lecture work (e.g. which reading assignments and/or videos to watch) in advance, are key to success with this approach. To many, this is an unfamiliar structure, and it can take some significant mental gymnastics to appreciate this approach at first. The rewards will come!

Ultimately, the knowledge that all students in class have mobile devices that they can use to annotate PDFs, draw images, complete Socrative quizzes, seek information on the internet, and send and share their work with others (peers and the instructor) can create a classroom that is much more active and "switched on" – an environment where learning from one another takes place! In conclusion, in my version of the tablet classroom, the main benefits of the tablet in the classroom are two:

1. I am assured that every student has access to a computer (and to the internet) - this facilitates actual student involvement in analyses that I would traditionally just describe while projecting screen shots, for example
2. It improves the efficiency of submitting/exchanging/evaluating ideas and assessing performance. It simultaneously provides anonymity to those who wish it, and thus grants a voice to those who might not otherwise speak up in class

Cheat-Proofing Exams III: Post-exam analysis

Now with two open-internet, tablet-based exams under our belts in Genetics class this term (each combining an individual exam and a group exam), it is time to briefly address concerns about holding exams where students have access to all data, everywhere. How (and what) do the students do?

Let's look at my most recent grade distribution (individual and group exam scores combined):

My students seem to be doing outstandingly well (but too well?), and we could (but won't here) enter in the discussion about whether an ideal grade distribution (like the bell curve) exists and should be sought.

There are three points to keep in mind about the student outcomes thus far:

First, I should point out that my letter grade schema is different than most:

100-80% = A
60-80% = B
40-60% = C
20-40% = D
0-20% = F

If I had used a more traditional 10% letter grade division (which I have in every previous class), I'd have a much more bell-shaped curve.

Second: of course the instructor can craft the exams to be as difficult as s/he wishes. After both exams so far this term, the vast majority of students have reported that they accessed electronic resources (notes, the internet) less or much less than they had expected, mostly because they didn't feel that they had time during the exam. From this perspective, possible conclusions might be that access to electronic resources:
1) might only be an advantage to the top students, who already excel at test-taking and perhaps the course material, offering only them the ability to double-check answers they're unsure of
2) doesn't impact grades, because few (if any) students use the available resource

In either case, I feel like I'm winning: either I'm reinforcing the concept of double-checking one's work (a good practice for anybody to use) and improving metacognition (students aren't checking work they're reasonably sure they knew the answer on), or my tests are long and/or hard enough that students generally realize there is no benefit to spending time looking up notes.

I can also report that, from my own observations during the tests, I rarely saw students using their tablets to access resources. When I did, most of the students were looking back at lecture slides and notes.

Third: because I'm a scientist and need something to measure, I looked at whether there is a statistical correlation between student test grades and the degree to which students self-reported accessing notes during the exam (as expected vs less than expected). If "cheating" were an issue, then I might predict that students who used digital resources as expected would have higher scores than those who felt like they had no time to use those resources. However, a t-test produces p = 0.95. That is, there is absolutely no difference in the distribution of grades between the group that used digital resources as expected vs. those who used them less than they thought they would. My conclusion? Written thoughtfully (specifically: incorporating high-level Bloom's taxonomy questions), open-internet exams don't necessarily lead to cheating.

In sum, my current take on developing cheat-proof exams is:

1. Give students the opportunity to use the resources they are used to using (textbook, notes, the internet, whatever) - otherwise, the exam situation is totally unlike what they'll face anywhere else (in grad school, as an employee, as a citizen…)
2. Incorporate higher-level Bloom's questions that really just inherently can't be cheated (or, if they were, it would be obvious to the instructor - I'm thinking plagiarism on written response answers, here)
3. Make the exams deliberately long, and grade consistently but harshly.
4. Embrace collaborative group work. I feel that I can ask much more difficult questions of a group than I would of an individual student, and I have been pleasantly surprised at the results.

As an example of this fourth point: on my recent test, I wrote a group exam where the group had to select two of three questions to answer. One of the questions was something that we had never practiced in class, requiring the group to use PubMed to access the abstract of a published journal article and analyze some data found in it. To my great surprise, most of the groups chose to answer the novel question over either of the other two (involving question formats we had more directly practiced in class), and most (if not all) groups arrived at the correct interpretations.

If you need more convincing about the qualities of group exams, check out this video from one of my exams, comparing the class during an individual portion and group portion. From what I've read about the quality of learning based on peer instruction, I am comfortable seeing that students are much more engaged (and probably learning more!) during the group exam than the individual exam. What a better use of class time!

Finally, I'll reiterate an important point about digital exams: the workflow is more efficient in some regards; less so in others. By distributing the PDF exam using Google Classroom, each student has their own version of the PDF file, which automatically includes the student name in the filename. All of the PDFs are returned to me in a single folder on Google Drive, meaning I don't have to download and rename e-mail attachments from 75 students. Most importantly to me, digital exams means I always have a copy of each student's exam, even after I return a scored copy to each student. This can be useful in all sorts of ways (including programmatic assessment, detecting patterns that might reveal cheating…assuming one is worried about such things, and formative assessment for the instructor).

Now, if I could only figure out how to more efficiently (and still privately) return PDF exams with my annotations (scores, notes) to students without writing 75 individual e-mails…

Be Bold: Tell the Truth

A fairly frequent comment I hear when I tell colleagues that I record and distribute my lectures to my students is, "Oh, I'd never do that - I don't want students to have recordings of any mistakes I make."

As a tenure-track professor, I suppose I'm pretty relaxed (more than I should be?) about making mistakes. I've been an honest person my entire life, which is probably much like the statistic that a majority of those polled report that they're above-average drivers. But, the origin of my fanaticism about telling the truth in the classroom is much more recent than my own origin. I was caught off guard in my first semester as a grad student when the professor (Dr. Mark Roth - I remember it clearly) told us that he believed in truth in teaching, and that, at the very least, if he ever lied to us, he'd tell us that he was lying. This was something I had never heard, nor ever expected to hear, a professor say. And therein lies the power.

It is important for students to know why we're the professor. On the first day of class, I always (now) tell my students why I'm the one standing at the front of the room. It is definitely not that it is because I am an expert in genetics - I have some expertise in particular aspects of genetics, yes, but I'm still learning. In fact, my credentials (a Ph.D.) suggest that I'm sufficiently accomplished at the art of learning that I should be able to lead others down the path of learning. In other words, I'm the mentor because I'm farther down the path of understanding the practice of genetics. This is critical to say, because it sets the stage for what should come next: admitting, openly and honestly, that you don't know everything about your discipline.

This facilitates (at least) three critical aspects of learning:

1. It provides the opportunity to model how to learn new things (or to change one's mind)
2. It helps establish an atmosphere where students might be less intimidated by the all-knowing professor
3. It makes it much easier to make the point (which I regularly do in biology courses) that there is not usually one correct answer. Even in the sciences, there are nuances and exceptions to rules, and one can only very rarely (if ever) make statements as "this is always, in every case, how this works."

Today, I'll focus on the first, as it pertains to having one's mistakes recorded and posted online. Now, why is verbal deceit (lying) an issue in the academic classroom? The painter Pablo Picasso had it nailed: "You mustn’t always believe what I say. Questions tempt you to tell lies, particularly when there is no answer." Now, I teach in a Socrative fashion (my father, Bob Ross - no, not the painter - put me on to a goal of mine, to somebody teach my classes by only asking questions). And I strongly encourage my students to think creatively and to practice science by being inquisitive and skeptical. So, I'm routinely barraged with excellent and relevant questions in class; I rarely know the answers. Being the holder of a doctoral degree and the instructor of the course, sometimes I'm tempted to tell half-truths so that we can move on with the orders of the day and not be sidetracked.  And there's some inherent pressure to maintain alpha status in the classroom and maintain control by always knowing the answer. But we're (students AND faculty) all adults, and we deserve the truth. So, when I'm asked a question I don't know the answer to, one of two things will happen:

1. I say that I don't know, I give (and justify) my best guess, and I tell the student I'll look into the answer and report back to the class (and I actually do so, because telling the truth about not knowing the answer, and then lying about looking into it, really isn't good practice)
2. I demonstrate, then and there, how I would find an answer (rather than the first approach, where my learning takes places in secret, behind closed doors)

I prefer the second approach, especially because it is the approach preferred in a blended learning setting.

So, what does this have to do with tablet pedagogy? As I said, all of my lectures are recorded and posted online almost as soon as we leave the classroom. I mentioned in an earlier post this semester that I'm also (now) regularly reviewing the videos to produces Tables of Contents to accompany the videos on YouTube. And that process of regularly reviewing my lectures means that I occasionally discover that I make mistakes during class.

How to correct mistakes that have been distributed online

Here's the three-step plan:

1. Post an immediate correction to your students (on your Learning Management System, via e-mail, however you like). If a student notified me of the error, I definitely mention that one of the students from class caught the error.
2. Add a similar comment on your lecture video webpage. Don't take down the video - you have nothing to hide other than not being perfect.
3. Mention this again at the start of the next class, solicit questions, apologize for any confusion your error might have added, and (most importantly of all): seize the teaching moment, if you have figured out how to explain how you went astray and what the correct approach/answer/strategy/etc. is - and why.

What happens next? Some combination of the following:

1. Students respect that you tell the truth and realize that you're just human
2. Students lose trust in your content knowledge
3. Students have observed an expert modeling how one's mind can be changed (thanks José Bowen for pointing out this important need) and how to respond to criticism
4. Students become horribly confused about whether the first polar body divides during oogenesis, and forever after will have no idea whether three or four haploid cells are produced (as a random example…of course…)
5. Students learn that you respect them and care about the quality of their education

Hopefully, if you rank the relative importance of those five possibilities, the likelihood of doing good by telling the truth in the classroom vastly outweighs potential costs. Although in the back of my mind, I'm sure I'll always be wishing I had produced more students who really deeply truly appreciated mastering #4, it certainly doesn't keep me up at night like it would if I hadn't taken the opportunity to #3 and #5.

So, keep recording and posting those lectures without trepidation, review them regularly, and own up to any errors. Model being a lifelong learner and a decent and bold human being: how to identify mistakes and how to properly address them.