In this post I reflect on the second ISTE Standards for Teachers. I use a four-step process. First, I pose a triggering question; second, I explore various resources and research the topic in an effort to answer my triggering question; third, I integrate suggestions from my DEL cohort; finally, I seek a resolution–at least an attempt. Feel free to leave feedback in the comment section or find me on Twitter.
ISTE Standards for Teachers 2: Design and Develop Digital Age Learning Experiences and Assessments
This is the second standard for teachers and includes recommendations for teachers to create an environment that promotes learning while using appropriate tools.
How can teachers design, develop, and evaluate authentic learning experiences and assessments incorporating contemporary tools and resources to maximize content learning in context and to develop the knowledge, skills, and attitudes identified in the ISTE Standards for Students?
How can university educators implement ongoing formative assessment process with their students that uses technology to empower timely feedback and enable the educator to enact knowledge gained and make changes to that course (thinking of how quickly a 10-week course goes by)?
Without the use of digital tools formative assessment can be delayed between class sessions. For example, a graduate course, in the quarter system, meets once per week consecutively for a total of 10 weeks. During the first session, week one, students get an overview of course requirements and learn their about their first assignment which is due week two. At the week two session students turn in their assignment and the professor uses week two class time to lecture new content. During the next week students complete another assignment that is due at the week three class session, which is the same day the professor returns the assessed week two assignment. The cycle continues throughout the quarter. In this example, a professor is unable to change the class session to meet the needs of the students based on the assignment and students are unable to make changes to their weekly assignments until after week 3 when their first assignment is returned from the professor.
To answer my triggering question I investigated Turnitin and Google Drive’s real-time productivity tools. My question is based around the idea of reducing feedback loop time and creating helpful formative feedback process in face-to-face graduate courses that either meet once per week or less during a 10-week quarter.
One helpful example is how Simpson (2012), a librarian, implemented Google Drive in her library instruction sessions that are usually offered once per quarter. Simpson lamented, “Even with the ‘right’ assessment tool, changes that I make to my instruction sessions will only affect the next class in the next semester” (528). She continues, “It is almost always too late for the students giving the feedback to benefit from the changes” (528, italics original). This experience is very similar to what may happen in a graduate course during a 10-week quarter. To provide formative assessment in the sessions, Simpson creates a Google Sheet with public access enabling students to use their mobile devices during the class session to enter relevant information about their sources. During the class session, Simpson views real-time work on the shared Google Sheet and makes adjustments and comments in the Google Sheet as necessary. Furthermore, Simpson delegates some work to the professor which frees her up to adjust the class session on the fly, accommodating and responding to what she sees students struggling with. She writes, “I can watch the student results and re-direct or re-explain when it’s clear that multiple students are not understanding a concept, or I’ve simply missed an important point” (530).
Another helpful online tool is Turnitin. Unlike Google Drive tools, Turnitin requires a subscription and is usually implemented into a university’s learning management system (LMS). Like many online-based educational technology tools, Turnitin quickly updates features, improves reliability on multiple devices, and increases throughput capacity to decrease system lag. These are important considerations when evaluating research from a year or more ago. However, 2013 research from a UK university indicates Turnitin saves professor time and provides quicker turnaround to students which supports formative assessment (Buckley & Cowap). The online service allows students to submit their assignments electronically and, based on the professor’s desire, screens the documents for originality and plagiarism. The professor is given an originality report and, if plagiarism is detected, includes source information. The professor may use online commenting through GradeMark to provide text comments, pre-defined stamps, and audio feedback directly on the assignment. When finished, the professor returns the assignment electronically to the student.
Because this process doesn’t require the physical submission of turning in an assignment the week lapse in my graduate course example is reduced to literally the time it takes the professor to get through the assignment. Therefore, the professor may collect the assignments the night before the week 2 class, quickly skim (or fully grade) the assignments before week 2 class, and make necessary adjustments to that class–saving a whole week. The research indicates time saved on the professor’s behalf and increased flexibility of when to grade the assignments. One focus group member says, “Yeah I think overall I was quicker” another comments on the flexibility, “I like the online but I think, like you were saying it’s nice to be able to access it when you get home” (567).
The largest concern is how to train professors to use the software and integrate into their workflow without a huge loss of time. The following comment from Buckley and Cowap (2013) highlights the importance of contextual training and not relying on the training material provided by the online service.
Staff were also very positive towards the bespoke training they received, praising the specific nature of it which provided contextualised training directly relevant to how they would be using the software. A number of staff had already attended general training in the use of the Turnitin system and had found it unhelpful as it was so generic. This is an important finding as future staff training can employ such methods to enable the successful training and engagement of staff (567).
In addition, software changes quickly so ongoing training is crucial.
Finally, the goal is to provide formative assessment quickly to students and adapt the class session time appropriately. These are two online tools, but simply starting the process of formative assessment, if it doesn’t exist at all, should be the initial focus. Maybe that begins no tech or low-tech such as creating a shared Google Doc for students to pass their ideas in class, much like Holly does with her English class (Gullen & Zimmerman, 2013, 65).
In my concerns/questions section, above, I expressed concern about how to train professors to use formative assessment. Prompted by feedback by my professor, @RobinHenrikson, I decided to explore this area more. In particular, I wonder how a center for teaching and learning at a university might motivate faculty to see the importance of formative assessment and implement a contextual and applicable training process where faculty learn theoretical components behind formative assessment and learn practical steps for implementing strategies into their learning environments. The main point I want to make: Just because there might be wonderful technology tools such as web apps, mobile device apps, or hardware devices that are well branded and provide amazing features doesn’t mean faculty will implement best processes for formative assessment nor will students necessarily glean the necessary information to provide essential feedback. Faculty must know what formative assessment is, understand how to use, and why it may be helpful to their learning environment. To that end, I researched more about about formative assessment.
Black & Wiliam (2009) provide a concise yet helpful definition of formative assessment, “Formative assessment is concerned with the creation of, and capitalization upon, ‘moments of contingency’ in instruction for the purpose of the regulation of learning processes” (10). Their concept, “moments of contingency,” can provide a helpful foundation for professors to understand the importance of formative assessment. Black & Wiliam provide practical insights on what these moments may look like in both synchronous and asynchronous learning environments. For example, in a classroom they may look like instant changes to how the professor is teaching based on feedback from students in a one-to-one situation or based on feedback from the entire class (10). Furthermore, for non-live situations the feedback that professors provide to the students on assignments or the students using a reflective assessment strategy where the professor requires students to submit an exit tick at the end of the class session to share feedback and the professor will read over and adapt for the next class session, or, if necessary, send out more information electronically via email or learning management system (10-11). These examples may seem basic and professors may already being using them; however, providing foundational education on formative assessment is an important starting point. The core is understanding the process. Black & Wiliam (2009) summarize it well as a “formative interaction”:
A formative interaction is one in which an interactive situation influences cognition, i.e., it is an interaction between external stimulus and feedback, and internal production by the individual learner…The teacher addresses to the learner a task, perhaps in the form of a question, the learner responds to this, and the teacher then composes a further intervention, in the light of that response (11).
Another area to consider is the perceptions of students when professors use formative feedback. Essentially, it is wise to consider how formative assessment is used and solicit feedback. Price, Handley, Millar, & O’Donovan (2010) researched three business schools at UK universities to better understand what assumptions students and teachers make. The research is enlightening and promotes better formative assessment implementation. There are two main takeaways from their research worth discussing in this blog post.
First, students want more discussion and dialogue in their feedback. It shouldn’t be a one-way street without turnout lanes. For example one teacher responded, “How does telling students to be more analytical help them acquire the skill?” (284). Students were frustrated with the lack of intentional conversation about the feedback and, based on the research, the authors noted a couple examples were the teachers were unwilling to meet with the students and essentially caused frustration on the part of the students. (Price, Handley, Millar, & O’Donovan, 2010). The authors summarized this part of their research writing, “Students indicated their hunger for more opportunities to have a dialogue with staff (284). It is all about the relationship between the student and professor and that “is at the heart of a successful feedback process” (285).
Second, the feedback must be genuine and authentic. This is particularly a concern as we explore the plethora of web-based apps available to mediate the assessment process. The tickbox nature of quick assessment was seen to some students as unhelpful. The authors write, “Unfortunately, students often very keenly felt (perhaps wrongly) that staff did not care enough to spend time on the feedback, particularly where tick box feedback sheets had been used which students regarded as ‘an insult’” (282). This goes back to the the first point, the need for dialogue and conversation–it’s about the relationship!
Using formative assessment is not just about using Google Forms or a public Google Docs document to solicit feedback from students. Nor is it about using an exit ticket at the end of class to prepare for the next session. Formative assessment is about the relationship between student and professor in which a line of open conversation is established where dialogue and flexibility can flourish. There are so many technology tools that flood the market every week, literally, but if we neglect helping professors learn how to use formative assessment well then our fancy apps are useless. The research from Price, Handley, Millar, & O’Donovan (2010) reveals the need to focus on the relationship and to provide authentic feedback where students can engage in a loop with the professor for more clarification. Equipped with a deep understanding of what formative assessment is, why it is helpful, how it may change a classroom environment from session to session (or even minute-to-minute), and a willingness to focus on the relationships with students create a strong foundation for implementing formative assessment. After a professor understands theses areas and has implemented them in her class, we can discuss finding the best (or multiple) apps to mediate the feedback process.
As with all my posts I encourage you to share feedback about what I wrote. This post was a new territory for me and I am excited about how much more there is to learn about formative assessment in higher education and how universities might create a training program to equip their faculty.
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education), 21(1), 5–31. http://doi.org/10.1007/s11092-008-9068-5
Buckley, E., & Cowap, L. (2013). An evaluation of the use of Turnitin for electronic submission and marking and as a formative feedback tool from an educator’s perspective. British Journal of Educational Technology, 44(4), 562–570. http://doi.org/10.1111/bjet.12054
Gullen, K., & Zimmerman, H. (2013). Saving Time with Technology. Educational Leadership, 70(6), 63–66.
Price, M., Handley, K., Millar, J., & O’Donovan, B. (2010). Feedback : all that effort, but what is the effect? Assessment & Evaluation in Higher Education, 35(3), 277–289. http://doi.org/10.1080/02602930903541007
Simpson, S. R. (2012). Google Spreadsheets and real-time assessment Instant feedback for library instruction. College & Research Libraries News, 73(9), 528–549.