Viewpoints discussion board | University of Illinois

National Coalition for Learning Outcomes Assesment

Entering content area for Viewpoints discussion board

blog posts

  • Why Student Learning Outcomes Assessment Is Key to the Future of MOOCs

    MOOCs are disrupting our educational frameworks, pushing our conventional boundaries, and challenging our traditional ways of thinking about teaching and learning—or so it would seem in much recent conversation about Massively Open Online Courses. As the potential “ultimate provider of educational access to all,” well-constructed MOOCs can expose students to a diverse assembly of knowledge and artifacts in a rich, engaging learning environment. Enabling twenty-first century learners to exercise more control over their own learning, in their own space, at a time they specify, is a very attractive proposition. MOOCs affirm the notion of students as lifelong learners, and in important ways they shift focus away from teachers and processes toward learners and outcomes.

    An opposing camp of educators, to be sure, is puzzled by the recent hype about MOOCs. They question the dearth of evidence of concern in the MOOC literature for concepts like pedagogy, curriculum quality, and academic integrity, not to mention the diverse learning styles of students that may impede such a self-directed learning process.

    With all of the regulatory and financial pressures on the higher education system, do MOOCs present an opportunity to re-envision how we deliver instruction, giving greater accessibility to all students, or do they merely represent a trendy approach, providing little or no assurance of quality student learning outcomes? There is little agreement on whether this disruptive innovation will morph into the status quo. What we know so far is that large numbers of individuals have embraced the MOOC idea, signing up in droves to enroll in free online courses offered by edX, Coursera, and Udacity, among others. Many of these students appear to be neophytes who gain a sense of empowerment as they experience a new approach to education and learning. Others are lifelong learners eager to attend a course instructed by a well-credentialed and purportedly famous professor.

    Even so, for us to be able to judge this new paradigm as a successful academic business model, the students who take these courses must demonstrate positive learning outcomes. Based on the current state of MOOC courses, much more progress is needed. Learning outcomes must be explicit, measurable, and effectively assessed, and evidence must be assembled demonstrating added educational value from the courses that will enhance students' lives, aspirations, and ambitions. Can the participating students reflect on what they learned in these courses, can they integrate the knowledge they obtained in them with what they gained in other courses, and can they apply their new knowledge in multiple settings (Kuh, 2012)? Without evidence of these outcomes, the effectiveness of MOOCs will remain in doubt.

    The following assessment issues need to be addressed for MOOCs to realize their promise.

    Peer Grading and Feedback

    Students in MOOCs often grade each other’s work according to the professor’s specifications, a sort of “calibrated peer review” that makes managing a large number of submissions possible. From a pedagogical perspective, this is a promising practice, as it extends the students’ learning experience by reinforcing concepts and broadening perspectives. In a class with tens of thousands of students enrolled through an open enrollment process, however, this practice can be problematic. First, many of these students are unprepared or not qualified to give meaningful feedback to one another. Gibbs (2012, a literature and mythology professor at The University of Oklahoma, is currently enrolled in a Coursera fantasy and science fiction course and has been blogging some of the struggles she and other students are facing with the course’s peer feedback feature: “[T]here is going to be a whole range of feedback, from the very zealous people who give feedback longer than the essay itself, to the ill-informed grammar police.” Watters (2012) also discusses problems associated with peer assessment grading such as the lack of feedback on peer feedback, the anonymity of feedback, and the lack of community in the course. Chuck Severance’s course on internet history offered through Coursera has recently removed the peer assessment requirement in recognition that it is not yet ready for primetime.  To ensure that the peer review process contributes successfully to students achieving their learning outcomes, rubrics that define criteria for performance levels should be made explicit.  If MOOCs are designed so that standards are available for both the grader and recipient of the grading, this will increase the likelihood that students will be equipped with the proper tools for peer review.   

    Plagiarism

    The Chronicle of Higher Education (Young, 2012) reported that students currently enrolled in MOOCs have related dozens of incidents of plagiarism. A University of Michigan professor who also teaches a free online course for Coursera has recently posted a plea to his 39,000 students to stop plagiarizing. Coursera’s leaders say they are reviewing the problem and are considering adding plagiarism-detection software to their system in the future. In what is described as their “first line of defense against plagiarism,” they have rolled out an honor code that students must accept when they first sign up for a course. As an additional step, reminders have been added in classes in which students must reaffirm the code before submitting any assignment. Plagiarism will always be a burning issue with MOOCs, as students may feel they have little to lose financially (the course is free) and academically (the home institution of the acclaimed professor teaching the course is not offering credit for the course). If the ultimate goal is for a student to receive academic credit by sending a completion certificate to his base institution, academic leaders will have to work hard to reign in plagiarism and to keep students honest.    To ensure that students’ work represent their own efforts and reflect the outcomes of their learning, plagiarism detection tools should be used to monitor student behavior in the MOOC environment. 

    Proctoring

    The MOOC providers edX and Udacity have recently introduced proctored final exams for their courses. Students at edX will be able to take on-site exams administered through the Pearson VUE service, which has more than 450 testing centers in more than 110 countries. Students who pass these exams will receive certificates noting the exams were proctored. Although edX has not yet disclosed the cost to students of these exams, Udacity’s proctored exams will cost $89. For MOOCs to be successful and credible in the future, proctoring will be important for students in the verification of their identities and to provide evidence that they are responsible for the achievement of their own learning outcomes.   

    Robograding

    Regarding current developments in grading student essays electronically, Bienstock (2012) points out that a human grader might be able to grade 30 essays in an hour while a robograder could grade 16,000 essays in 20 seconds. Incredibly efficient e-graders have obvious potential but need refinement and enhancement as we move forward. A simple Google search on “robograding” produces numerous stories, funny (and not so funny) anecdotes, and examples of how this technology often rewards students with incoherence, unintelligibility, and inarticulateness.   As the technology advances and is refined, professors and graduate assistants should be able to spend less time grading and more time interacting with students on activities like tutoring and advising to help them achieve learning outcomes.  While these technologies are being developed, human interaction will be necessary to accommodate students who do not readily learn from this format of instruction and feedback.  

    Establishing a Workable Business Model

    While the MOOC business model continues to evolve, some advocates assert the value of MOOCs in identifying highly capable students for recruitment either by colleges or by employers. Others believe that the cost of a college education can be substantially reduced if students complete many free MOOC courses that are accepted for credit by an accredited institution or that are recognized by a prospective employer as evidence of bona fide skill. In any of these scenarios, much progress is still needed in the realm of student learning outcomes and assessment. If the conversation about MOOCs can shift from technology to a thoughful assessment of what students know and are able to do, to pedagogy and the impact of differing  approaches, to some greater assurance of academic integrity as we move forward, the likelihood of success for the new MOOC paradigm will be greatly enhanced. The current infrastruture must support this conversation and guide higher education's response if MOOCs are to survive their "honeymoon" and enter the higher education mainstream.


    References:

    Bienstock, J. (2012, May 10). Grading essays: Human vs. Machine. [Web log comment]. Retrieved from: http://schoolsofthought.blogs.cnn.com/2012/05/10/grading-essays-human-vs-machine/

    Gibbs, L. (2012, August 7). Blogging my way through a MOOC. [Web log comment]. Retrieved from: http://courserafantasy.blogspot.com/2012/08/peer-feedback-good-bad-and-ugly.html

    Kuh, G. (2012, October). What matters to student success. Keynote address to the annual National Symposium on Student Retention, New Orleans, LA.

    Watters, A. (2012, August 27). The problems with peer grading in Coursera. Inside Higher Ed. Retrieved from: http://www.insidehighered.com/blogs/hack-higher-education/problems-peer-grading-coursera

    Young, J. (2012, August 16). Dozens of plagiarism incidens are reported in Coursera's free online courses. The Chronicle of Higher Education. Retrieved from: http://chronicle.com/article/Dozens-of-Plagiarism-Incidents/133697/