Viewpoints discussion board | University of Illinois

National Coalition for Learning Outcomes Assesment

Entering content area for Viewpoints discussion board

blog posts

  • What Assessment Personnel Need to Know About IRBs

    By: Curtis R. Naser, Ph.D.
    Associate Professor of Philosophy and Bioethics/Facilitator for Academic Assessment, Fairfield University
    Chief Product Officer and Founding Partner, Axiom Education

    Because assessment projects across all disciplines are now employing systematic research methods that include access to students’ confidential data and artifacts, faculty need to be cognizant of our obligation to protect human subjects in our research. Beyond simple compliance, we want to be respectful of students and to be sure we are acting ethically. By the same token, it is easy to misunderstand the policies and procedures of Institutional Review Boards (IRBs). What is the proper role of IRBs in student learning assessment?

    IRBs were federally mandated in response to particularly egregious research practices, like those used in the Milgram experiments and the projects at Tuskegee and Willowbrook that came to light in the 1960s and 1970s. Given the high risks of serious physical and psychological harm that can arise from improper research methods, the regulatory role of IRBs as gatekeepers to access human subjects is a justifiable burden for investigators. Furthermore, as a peer review process, these IRBs can assist investigators in strengthening the quality of their research.

     

    While providing protections, the federal regulations (45CFR Part 46) indicate that as the risk of harm to subjects decreases, so too does the need for IRB scrutiny. Research projects judged exempt from IRB review involve activities with minimal risk, such as evaluating the effectiveness of educational programs (46.101[b][1]), surveys, and other data collection methods that do not obtain information that could put the subjects at risk of social, legal, or economic harm (46.102[b][2]).

     

    By carving out these exemptions, the regulations recognize a wide domain of research activities for which the risks of harm are so low as not to warrant the intrusion of the IRB process as a regulatory burden necessary to protect human subjects. These regulations recognize that educators are already obligated by the norms of teacher-student confidentiality, now codified in the Family Educational Rights and Privacy Act (FERPA), and that additional scrutiny by an independent peer-review board imposes burdens on researchers that result in little or no additional protection of the students involved.

     

    The regulations do not stipulate, though, how research qualifies for exempt status. While in most institutions this decision is the IRB chair’s call, some IRBs are prone to “mission creep,” requesting a fully detailed protocol from the investigator even for exempt research. Some IRBs go so far as to require a full-board review of exempt protocols. This is grossly inappropriate, taking the IRB’s focus away from research that presents genuine risk and unnecessarily delaying research. In most cases, a simple one- or two-paragraph description of the project is sufficient for the IRB to determine exempt status, especially if the IRB does a good job of coaching the investigators on when exemption is appropriate and what the key factors are. The IRB’s goal should be to intrude as little as possible and to expedite a decision consistent with the protection of human subjects as defined by the regulations.

     

    Mission creep is a serious problem particularly with social science IRBs, which typically review minimal risk research and research in which the primary risk is the breach of confidentiality. Lacking a context in which serious physical and psychological harms could arise, such IRBs often magnify breach of confidentiality risks, generally defined in the regulations as “minimal” (46.102[i]). Unless the investigators are maintaining identifiable records linked to potentially harmful data if disclosed (e.g., illegal behavior, sexual history, etc.), the expectation should be that all involved in the research will abide by the basic protections of confidentiality that are already part of institutional practice. IRB review generally does not add any protection above this level.

     

    Unless specifically tasked by institutional policy to impose a greater level of scrutiny, IRBs should stick to the regulations, which in most cases represent a well-thought-out balance of intrusiveness against the need to protect human subjects. Indeed, the problem now is that the regulations are probably too intrusive in the domain of social science research, which doesn’t fit the biomedical model on which the regulations are founded. IRBs should not apply their review authority beyond that stipulated by the regulations unless the faculty and administration have specifically charged them with this responsibility, a move I would caution against.

     

    Faculty conducting assessment projects may be concerned that they will not be able to publish the results of their projects without IRB review, as many journals require. In all cases, a letter from the IRB stating that a project is exempt from IRB review should be sufficient. No journal can legitimately demand more than the regulations require of the IRB.

     

    A misconception also exists that a study is considered research—therefore, requiring IRB review—only if its results are to be published. Yet the word “publication” does not appear anywhere in the regulations, and the publication of research is not a criterion of IRB review. Most assessment projects do not qualify as research in the technical sense, as the regulations define it, i.e., producing generalizable knowledge (46.102[d]). Assessment projects are typically not controlled trials. The variations among higher education institutions in mission, learning outcomes, student population, culture, pedagogy, and curriculum are such that assessment results at one institution are generally not applicable to other institutions. What is applicable and what makes these activities a valuable part of the scholarship of teaching and learning are the assessment methods employed, not the reported results. Assessment is by and large an activity of internal quality assurance, not research—and, thus, it generally lies outside the scope of the IRB.

     

    Given that most assessment projects, for the reasons provided above, do not fall under the purview of the IRB, questions naturally arise as to the ethical obligations that faculty and administrators who are conducting these activities have to their students. Lacking the force of IRB regulatory authority, there is no specific requirement to seek informed consent. I would suggest that institutions think carefully about imposing burdens on assessment activities that may limit their usefulness or make them more difficult to carry out. In most assessment projects, requiring the informed consent of individual students is both impractical and severely limiting.

     

    Rather than requiring informed consent, institutions should be explicit with their students when they arrive on campus and throughout their education that the institution is obligated to carry out continuous quality improvement, that it will use students’ work and data (including grades, when necessary) to assess program effectiveness, and that it will protect student confidentiality in these assessment activities. As the beneficiaries of assessment activities that involved past students, current students have a general obligation to participate in further such investigations for the sake of students that follow them. We can respect student autonomy and protect particularly sensitive information by allowing students to opt out when specific artifacts may be used for assessment.

     

    While removing identifiers from student artifacts is one way to ensure confidentiality and, in some cases, prevent evaluator bias, this process is time consuming and has the unfortunate effect of limiting the linkage of these artifacts to other student data once the identifiers have been removed. There is no regulatory requirement, either in FERPA or in the IRB regulations, that requires anonymity of data or artifacts. In most cases, there is little risk of harm to the students in the use of identifiable artifacts if the faculty and administrators involved live up to their normal obligations to protect student confidentiality. Because assessment activities themselves generally do not introduce any further risks, the burden of removing identifiers is disproportionate to the incremental protection it offers.

     

    While the IRB regulatory model is very effective in protecting human subjects from serious harm that can result in biomedical and some behavioral research, it is not well adapted to social science research. Mercifully, the model exempts most educational research, based on a considered judgment of the relative risks and the burdens of IRB review. Assessment activities in higher education generally do not rise to the level of risk that would justify IRB intrusion.

     

    One quite valuable aspect of the IRB process is the peer review it brings to the investigator. Far better than involving the IRB in assessment, however, is an assessment office creating its own peer-review culture. With a range of assessment plans and reports now in use at most institutions, effective models exist for the peer review of assessment practices using experienced faculty as evaluators. Let’s create our own models of peer review and leave IRBs to do what they do best.

     

    Resources

     

    Beecher, H. K. (1966). Ethics and clinical research. The New England Journal of Medicine 274(24), 1354–1360. Available from the Office for Human Research Protections at http://www.hhs.gov/ohrp/archive/documents/BeecherArticle.pdf

     

    Jones, J. H. (1981). Bad blood: The Tuskegee syphilis experiment. New York, NY: Free Press.

     

    Levine, R. J. (1988). Ethics and regulation of clinical research. New Haven, CT: Yale University Press.

     

    National Commission for the Protection of Human Subjects of Biomedical and Behavior Research, U.S. Department of Health and Human Services. An archive of the commission’s reports is available from the Office for Human Research Protections at http://www.hhs.gov/ohrp/archive/nationalcommission.html

     

    Office for Human Research Protections, U.S. Department of Health and Human Services. The office website is at http://www.hhs.gov/ohrp/index.html

     

    Schrag, Z. M. (2010). Ethical imperialism: Institutional review boards and the social sciences, 1965–2009. Baltimore, MD: Johns Hopkins University Press.

     

    U.S. Congress. (2009, January 15). Code of federal regulations. Title 45. Public welfare. Department of Health and Human Services. Part 46. Protection of human subjects Available at http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html