POST 88: EVALUATION FEB14

After Sundays weekend session, I wanted to provide evidence of my understanding of the requirements for Quality in Educational research.

  1. Relevance:

To what extent the results achieve the aims.

  1. Reliability:

In qualitative studies, to what extent do results match the context.

In quantitative studies, to what extent can results be repeated without measurement error.

The question here is what level of feedback is appropriate during the study to remain reliable.

  1. Accuracy/validity:

Integrity of the conclusion in reference to the aims.

  1. Ethics:

What is the role of the researcher?           Is there impact from the researcher’s’ intent?       Who is the research intended for?            What is the purpose of the research?

The aim may exclude, cause conflict of care or be prejudice to stakeholders.

Summary of some ethical principals that various codes address*:

Honesty

Strive for honesty in all scientific communications. Honestly report data, results, methods and procedures, and publication status. Do not fabricate, falsify, or misrepresent data. Do not deceive colleagues, research sponsors, or the public.

Objectivity

Strive to avoid bias in experimental design, data analysis, data interpretation, peer review, personnel decisions, grant writing, expert testimony, and other aspects of research where objectivity is expected or required. Avoid or minimize bias or self-deception. Disclose personal or financial interests that may affect research.

Integrity

Keep your promises and agreements; act with sincerity; strive for consistency of thought and action.

Carefulness

Avoid careless errors and negligence; carefully and critically examine your own work and the work of your peers. Keep good records of research activities, such as data collection, research design, and correspondence with agencies or journals.

Openness

Share data, results, ideas, tools, resources. Be open to criticism and new ideas.

Respect for Intellectual Property

Honor patents, copyrights, and other forms of intellectual property. Do not use unpublished data, methods, or results without permission. Give proper acknowledgement or credit for all contributions to research. Never plagiarize.

Confidentiality

Protect confidential communications, such as papers or grants submitted for publication, personnel records, trade or military secrets, and patient records.

Responsible Publication

Publish in order to advance research and scholarship, not to advance just your own career. Avoid wasteful and duplicative publication.

Responsible Mentoring

Help to educate, mentor, and advise students. Promote their welfare and allow them to make their own decisions.

Respect for colleagues

Respect your colleagues and treat them fairly.

Social Responsibility

Strive to promote social good and prevent or mitigate social harms through research, public education, and advocacy.

Non-Discrimination

Avoid discrimination against colleagues or students on the basis of sex, race, ethnicity, or other factors not related to scientific competence and integrity.

Competence

Maintain and improve your own professional competence and expertise through lifelong education and learning; take steps to promote competence in science as a whole.

Legality

Know and obey relevant laws and institutional and governmental policies.

Animal Care

Show proper respect and care for animals when using them in research. Do not conduct unnecessary or poorly designed animal experiments.

Human Subjects Protection

When conducting research on human subjects, minimize harms and risks and maximize benefits; respect human dignity, privacy, and autonomy; take special precautions with vulnerable populations; and strive to distribute the benefits and burdens of research fairly.

* Adapted from Shamoo A and Resnik D. 2015. Responsible Conduct of Research, 3rd ed. (New York: Oxford University Press).

A Guide to Ethical Issues and Action Research[1] JANE ZENI University of Missouri–St Louis, USA

INSIDE AND OUTSIDE VIEW

Traditional ‘human subjects’ reviews may not address the ethical issues that arise when practitioners study their own contexts.

Guidelines for the outsider doing a classic experiment (random selection, control groups, removing the personal influence of the researcher) are either irrelevant or problematic for the teacher investigating in their classroom/studio.

Do not apply  here

In the same way, guidelines for the outsider doing qualitative research (anonymous informants, disguised settings) may subvert the value placed by ‘insider’ research on open communication with colleagues, students, and parents.

For this reason consideration of the basis for ethical decision is needed for teachers studying their own practice.

Introduction Practitioner enquiry is enforced by GTCS to encourage teachers into the process informing the practice through research. Whether the class teacher or a collaboration between an enquiry group complete the study, the process and its findings are subject to ethical considerations.

In quantitative research, procedures are planned in advance (questionnaires or scientific protocols). Researcher outline the data they will collect and justify safety.

In qualitative enquiry, practitioners pursue a question through an often-meandering route, finding appropriate data sources as they go.

In qualitative research (teacher is studying their own practice), many of the traditional quantitative guidelines collapse and are replaced with complex ethical issues which must be addressed. Practitioner enquiry

Qualitative methods and multiple perspectives of educational ethnography.

No big numbers, random samples or manipulated variables, but with the human drama as lived by self-conscious actors.

Performed by insiders responsible to the students whose learning we document.

  1. The ethical safeguards of the outsider doing a classic experiment (random selection, control groups, removing the personal influence of the researcher) either irrelevant or problematic for us as insiders. In the same way, the ethical safeguards of the outsider doing qualitative research (anonymous informants, disguised settings) may defeat the action researcher’s goal of open communication and dialogue with colleagues, students and parents. When does good teaching become research? The line may be hard to draw until a study is well underway. Action research tends to involve: 1. more systematic documentation and data gathering;2. more self-reflection in writing; 3. a wider audience (collaboration, presentation, publication). It is this third feature that most often leads to ethical dilemmas. If our journals remain private and our videotapes aren’t played, we can inquire with equanimity. However, in action research, though we document our own practice, we rarely work in isolation. We need the support and collaboration of a colleague, a seminar group or an outside researcher. Often this partnership creates an opportunity for sharing the work with a still larger audience at conferences or in print. Dilemmas of ownership and responsibility arise, and our academic codes of conduct are silent. This Guide emerged from discussions in the Teacher Educator Seminar of the Action Research Collaborative. Monthly, a dozen or more faculty from several colleges and the Writing Project meet to share our own action research and to discuss issues in facilitating teachers’ work. As some teacher inquiries led to proposals for academic theses and school district grants, I found myself struggling with the language and assumptions of the mandated ‘human subjects’ review, the HSR. (For an enlightening history of institutional reviews and their roots in medical research, see Anderson, 1996.) I raised the issue of ethics and teacher inquiry at the Seminar and found that others had similar concerns. When a dissertation proposal I had advised was rejected by the HSR committee at my university, I protested to the Dean of our Graduate School. Most of the HSR’s questions, I explained, did not fit research by teachers in their own workplace. To make matters worse, some important ethical issues were not addressed at all. The Dean agreed that the HSR form was less than ideal, and made the obvious suggestion: “Why don’t you revise the HSR to make it appropriate for your work?” I accepted the challenge, hoping to use the ARC Seminar as a sounding board. After examining questionnaires, ethical reviews and policy documents from local universities and from the American Anthropological Association, the Oral History Association, and the American Educational Research Association, I began drafting the Guide. Drafts were discussed by ARC teacher educators at four seminar meetings. Feedback came from a wider audience of teachers and administrators at several conferences, and in my own graduate courses. What at first seemed a rather straightforward exercise in translation proved a formidable task. The more I tried to account for the different contexts and communities in which action researchers pursue their inquiries, the more complicated and muddled our ethical guidelines became. As teacher educators, we began to see that a “new paradigm code of ethics” would itself become “procrustean” (Gregory, 1990, p. 166). At last, I abandoned the goal of an alternative ‘human subjects review’. Even if an adequate HSR could be designed, putting such decisions in the hands of administrators who were not grounded in action research might do more harm than good. Instead, the following document provides a set of more-or-less provocative questions as a heuristic for reflection. An action research team ETHICAL ISSUES AND ACTION RESEARCH 11 Downloaded by [90.195.21.72] at 14:44 14 February 2016 or university thesis advisor can work through the Guide with a practitioner developing a plan for research. Most of the questions ask the researcher to discuss a potential ethical problem, to consider alternative actions and to explain his or her choices. The Guide to Ethical Issues and Action Research uses the categories of a traditional ‘human subjects’ review only as a point of departure. Part I requests an overview of the project. Part II asks whether the activities fall within the everyday decision-making of a teacher or whether there is some further intervention. Part III examines the ‘subjects’ and the notion of subjectivity in action research. Part IV considers ways to reduce risks to participants – either through informed consent and anonymity, or through openness, dialogue and acknowledgement. Parts V and VI pose questions which, though generally ignored in an HSR, have been especially problematic for action researchers. Guide to Ethical Issues and Action Research Questions for Review and Reflection Part I: overview 1. Briefly describe your project as you see it today. 2. What is the time frame of your project? Is it a one-shot enterprise or does it involve several cycles? Have you already done a pilot study? 3. What problem does your research address? What (initial) action will you take? What do you hope to accomplish? 4. List the research questions as they appear at this time. (Questions will be revised or refocused during your project.) Part II: methods and setting 1. Are you, the researcher, also a participant in the setting where this research will take place? Specify your role (teacher, supervisor, principal, counsellor, social worker, etc.) 2. For this research, will you gather data on your normal educational practice and on changes in curriculum, instruction and assessment that you could make in your role (above) according to your own professional judgement? Explain briefly. 3. What kinds of data will you collect (e.g. field notes, taped interviews, writing samples)? Explain any changes from the way you normally document your practice. Consider how else you could get data on your question. (Can you discuss three alternatives?) 4. What does your research aim to understand? What does your research aim to change? Comments on Part II. Traditional academic research in education is conducted by outsiders who intervene in the instructional process to answer questions that may benefit themselves or the profession in general. Whilethere is often a goal of improving teaching, rarely do the teachers or students under investigation benefit directly from the findings. Action research involves practitioners studying their own professional practice and framing their own questions. Their research has the immediate goal to assess, develop or improve their practice. Such research activities belong to the daily process of good teaching, to what has been called the “zone of accepted practice”. The concept of a zone of accepted practice is often used to determine whether research is exempt from formal review. If a researcher answers ‘yes’ to questions 1 and 2, the project does not need a full review by a university or district research board. Most educational action research would thus seem to be exempt. We urge academic institutions to support reflective teaching and to minimise the bureaucratic hurdles that discourage research by teachers to improve their own practice. However, research in the “zone of accepted practice” may still involve risks to participants. As a precaution, we suggest grappling with question 3 and consulting people who can speak from a variety of perspectives: x An action research project must conform to local school policy; discuss any troubling issues in this Guide with a principal, supervisor or district director of research. x Action research is best developed through collaboration; review the questions with a team leader, professor or consultant. Question 4 begins a closer look at how we choose to change our own practice. According to many reviews, ‘subjects’ are ‘not at risk’ if the research is merely ‘unobtrusive observation’ of behaviour not ‘caused’ by the researcher. However, action research is never detached; a teacher inevitably causes things to happen. (The classic ethnographer observes change, but does not usually try to cause it. On the other hand, the action researcher consciously tries to change and improve his or her own teaching.) Part III: ‘subjects’ and subjectivity 1. Describe the individuals, groups or communities you plan at this point in the research to study. Estimate the ages of the people involved. 2. Analyse the power relations in this group. Which people (e.g. students, parents) do you have some power over? Which people (e.g. principals, professors) have some power over you? 3. What shared understandings do you have with these people? Do you have personal bonds, professional commitments? Will your research strengthen this trust or perhaps abuse it? 4. Will your study attempt to read and interpret the experience of people who differ from you in race, class, gender, ethnicity, sexual orientation or other cultural dimensions? How have you prepared yourself to share the perspective of the ‘other’ (coursework, experiences, other sources of insight)?
  2. Will an ‘insider’ review your questionnaires or teaching materials for cultural bias? Have you provided for consultation by adult members of the community? How will you reduce or correct for your misreading of populations who differ from you? 6. Does your inquiry focus on people with less power than you? Children in classrooms are always vulnerable – especially if their families have little money or education. (“Where are the ethnographies of corporate boardrooms?” asks House, 1990, p.162.) How does your project demonstrate mutual respect and justice? 7. What negative or embarrassing data can you anticipate emerging from this research? Who might be harmed (personally, professionally, financially)? What precautions have you taken to protect the participants? 8. Might your research lead to knowledge of sensitive matters such as illegal activities, drug/alcohol use or sexual behaviour of participants? How do you plan to handle such information? Comments on Part III. We must examine the impact of our research on the people whose lives we document. A classroom teacher may write field notes in order to improve her own practice. However, what if her notes focus on certain members of the class (‘at-risk’/’Black male’/’learning disabled’)? Our students and colleagues are more than ‘subjects’. The following distinctions are useful: Fully involved in planning and interpretation. Collaborator : More involved; perspective considered in research.Participant : Knowingly gives information to researcher.Informant : Observed by researcher; no active participation. (Not applicable to action research.) Subject: Perhaps most of all, we need to examine our own subjectivity as researchers. Since I cannot be a fly on the wall in my own classroom, I must deal with my own emotional and interpersonal responses as part of my data. Hammersly & Atkinson (1983) call this the principle of ‘reflexivity’. Sullivan (1996) writes of the “problem of the ‘other’.” Teacher research is engaged and committed. It is appropriate – essential – for our discussions and writing to look at ourselves in relationship with other participants. Part IV: risks and benefits How can we protect K-12 students but not inhibit teachers’ right to gather and reflect on data from their own teaching? 1. Describe the possible benefits of your research – to students, teachers or other participants; to society or to the profession. 2. Describe any risks to people participating in this study. For example, will your current students be disadvantaged for the possible benefit of future students? What steps are you taking to minimise risks?
  3. Show how you will protect the people from whom you collect data through surveys, interviews or observations. For example, participants are usually considered free from risk IF: a. they are first informed; they must know the general nature of the study and what is expected of them; b. they give informed consent; c. they can refuse to participate and they can withdraw without penalty after beginning the research; d. anonymity of persons and/or confidentiality of data are protected if appropriate. 4. Describe your method of obtaining informed consent. Who will explain the consent document to the participants? How? 5. Are different kinds of consent needed at different stages in the project? For example, many teachers use two consent forms: a.  a blanket consent to be in the study; if you regard classroom inquiry as part of your regular practice, this blanket consent form may be given to all students at the start of each year; b.   a special consent to eventual publication; this will be needed when you prepare to publish student writing samples, taped discussions, photographs, or field notes that focus on a recognisable student. 6. Do you wish to protect the anonymity of students, teachers, parents and other participants? If so, it is wise to use pseudonyms even in your field notes. If your report is eventually published, you can also interchange physical description, grade level, gender, etc., or develop composite rather than individual portraits. What are the gains and losses of anonymity? 7. On the other hand, instead of anonymity, it may be wiser to seek full participation and credit for students and colleagues. Research by an educator in his or her own classroom is rarely anonymous. Even if names are changed, students will be recognised in a well-written case study or classroom scene. What are the gains and losses of open acknowledgement? Comments on Part IV. These questions deal with the welfare of students and colleagues. Most university definitions of ‘informed consent’ resemble this one from the AERA’s Qualitative Research SIG: a decision made free of coercion and with full knowledge [of] possible effects of their participation, their role in reviewing written accounts …, an understanding that the researcher will protect them from potential harm, and that there will be a mutually respectful relationship. Informed consent is granted at the initiation of the study and codified in signed consent forms. Because informants may withdraw at any time, informed consent is ongoing, continual negotiation. (Mathison et al, 1993, p. 3) How informed is ‘informed consent?’ Lou Smith argues that “field research is so different from the usual experimental approaches that many individuals, even responsible professional educators, do not understand what … they are getting themselves into” (Smith, 1990, p. 151). He stresses the need for

‘dialog’, moving beyond ‘contract’ relationships to ‘covenants’of trust (p. 150). As teacher researchers, our primary responsibility is to our students. We need to balance the demands of our research with our other professional demands. This issue becomes far less troublesome when classroom inquiry becomes an intrinsic part of how we teach, and when students take an active role in our research – and their own. Part V: ethical questions specific to ‘insider’ research These questions don’t appear on most ‘human subjects’, reviews, but they are central to research by K-12 and college teachers. Yvonna Lincoln (1990) says, “privacy, confidentiality, and anonymity regulations were written under assumptions that are ill suited” to action research. Our colleagues, administrators, and parents might better participate “as full, cooperative agents,” our co-researchers (pp. 279 – 280). 1. Which of the research participants at your school/college have read your proposal? Which ones have been informed of the research orally in some detail? Which ones know little or nothing of this project? Explain and justify the decisions behind your answers. 2. What do your students know of this project? Who told them? What are the risks to them or their families of their knowing (or not knowing) what you write or collect? Explain your decisions. 3. Who else will read your field notes or dialogue with you to provide ‘multiple perspectives? Lather describes “the submission of a preliminary description of the data to the scrutiny of the researched” (p.53) as an emancipatory approach to inquiry and also as a way to establish ‘face validity’ (p. 67). Incorporating quotes from other participants, especially when their views differ from yours, can make your work richer, more nuanced. 4. You will inevitably gather more data than you ‘need’. Consider why you choose to report some data to a wider audience and why you choose to keep some for your colleagues, your students or yourself. (What do you tell and what do you store?) Consider the political implications of the way you focus your story. 5. How will you store and catalogue your data during and after the study? (File cabinets? Computer, tapes, transcripts?) Who will have access? Should you take special precautions with your notes and other data? 6. Will this study evaluate your own effectiveness or a method to which you are committed? Will your findings be confirmed by observers who do not share your assumptions? How will you protect yourself from the temptation to see what you hope to see? 7. Who is sponsoring this research through grants, contracts, released time, course credit, etc.? Will you evaluate the sponsor’s programme, textbook, method, etc.? Can you protect yourself from pressure to report favourably on the sponsors?

  1. How do your school administrators see your work? Is action research under suspicion or is it mandated from the top in a drive for organisational quality control? Is there protection for your own thoughts, feelings, interpretations? How safe do you feel in this institutional environment pursuing this research? Reporting what you learn to a wider audience? 9. What data will be contributed by others? Will you be recording case studies, oral histories or other stories that may be considered the property of others? How have you arranged with colleagues or other participants for: x credit in your manuscript? x publication rights? x royalties? x other recognition? 10. If your study is collaborative, how are you negotiating authorship and ownership? University researchers, colleagues, students and parents are likely to interpret their stake in the research in quite different ways. Who owns the videotape of a classroom writing group, the dialogue journal between teacher and mentor, the transcription of talk by teacher researchers in a college seminar? 11. Who is responsible for the final report? Will other stakeholders (teacher? principal? school board?) review your report in draft? Will this: (a) improve your accuracy? (b) compromise your candor? Participants may not agree with part or all of your interpretation. If so, you may revise your views; quote their objections and tell why you maintain your original view; or invite them to state alternative views in an appendix. 12. Have you decided on anonymity or on full acknowledgement if your study is eventually published? Perhaps you will identify teachers, but use pseudonyms for students. How and when have you negotiated these issues? Part VI: The Golden Rule At the most basic level, (Smith, 1990, p. 149) suggests that we, as class-room researchers, ask ourselves these questions: x What are the likely consequences of this research? How well do they fit with my own values and priorities? x If I were a participant, would I want this research to be done? What changes might I want to make me feel comfortable? Teacher-researcher Marian Mohr states it this way: “Teacher researchers are teachers first. They respect those with whom they work, openly sharing information about their research. While they seek knowledge, they also nurture the well-being of others, both students and professional colleagues” (Monr, 1996). Action researchers need to discuss with their constituencies the role of classroom inquiry in their professional lives. For example, some teachers display for parents their own publications as well as the writing of their students informally printed, illustrated and bound. Teacher-researcher

Kathryn Mitchell Pierce asks at an ‘open house’ for parents’ support in gathering data for professional development. She adds, “I’ll come back to you again for more specific permission if your child appears in anything I plan to publish.” In this way, parents and students are knowingly involved in the work from the beginning, with time to ask their own questions and make thoughtful suggestions. Open communication is the key to overcoming the split between researcher and researched, between theory and practice. Correspondence

 

References

Anderson, P. V. (1996) Ethics, institutional review boards, and the involvement of human participants in composition research, in P. Mortensen & G. E. Kirsch (Eds) Ethics and Representation in Qualitative Studies of Literacy. Urbana: National Council of Teachers of English.

Gregory, T. B. (1990) Discussion on ethics, in E. Guba (Ed.) The Paradigm Dialog, pp. 165–166. Newbury Park: Sage. Hammersley, M. & Atkinson, P. (1983) Ethnography: principles in practice. London: Routledge. House, E. R. (1990) An ethics of qualitative field studies, in

E. Guba (Ed.) The Paradigm Dialog. Newbury Park: Sage. Lather, P. (1991) Getting Smart: feminist research and pedagogy with in the postmodern. New York: Routledge.

Lincoln, Y. (1990). Toward a categorical imperative for qualitative research, in E. Eisner & A. Peshkin (Eds) Qualitative Inquiry in Education. New York: Teachers College Press.

Mathison, S., Ross, E. W., & Cornell, J. (Eds) (1993) Casebook for Teaching about Ethical Issues in Qualitative Research. Washington: Qualitative Research SIG, American Educational Research Association.

Mohr, M. (1996) Ethics and standards for teacher research: drafts and decisions, in Research in Language and Learning: reports from a teacher research seminar. Fairfax: Northern Virginia Writing Project, George Mason University. (Informal publication distributed by the Writing Project.).

Mortensen, P. & Kirsch, G. E. (1996) Ethics and Representation in Qualitative Studies of Literacy. Urbana: National Council of Teachers of English.

Pierce, K. M. (1997) Presentation at seminar on ethical issues in action research, Clayton (MO) Public Schools.

Smith, L. (1990) Ethics, field studies, and the paradigm crisis, in E. Guba (Ed.) The Paradigm Dialog. Newbury Park: Sage.

Sullivan, P. (1996) Ethnography and the problem of the ‘Other’, in P. Mortensen & G. E. Kirsch (Eds) Ethics and Representation in Qualitative Studies of Literacy. Urbana: National Council of Teachers of English.

Zeni, J. (1996) A picaresque tale from the land of kidwatching: teacher research and ethical dilemmas, Quarterly of the National Writing Project and the Center for the Study of Writing, 18(1), pp. 30– 35. ETHICAL ISSUES AND ACTION RESEARCH

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s