QTM 385 - Experimental Methods

Lecture 12 - Ethics in Experimental Research

Danilo Freire

Emory University

Hello, everyone!
Nice to see you! 😉

Brief recap 📚

Last class

  • Attrition: Loss of participants before study completion
  • Two main types:
    • Random attrition: Missingness unrelated to treatment/outcome
    • Non-random/Systematic attrition: Missingness correlates with variables
  • Impacts include reduced statistical power, potential selection bias, and compromised generalisability
  • RAND HIE example: Health insurance experiment where higher dropout in cost-sharing plans (25%) vs. free care (8%) potentially biased results
  • Analysis methods:
    • Inverse probability weighting (IPW): Upweights observations similar to missing ones
    • Extreme bounds analysis (EBA): Evaluates sensitivity by assuming best/worst case scenarios
    • Lee bounds: Narrows bounds using monotonicity assumption
  • R implementation: attritevis package for visualising and analysing attrition patterns

Today’s plan 📅

Ethics in experimental research

What should we consider when conducting experiments?

  • A brief history of research ethics
  • Ethical considerations in experimental research
  • Informed consent
  • Deception in experiments
  • Ethical review boards
  • Ethical guidelines for research

Source: XKCD

A brief history of research ethics 📜

In the beginning…

…there was no ethics!

  • Early research often proceeded with little formal ethical oversight
  • Harmful experiments on humans were conducted without consent or consideration for subject well-being
  • The focus was exclusively on scientific advancement, sometimes at the expense of human rights
  • Examples of unethical research existed throughout history, but the scale and insensitive nature of some events in the 20th century prompted change

The Tuskegee Syphilis Study (1932-1972)

  • The Tuskegee Study of Untreated Syphillis was a clinical study conducted between 1932 and 1972 by the U.S. Public Health Service, the Centers for Disease Control and Prevention, and the Tuskegee Institute
  • The goal was to observe the natural progression of untreated syphilis in rural African Americans who thought they were receiving free health care from the US government
  • 399 men had syphilis, 201 did not, and none were informed of their diagnosis or the study’s true purpose
  • The study was terminated in 1972 after its unethical nature was exposed
  • By the end of the study, 28 participants had died of syphilis, 100 were dead of related complications, 40 of their wives had been infected, and 19 of their children were born with congenital syphilis
  • In 1997, President Bill Clinton apologised on behalf of the US government for the study

The Nuremberg Code (1947)

Some principles we still follow today

  • The Nuremberg Code was a set of research ethics principles for human experimentation created as a result of the Nuremberg Trials at the end of World War II
  • Why? Well, I am quite sure you know the reason…
  • The code established 10 principles for the conduct of human experiments:
  • Voluntary consent
  • Beneficence towards participants
  • Avoidance of unnecessary suffering
  • No experiments without prior animal testing
  • Avoidance of unnecessary physical and mental suffering
  • No experiments if death or disabling injury is expected
  • Risks should not exceed benefits
  • Proper preparation and facilities
  • Qualified scientists
  • Participants can withdraw at any time
  • The experimenter must be prepared to stop the experiment at any time
  • Full text here

The Belmont Report (1979)

  • The Belmont Report was written by the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research
  • The background was the Tuskegee Syphilis Study and other unethical research
  • The report outlines three fundamental ethical principles for research involving human subjects:
    • Respect for persons
    • Beneficence
    • Justice
  • These principles are the basis for the Common Rule, which governs federally funded research in the US
    • The Common Rule requires institutional review boards to review research proposals
  • The report also introduced the concept of informed consent

Full text of the Belmont Report

Ethical considerations in experimental research 🤔

Ethical research is good research

  • Ethical considerations are strongly linked to the quality of research
  • More specifically, ethical practices reduce attrition and improve data quality
  • How so?
    • Participants who feel respected are more likely to provide accurate and complete information
    • Participants deceived about study purpose might intentionally skew responses
    • Damaged relationships with communities can hinder future research and interventions
    • Example: The Tuskegee Study led to deep mistrust of medical institutions

Respect for persons

Autonomy and protection

  • Experiments are by definition interventional, meaning participants are exposed to treatments or conditions
  • So individuals have the right to choose whether or not to participate in research
  • This is known as informed consent and should include information on:
    • Study purpose
    • Procedures
    • Risks and benefits
    • Compensation
    • Confidentiality
    • Right to withdraw
  • Consent can be written or oral, depending on the study
  • Oral consent is common in phone surveys, field experiments, or when written consent is impractical (e.g. due to literacy issues)
  • Special considerations for vulnerable populations (e.g. children, prisoners, individuals in economic hardship)
  • What to do in those situations?
    • Use independent advocates or representatives to assist vulnerable individuals
    • Seek the individual’s agreement as much as possible
    • Use age-appropriate language to obtain assent

Anonymity and confidentiality

  • Anonymity must be used by definition (with all identifying information of the subjects destroyed after the study), as it provides the greatest protection to those subjects

  • Sometimes anonymity is not possible

    • Example: elite interviews
  • But bear in mind that promises of confidentiality from researchers to subjects can be overridden by court orders

  • For example: Boston College and oral history tapes of Northern Ireland’s Troubles (https://www.bbc.com/news/uk-northern-ireland-27238797).

  • What would you do in this situation?

Members of the Irish Republican Army (IRA) in Northern Ireland

Institutional Review Boards (IRBs) 🏛️

What are IRBs?

  • Institutional Review Boards (IRBs) are committees that review research proposals to ensure they meet ethical standards
  • IRBs are required for federally funded research in the US and in many other countries
  • They are composed of scientists, ethicists, and community members who evaluate the risks and benefits of research
  • There is a formal (standard) application for the approval of research with human beings
  • This is not optional. You cannot interact with human research subjects before this is approved
  • The board may consider that your research is enabled to continue, but it is the board that decides this (not you)
  • You are obliged to inform the board about “adverse events” of the research

What do IRBs consider?

  • Description of the population of subjects, how they will be recruited, how they will be compensated (if they are), the context of the research and what you will do in it
  • Consent form and standard elements necessary to explain the research to the study subjects
  • The data collection instrument (survey questionnaire, etc.)
  • Statement of the expected benefits and risks, including how the results will be disseminated
  • How subjects can ask questions and report any problems to the review committee and/or you in real time
  • How you will ensure the confidentiality, anonymity, and security of the data

Ethical Review Committees Outside the US

  • In some countries, the ethical review committee is located in a ministry of technology/science and will be more accustomed to dealing with medical research than with social science research
  • Universities and research institutes or organisations often have their own ethical review committees
  • Prepare to explain your research a little more
  • Perhaps you could plan a meeting over coffee to contextualise social science research for a member of a medical review committee
  • Knowledge of the local language and legislation is very important

Two recent examples of controversial research

Facebook’s emotional contagion study

Source: Kramer et al. (2014)

Two recent examples of controversial research

Protests in Hong Kong

Source: Bursztyn et al. (2021)

Beneficence: “Do no harm, maximise benefit”

  • Two key points:
    • Do no harm (non-maleficence)
    • Maximise benefits, minimise risks (beneficence)
  • Risk-benefit assessment is important for every study and it is your responsibility to ensure that the benefits of the research outweigh the risks
  • Consider Equipoise: Genuine uncertainty about best approach
  • Knowledge gain is great, but only if research is credible and well-designed
  • Sometimes it is better not to run the experiment at all!

Do parachutes save lives?

One of my favourite papers ever 😂

Source: Yeh et al. (2018)

Adaptive research design 🔄

Adaptive randomisation

  • Adaptive randomisation is a technique that allows researchers to adjust the allocation of participants to different treatment groups based on interim results
  • That is, the randomisation probabilities can be adjusted during the study, usually to include more people in the group that is performing better
  • No more 50/50 randomisation!
  • This procedure is quite popular in clinical trials and can be used to maximise the benefits of the research
  • But it requires that the best solution is found quickly and that the researchers are able to adapt to the new information

Adaptive randomisation

An example

Source: Pallman et al. (2018)

Adaptive randomisation

An example

Source: Pallman et al. (2018)

Justice: Fair distribution of burdens and benefits

  • Justice in participant selection is another ethical principle to consider
    • Who is asked to participate? Who is excluded?
  • Who bears risks? Who benefits?
    • Is distribution of risks and benefits equitable?
    • Avoid disproportionate burden on vulnerable
  • The concept is quite complex because the idea of justice is not universal
    • Equality, need, merit, societal contribution
  • Randomisation can promote justice, as it ensures:
    • Equal expected inclusion in research
    • Equal expected access to intervention
    • Promotes procedural fairness: everyone has equal chance and there’s no favouritism
  • Limitations of randomisation
    • Doesn’t address broader societal inequalities: should everyone have equal chance?
    • May not be “just” if some populations are over-researched (India, Kenya, etc)

Implementing justice in practice

  • Inclusive research
    • Target populations experiencing the problem
    • Focus on those who benefit from knowledge
    • Avoid convenience samples of less relevant groups
    • Example: Health disparities in minority groups
  • Avoid exclusion for convenience
    • Resist excluding groups due to logistics/cost
    • Exclusion biases data, perpetuates inequities
    • Example: Accommodating participants without internet
  • Consider Heterogeneous Effects
    • Power study to detect subgroup impacts
    • Understand equity implications
  • Carefully Justify Sampling
    • Ethical reasons for oversampling subgroups
    • Oversampling for convenience is unethical
    • Provide ethical rationale, IRB review

Another example for discussion

Source: Bertrand et al. (2006)

Research staff ethics 🧑‍🔬

Ethical considerations for research staff

  • Belmont principles focus on participants, not staff
  • Research staff also face ethical risks and burdens
  • Ignoring staff well-being is unethical and harms research
  • Physical sisks: Violence, accidents, health hazards
    • Example: Enumerators in high-crime areas
  • Psychological sisks: Emotional burden, secondary trauma
    • From sensitive topics (trauma, poverty)
    • Example: Surveyors interviewing abuse survivors
  • Staff well-being is crucial
    • For ethical research and data quality
    • Burnt-out staff make more errors, leave projects

Protecting Research Staff in Practice

  • Safety training
    • Prepare for worst-case scenarios
    • De-escalation techniques, emergency procedures
    • Example: Responding to aggressive respondents
  • Debriefing and support
    • Regular sessions for processing emotional burden
    • Counseling access for secondary trauma/burnout
    • Supportive team environment
    • Example: Weekly team meetings for surveyors
  • Clear communication
    • Explain study rationale, protocols to staff
    • Address moral dilemmas, staff concerns
    • Equip staff to explain study to participants
    • Example: Workshops explaining randomization ethics
  • Fair compensation and working conditions
    • Fair wages, benefits, reasonable workload
    • Prevent burnout, ensure well-being

Another example for discussion

Source: The PREVAIL II Writing Group (2016)

Community ethics 🏘️

Ethical considerations for the community

  • Ethics extends beyond direct participants too!
  • Consider broader community impacts
  • Research can have intended and unintended consequences for communities
  • Spillovers: Research affects non-participants
    • Can be positive or negative
    • Example: Job program increasing competition
  • Confidentiality Agreements in group settings
    • Protect information about non-participants
  • Minimise Public Interventions
    • Conduct activities discreetly
  • Spillover Measurement
    • Design research to capture community effects
    • Collect data from participants and non-participants
    • Example: Health intervention, community health outcomes
  • Reconsider study if negative spillovers are likely and unmeasurable/unmitigable

Last example

Source: Banerjee et al. (2008)

Conclusion 🎓

Conclusion

  • Ethical conduct is integral to good research
    • Not just compliance, but core aspect
  • Respect, Beneficence, Justice are essential principles
    • Practical guides for ethical decision-making
    • Apply principles throughout research
  • Anticipate challenges, plan for mitigation
  • Ongoing reflection and dialogue needed
  • Think of adaptive research designs!
  • Ethical research leads to:
    • Better data (validity, reliability)
    • Stronger partnerships (trust, cooperation)
    • Greater positive impact (sustainable change)

And that’s all for today! 🎉

Thank you! 🙏