Home / National Ethical Standards / Part two / 9. Research development and design

9. Research development and design

Introduction

For research to be ethical, it must be well designed, and the research question must have the potential to lead to improvements in health or wellbeing. Well-designed research is scientifically robust, and uses a research methodology that takes account of relevant cultural, social and economic factors.

The design of a research study is critical in determining whether the research achieves its proposed outcomes, benefiting participants and communities. Tika refers to what is right or good in any given situation. In this context, it relates to the validity of the research proposal.

Research design

9.1 Researchers must ensure that their study design is appropriate to answer the research question.

9.1.a Only appropriately designed research will justify the risk of harm, any inconvenience associated with it and resources allocated to it. A study design has a strong impact on whether the study meets its objectives. It also influences the study methods, conduct, costs, outcomes, interpretation and potential for translation of findings into practice.

9.1.b Research that is methodologically sound will meet generally accepted requirements relating to the subject matter, the population under study, and the research method and analysis. Internal validity, reliability, generalisability and translatability of study methods and results may be important aspects of the study’s scientific value.

9.1.c In the case of international research, where a design has already been developed, New Zealand researchers should consider how they can adapt the local documentation for a New Zealand context.

9.2 Researchers must have the necessary skills and resources to undertake and design the research.

9.2.a Researchers conducting clinical trials should be trained in Good Clinical Practice.[1]

9.3 One or more individuals with the appropriate disciplinary and cultural knowledge, skills and experience must review the research. Reviewers must be impartial to and independent of the research team.

9.3.a Some types of research (e.g. cross-cultural research) should entail an ongoing process of consultation.

Co-design, co-production or participatory research designs

Participant engagement in the entire research or QI process, encompassing activities from question identification through research design, data collection and analysis to interpretation, can often translate into better outcomes. Participant engagement may occur within any or all of the above stages. Co-design, co-production[2] or participatory research designs[3] involve a mutually advantageous collaboration between researchers and participants who may also be the end-users of research discoveries.

The nature of such designs makes it difficult to specify the study interventions or QI measures, or the roles of participants, in advance. It is therefore difficult for researchers to gain informed participant consent (and ethics committee approval) up front, as they would for traditional designs. This is one of a number of distinct challenges this methodology raises for researchers (and ethics committees); another is power imbalances.

9.4 Researchers should ensure they allow adequate time for the ‘design’ phase with participants, prior to formal trial and evaluation.

9.5 Researchers should be able to justify why the co-design approach is likely to maximise the impact and minimise the harms of the research, and should identify which of the study features are likely to be integral and stable components of the research, and which will subject to openness and co-creation with community partners.

9.6 Researchers should also be able to define the potential benefits of power-sharing in terms of reducing inequalities and empowering vulnerable communities, and should have a developed plan for managing the relationships between themselves and their participants in ways that are likely to achieve these benefits.

9.6.a Researchers will find it helpful to establish a cooperative relationship with the ethics committee responsible for safeguarding participants’ safety and welfare, and to make staged applications[4].

Protocol

A research protocol is a document that details the plan for conducting a study, including its purpose and how the research will be conducted. This information helps demonstrate that the researcher has considered and addressed the ethical and scientific or methodological issues associated with the study.

9.7 Researchers must conduct their research according to a suitably detailed protocol.

9.7.a The level of detail the protocol contains should be commensurate to the risk of the activity.

9.8 Protocols must include all information that is relevant for the type of study. Unless not relevant to the study type, all research protocols should include:[5]

  • the study title
  • the principal researcher, study site(s) and sponsor
  • a literature review summarising existing knowledge and highlighting gaps in knowledge
  • a clear statement of the justification for the study, including the expected benefits and merit of the research, and how they outweigh the harms
  • a summary of the proposed research
  • a description of the ethical and regulatory aspects, including the ethical risks and considerations raised by the study, and how researchers propose to deal with them
  • a description of consultation undertaken, and how researchers have incorporated feedback into the research design
  • any partnership arrangements in place with whānau, hapū and iwi
  • the study hypotheses or objectives
  • the main outcome(s) of interest
  • a detailed description of, and clear justification for, the study design
  • a clear and ordered plan of study conduct
  • criteria for including or excluding potential participants, with justifications
  • criteria for terminating the study, if appropriate
  • what data will be collected, stored and used, and how it will be collected, stored, used and kept private
  • the number of participants required to achieve the study objectives, and how the researchers determined this, for example using statistical methods
  • an analysis plan appropriate to the study design
  • how the study results will be shared publicly and communicated to participants
  • disposal of study data
  • actual or potential conflicts of interest, and how researchers will manage them.

Research population

All groups have the right to benefits from advances in health care and disability support arising from research. For this reason, it is important that researchers design their research to be inclusive. A study’s focus and objectives, and the nature and context of the research, should determine its inclusion and exclusion criteria.

9.9 Researchers should not exclude participants on the basis of their age, disability, sex, sexual orientation, gender, place of residence, ethnicity, nationality, religion, education or socioeconomic status, except where excluding or including them on these grounds can be justified for the purposes of the research.

9.9.a Inclusion and exclusion of participants affect the extent to which researchers can generalise their findings. Researchers should enrol participants who represent all those to whom the research findings may apply, thereby contributing to an equitable distribution of research benefits and burdens.

9.10 Researchers must collect ethnicity data, unless there is a valid justification why this is not necessary.

Research with women

Women have historically been excluded from much health-related research because of their childbearing potential. However, as women have distinctive physiologies and health needs, they must be included in research. Furthermore, these Standards recognise the importance of women leading and contributing to research about women. In all cases, but particularly where research concerns issues where women, primarily, are victimised (such as sexual assault and domestic violence), researchers must be competent in their understanding of gender stereotypes and gender-based power structures within society, and sensitive to the impact of these factors on women’s lives and wellbeing.

9.11 Researchers should not exclude potential participants on the grounds of sex or gender expect where this is necessary for the purposes of the research.

9.12 Researchers should not exclude women from research without sufficient justification, or simply because they are biologically capable of becoming pregnant.

9.13 Researchers must recruit a sufficient number of women so that results are generalisable, and so that they reliably account for gender differences in treatment, disease processes or basic biological processes.

9.14 Researchers should be aware of specific circumstances in which women could be vulnerable in research, including: research with sex workers, research on family violence, research with trafficked women, research concerning abortion and research with women from a cultural context in which it is customary for them to make decisions in conjunction with a spouse or male relative.

9.14.a When women in such situations are potential participants in research, researchers must ensure that they freely give their informed consent and provide suitable mechanisms for obtaining this consent.

Pregnancy

Because pregnant and breastfeeding women have distinctive physiologies and health needs, research designed to build knowledge relevant to the health needs of pregnant and breastfeeding women is important. There is a substantial gap in the medical literature regarding safe and effective health interventions for pregnant women, this has a negative impact on the health and wellbeing of pregnant women, their foetuses and future children. This lack of knowledge is both harmful and unjust. Therefore, research relevant to pregnant women's health needs must be promoted (Van Der Graaf, Van der Zande, & Van Delden 2019).

9.15 Researchers must not routinely exclude pregnant or breastfeeding women from participating in research. Researchers must carefully consider the available data and identifying important gaps within it. Researchers must not consider pregnant women to be vulnerable simply because they are pregnant.

9.15.a All participants (including pregnant women) are eligible to participate in studies unless there is a clear scientific reason to exclude them

9.16 Pregnant women and their foetuses are often physiologically vulnerable. Therefore, research must be designed to minimise the risk of harm to pregnant women and their foetuses.

9.17 Researchers should carefully consider what risks of harm their study may pose to the foetus, including stress or pain in utero.

9.18 In research involving women who are pregnant, the wellbeing and care of the woman who is pregnant take precedence, and then the wellbeing and care of her foetus takes precedence over research considerations (NHMRC 2018).

9.19 Rather than automatically removing women from a study when they become pregnant, researchers must carefully consider whether it is safe for women in the specific circumstance to continue to participate, taking into account that there is a lack of research data for pregnant women, and recognising that women might want to stay in the study.

9.19.a Researchers must request consent to follow-up until birth for women who become pregnant during a trial where there is any possibility participation could have had adverse effects on the foetus.

Ethnicity data collection

One important step in addressing inequalities and achieving health equity is to consistently collect good-quality ethnicity data. This can be a source of comparative data, and can influence the outcomes and recommendations of research. Ultimately, it can contribute to improving Māori health outcomes and reducing inequities.

New Zealand is recognised as a world leader in its ability to analyse health data by ethnicity.[6] Health research helps to track the growing diversity of the population, and to provide more detailed information for planning, funding and monitoring health services.

9.20 All researchers conducting health research in New Zealand must collect good-quality ethnicity data[7].

9.20.a The process of collecting and reporting ethnicity data in New Zealand has evolved significantly over time. New Ethnicity Data Protocols, released in October 2017 (Ministry of Health 2017a), are a standard of the Health Information Standards Organisation that cover research.

9.20.b In some cases (e.g. small studies with specific population groups), it may not be necessary for researchers to collect ethnicity data. However, it is good practice to include ethnicity as a variable as part of any demographic data. See Equal explanatory power below.

Equal explanatory power

In the New Zealand context, ‘equal explanatory power’ refers to the power of research to generate findings and offer explanations that are specific to minority communities. An aim to achieve equal explanatory power is an aim to produce information to improve Māori health to at least the same depth and breadth as information to improve non-Māori health.

Typically, nationally representative population samples aimed for 15 percent Māori but often underachieved due to inappropriate and/or ineffective methods of encouraging Māori participation. Smaller Māori samples limit the robustness of the Māori data analyses and therefore the relevance of the conclusions and recommendations for Māori.

Achieving equal explanatory power will not be practical for all contexts, especially for interventional studies that typically have a limited number of participants. In such situations, researchers should be aware of potential statistical issues with subgroup analysis within prospective interventional clinical trials and must avoid erroneous or dangerous conclusions.

Equal explanatory power in both quantitative and qualitative research in New Zealand is important, to prevent research conclusions from contributing to increasing inequality.

9.21 Researchers must consider the degree to which equal explanatory power is relevant for their study or hypothesis.

9.21.a In quantitative research of the general population, such as population surveys, this consideration may involve oversampling Māori participants.

Exclusion criteria

Exclusion criteria are not the inverse of the inclusion criteria. Instead, they identify individuals who meet the inclusion criteria but cannot be included in the study for some other reason. Exclusion reduces the generalisability of study results. Exclusion is justifiable when inclusion poses potential safety concerns to participants, or when their inclusion may impact on scientific validity.

Overprotective attitudes or practices on the part of researchers can exclude members of some groups in society from participating in research. In many cases, if knowledge about the health experiences and needs of these groups is to advance, they need to participate in research in appropriate ways.

9.22 Researchers should give special consideration to including individuals from all groups in society in their research.

Researchers’ skills and resources

9.23 Researchers must be suitably skilled and resourced (or, if appropriate, be supervised by an appropriately skilled and resourced person), to minimise risk to participants and realise the potential benefits of their study.

9.23.a Some procedures may only be performed by health practitioners under the Health Practitioners Competence Assurance (Restricted Activities) Order 2005.

9.24 Researchers must have adequate facilities, time and resources available to them, to conduct their study safely and in the intended timeframes.

9.24.a Appropriate skills and resources may include:

  • being competent in the relevant field of research and research methods, as demonstrated by knowledge, qualifications, experience and current awareness of good practice guidelines
  • having experience in identifying and applying relevant research methods, and the ability to take full responsibility for appropriate research design, conduct and analysis
  • appreciating the research context and environment, including understanding different community and cultural views
  • understanding the inequalities in the health and wellbeing of populations, in particular those experienced by Māori and Pacific peoples
  • the ability and resources to monitor participants throughout the research
  • knowing researchers’ ethical responsibilities and demonstrating ethical principles
  • the ability to suitably protect confidentiality and data
  • appropriate skills and resources to deal with unexpected events that may affect participants or researchers
  • a suitable research setting (e.g. with qualified staff and appropriate infrastructures for safe and ethical conduct)
  • an appropriate budget to allow researchers to complete their study in a timely way
  • appropriate indemnity cover
  • the capacity to disseminate and communicate research findings.

Peer review

A peer review process should be commensurate with the type of proposal, the potential risk to participants and the location of the research. The type of peer review process used must be fit-for-purpose and justifiable. For example, peer review of a graduate student project carried out largely within a tertiary institution will differ from that of a multi-centre clinical trial. Researchers may seek opinions from one or more peers who are independent of the study; the extent of peer review, like its type, should be fit-for-purpose.

9.25 In order to determine scientific validity, the peer review process should specifically determine the following factors:

  • The relative merit of the research: As a key consideration, peer reviewers should determine whether the proposed work is important, worthwhile and justifiable. The research should address a health issue that is important for health and/or society. The aims, research questions and hypotheses should build on and address gaps in existing knowledge.
  • The design and methods: Peer reviewers should review the quality of a study’s design and methods to assess its robustness. This might cover study methodology, a description of sample recruitment and characteristics (including number, gender and ethnicity, where relevant) and proposed methods of data analysis.
  • The feasibility of the research: Peer reviewers should consider whether the overall strategy, methodology and analyses are well reasoned and appropriate to achieve the specific aims of the project. They should determine whether the research has the likelihood, on balance, of improving scientific knowledge, concepts, technical capacity or methods in the research field, or of contributing to better treatments, services, health outcomes or preventive interventions. Peer reviewers should assess whether the research will be achievable within the specified timeframe, and whether the research team has the appropriate experience and expertise to undertake the research.

9.26 Peer review should address the validity and feasibility of the design, methods and analysis of the study. Additional specialist (e.g. statistical, economic, cultural or analytical) review may be required.

9.27 Peer review must include a consideration of cultural relevance and appropriateness.

9.28 All research proposals should be peer reviewed in a way that is fit-for-purpose and proportional. Suitable reviewers will have appropriate expertise and an appropriate skill set. It may be appropriate to involve more than one reviewer.

9.29 Reviewers must be sufficiently independent to be able to conduct their review of the study without bias.

9.30 Researchers should give peer reviewers sufficient details of the proposed study for them to consider the scientific validity of the study. Commercial sensitivity is not an acceptable justification for failing to seek independent review.

9.31 Reviewers should consider the ethical aspects of the study. Studies can be of satisfactory scientific quality, as judged by peer review, but still pose ethical concerns.

Core features of the peer review process

9.32 An appropriate process for ensuring scientific validity will have the following features:

  • Peer review delivers an informed opinion: An effective peer review process provides perspectives from subject matter experts. It may be appropriate for researchers to seek informed perspectives from individuals in the same organisation as the researcher, as long as the requirements of freedom from bias, equity and fairness can be met. An appropriate peer is one who can deliver an informed opinion on some or all of a proposal. Peer reviewers will be knowledgeable about the topic and/or context for the research; have the appropriate expertise relative to the breadth and scope of research under review; and, as a result, will be well placed to make a statement as to whether the research has verifiable scientific merit.
  • Peer review delivers an objective opinion: Peer reviewers are charged with delivering a balanced and considered analysis of the research. Generally, the success of the peer review process is determined by the extent to which these evaluations can be considered free of bias, equitable and fair. Objectivity can be compromised if peer reviewers have conflicts of interest, and so appropriate peer reviewers typically will not be materially connected to the researcher(s) in a way that might undermine objectivity, and be free from either positive or negative inducements.
  • A consensus opinion on scientific validity is formed: An ethics committee needs to receive assurance that the peer review process has delivered support for the scientific validity of the proposed research. When a peer review process involves a range of experts, it needs to result in a consensus opinion about the quality of the research.
  • Intellectual capital in the research proposal is respected: A peer reviewer is in a privileged position, through having access to the unexploited ideas and intellectual capital of the researcher. A peer review process should require that reviewers do not disclose the substance of any research proposal, unless they have explicit permission to do so.

[1] For more information see part 11 of the Guideline on the Regulation of Therapeutic Products in New Zealand: Clinical Trials – Regulatory Approval and Good Clinical Practice Requirements.

[2] Co-production –‘citizens are not only consulted, but are part of the conception, design, steering, and management of services’- Christian Bason Leading public sector innovation: Co-creating for a better society, Bristol, Policy Press, 2010 or see Wiewiora A, Keast R, Brown K. Opportunities and Challenges in Engaging Citizens in the Co-Production of Infrastructure-Based Public Services in Australia. Public Management Review 2016; 18:483–507.

[3] Participatory research “besides the mere participation of co-researchers in the inquiry, participatory research involves a joint process of knowledge-production that leads to new insights on the part of both scientists and practitioners” - Bergold, Jarg & Thomas, Stefan (2012). Participatory Research Methods: A Methodological Approach in Motion [110 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 13 (1). Art. 30.

[4] Staged applications involves returning to the ethics committee through each stage of the research process, as aspects of the study may not be known during the first review.

[5] For further information on developing and designing protocols, see:

[6] The Ministry of Health, through the Health Information Standards Organisation (HISO), publishes standards for the New Zealand health and disability sector.

[7] Please note an additional legal requirement to obtain approval to use cervical screening ethnicity data under Part 4A of the Health Act and the Health (Cervical Screening (Kaitiaki)) Regulations 1995.