Introduction to Meta-Analysis: A Guide for the Novice
Pharmaceutical companies use meta-analysis to gain approval for new drugs, with regulatory agencies sometimes requiring a meta-analysis as part of the approval process. Clinicians and applied researchers in medicine, education, psychology, criminal justice, and a host of other fields use meta-analysis to determine which interventions work, and. Meta-analysis leads to a shift of emphasis from single studies to multiple studies. It emphasizes the practical importance of the effect size instead of the statistical significance of individual studies. This shift in thinking has been termed "meta-analytic thinking". The results of a meta-analysis .
Metrics details. The effect of cognitive behavioural therapy for psychosis CBTp on the core symptoms of schizophrenia has proven contentious, with current meta-analyses finding at most only small effects. However, it has been suggested that the effects of CBTp in areas other than psychotic symptoms are at least as important and potentially benefit from the intervention.
We meta-analysed RCTs investigating the effectiveness of CBTp for functioning, distress and quality of metq- in individuals diagnosed with schizophrenia and related disorders. Data from 36 randomised controlled trials RCTs met our inclusion criteria- 27 assessing functioning participants ; 8 for distress participants ; and 10 for quality of life participants.
The pooled effect size for functioning was small but significant for the end-of-trial 0. Although a small benefit of CBT was evident for reducing distress 0. Finally, CBTp showed no whah for improving quality of life 0. CBTp has a small therapeutic effect on functioning at end-of-trial, although this benefit is not evident at follow-up.
Although CBTp produced a small benefit on distress, this was subject to possible publication bias and became nonsignificant when adjusted.
We found no evidence that CBTp increases quality of life post-intervention. Peer Review reports. The first use of cognitive therapy to help people with schizophrenia was in [ 1 ]. Beginning somewhat later, with Kuipers anapysis al. These trials have typically looked at the effectiveness of CBTp in improving the core symptoms of schizophrenia i. Recent meta-analyses of these trials have converged on finding symptomatic improvement that is in the small range e.
The most comprehensive of these meta-analyses - that of Jauhar et al. They argued that this view of CBTp was inappropriate and that the intervention was more likely to have a distinctive profile of effects that are complementary to rather than substituting for drug treatment.
These should include interventions to improve symptoms but also those that address vulnerability, which are embedded in developmental processes. The aims, therefore, include: reduction of distress associated with psychosis symptoms… promoting social and educational recovery; reducing depression and social anxiety how to repartition c drive in windows 7 and relapse neta- p.
Similar sentiments are expressed in guidelines from elsewhere in the world, e. Nevertheless, the effect of CBTp on non-symptomatic outcomes in schizophrenia has been relatively less investigated than its effect on symptoms. Nearly 10 years ago, Wykes et al. They did not examine effect sizes for any follow-up period. The small numbers of trials analysed however, limits the reliability of findings from some of the NICE meta-analyses.
The other main limiting factor concerning the meta-analyses by Wykes et al. No meta-analysis appears to have examined the effects of CBTp on distress. The aim of the series of meta-analyses reported here was to determine whether evidence shows that CBTp improves aspects of the patient experience beyond symptom-reduction.
Based on there being enough trials to permit meaningful pooling of data, we selected three outcome variables: functioning, distress and quality of life. We initially considered the 52 RCTs retreived by Jauhar et al. We also searched the trials previously excluded by Jauhar et al. Searches were unrestricted regarding language and whether material was published or unpublished. We also searched through reference sections of papers that were considered eligible. Multiple searches were conducted using the following terms and combinations of terms:.
This search produced a further 16 what is meta- analysis in psychology. All anqlysis studies were then hand-searched by one of us ND for the outcome measures of interest and counter-checked by another KRL. Our inclusion criteria paralleled those used by Jauhar et al. Thus, studies were included if a majority of the patients had a diagnosis of schizophrenia, schizoaffective or non-affective functional psychosis, either made clinically or according to diagnostic criteria.
Trials could use any measure of functioning, distress or quality of life for details, see below. Studies also had to include a parallel control group of any type, i.
We naalysis non-randomised trials and those which used inappropriate randomisation methods e. The four non-randomised trials that were located all also used non-blinded outcome assessment and were low in overall quality see [ 16171819 ].
Determination of what types of therapy constituted Psycholgoy was relatively broad and followed Jauhar et al. Following previous meta-analyses, we did not include studies that delivered CBT as part of a multicomponent package of care that involved several other interventions sometimes referred to as integrated treatment or similar. We included trials using both individual and group CBTp. Studies were included if they measured the distress associated with the symptoms of psychosis.
Outcomes relating to depression and anxiety alone were not included as these were considered to represent symptomatic measures. Pooled effect sizes for the data were created using Comprehensive Meta-analysis, version 2 [ 38 how to cover brick fireplace. A random-effects model was used in all analyses.
Effect sizes were derived from the post-intervention or follow-up scores using Ia g i. When these data were not available in a paper, authors were contacted. Heterogeneity how to build a book binding press examined with Q and I 2 statistics. Moderator analyses, where feasible, followed Jauhar et al.
Twenty-six samples assessed functioning, 8 assessed distress analysos 10 quality of life. See Table 1 for excluded studies and main reason for exclusion. Functioning was assessed in 25 trials with 26 samples: see Additional file 1 providing a total of participants received CBTp and what is urgent care copay in the control condition.
Of im 26 samples, 17 compared CBTp to treatment as usual TAUwhile the remaining 9 compared it to another intervention psychoeducation, befriending, cognitive remediation, social activity therapy, wbat therapy, goal focused supportive contact. The pooled effect size for functioning across 26 samples was 0. Forest plot for post-intervention scores on functioning. Edwards et al. Clozapine and Thioridazine respectively.
We compared 19 studies where assessors were blinded analysiz to treatment condition with 7 where assessment was not blinded unmasked to the treatment group. The unmasked trials revealed a small and significant effect size of 0. The masked trials revealed a small significant effect size of 0.
We compared 19 trials using treatment as usual TAU as a control versus 7 trials using active control conditions. The effect size for TAU was significant at 0. The effect size for trials with an active control was nonsignificant at 0. Follow-up data were available in 16 of the trials, with a median follow-up time of 12 months range 3—18 months. The pooled effect size for What is meta- analysis in psychology on functioning at follow-up was nonsignificant 0.
Distress was analysed in 8 studies see Additional file 2 with a total sample size of receiving CBTp and in control conditions. Of these studies, 7 were against a treatment as usual TAU and 1 was against a waitlist control. The pooled effect size was significant at 0. The forest plot is shown ehat Fig. Quality of life was assessed in 10 samples from 9 trials see Additional file 3 with a total sample size of received CBTp and in the control condition. CBTp had no significant impact on quality of life, with an effect size close to zero at 0.
The forest plot in Fig. The five trials examining QoL under blind conditions had a nonsignificant mean effect size of 0. As noted in the introduction, while more than a dozen meta-analyses have examined whether CBTp reduces the positive and negative symptoms of schizophrenia, non-symptomatic outcomes have been somewhat neglected.
Two previous meta-analyses — both now a decade old — have examined the impact of CBTp on functioning [ 1114 ] but what is meta- analysis in psychology is the first to examine the impact of CBTp across a range of non-symptomatic outcomes, including: functioning at end-of trial and follow-up and the impact on quality of life and distress.
Although a small benefit of CBTp for functioning emerged at end-of-trial, this was non-significant at psychokogy. In 8 trials, Pyschology was found to produce a small significant reduction in distress; however, evidence of potential publication bias led to the imputing of 3 studies, halving the effect size and making it non-significant. The effect was also moderated by blinding — significant whaf reduction was only found in trials using non-blind outcome assessment.
Quality of life was unaffected by CBTp and indeed, none of 10 samples documented a significant benefit. With respect to functioning, our effect size of 0. Importantly, more recent studies also included large well-controlled trials e. Furthermore, our analysis of follow-up data derived from 16 samples revealed that CBT did not significantly improve functioning. This latter finding contrasts with the findings reported by NICE; it seems likely that this reflects the fact that the current meta-analysis is much larger - involving four times as many trials.
Our findings provide an important update on the multiple meta-analyses carried out for NICEwhich was on small analysiw of trials and produced mixed findings. NICE have still failed to update their meta-analyses, which contain no trials post; and so, it might seem an appropriate time to update their analyses and potentially, their recommendations given the findings here. With an effect size that was close to zero, we found no suggestion that CBTp improves quality of life in people diagnosed with schizophrenia.
Our findings accord with earlier smaller analyses of quality of life by NICE [ 11 ] and the Cochrane Collaboration [ 6 ], both of which found no evidence of CBTp being efficacious for this outcome.
Indeed, every published trial has reported a nonsignificant effect of CBTp on quality of life; particularly noteworthy is one trial by van der Gaag et al. Only 8 in 67 RCTs that met our eligibility criteria reported distress as an outcome and this was always as a secondary measure. Although significant at 0. Also noteworthy is that several RCTs assessing distress had small samples and so their power to detect true small effects is likely to be low.
Following Button et al. Doing this revealed that the power in CBTp trials assessing distress was low at. The low level of power also accords with the evidence of potential publication bias in trials measuring distress; and may reflect the publishing of unreliable small trials with positive, but what channel is the heat celtics game on tonight negative results.
Future studies of distress would need four times the current mean sample size of 40 per group to reliably detect the effect size reported in existing trials.
Apr 01, · This meta-analysis shows the most promising findings for stress. The observed effect of online MBIs on stress, including the outlier, is comparable to the effect size found for traditional MBSR and MBCT (d = ) as found in a recent systematic review and meta-analysis of systematic reviews of RCTs (Gotink et al., ). Jul 17, · Our meta-analysis is the first to assess whether CBTp improves quality of life or reduces distress in individuals diagnosed with schizophrenia. We also present an updated meta-analysis assessing the impact of CBTp on functioning. On current evidence CBTp leads to a small improvement in functioning which, however, is not sustained. Therefore, our objective was to conduct an effect size analysis of this popular intervention for anxiety and mood symptoms in clinical samples. Method: We conducted a literature search using PubMed, PsycINFO, the Cochrane Library, and manual searches. Our meta-analysis was based on 39 studies totaling 1, participants receiving mindfulness.
MetaXL Version 2. Opposing theories and disparate findings populate the field of psychology; scientists must interpret the results of any single study in the context of its limitations. Meta-analysis is a robust tool that can help researchers overcome these challenges by assimilating data across studies identified through a literature review.
In other words, rather than surveying participants, a meta-analysis surveys studies. Despite the utility of this statistical technique, it can intimidate a beginner who has no formal training in the approach. However, any motivated researcher with a statistics background can complete a meta-analysis.
This article provides an overview of the main steps of basic meta-analysis. Meta-analysis has many strengths. First, meta-analysis provides an organized approach for handling a large number of studies. Third, meta-analysis allows researchers to examine an effect within a collection of studies in a more sophisticated manner than a qualitative summary.
However, meta-analysis also involves numerous challenges. First, it consumes a great deal of time and requires a great deal of effort. Second, meta-analysis has been criticized for aggregating studies that are too different i.
Third, some scientists argue that the objective coding procedure used in meta-analysis ignores the context of each individual study, such as its methodological rigor. Fourth, when a researcher includes low-quality studies in a meta-analysis, the limitations of these studies impact the mean effect size i.
As long as researchers are aware of these issues and consider the potential influence of these limitations on their findings, meta-analysis can serve as a powerful and informative approach to help us draw conclusions from a large literature base. Similar to any research study, a meta-analysis begins with a research question. Meta-analysis can be used in any situation where the goal is to summarize quantitative findings from empirical studies. It can be used to examine different types of effects, including prevalence rates e.
To select the effect metric, researchers should consider the statistical form of the results in the literature. Any given meta-analysis can focus on only one metric at a time.
While selecting a research question, researchers should think about the size of the literature base and select a manageable topic. At the same time, they should make sure the number of existing studies is large enough to warrant a meta-analysis. After choosing a relevant question, researchers should then identify and explicitly state the types of studies to be included.
These criteria ensure that the studies overlap enough in topic and methodology that it makes sense to combine them. The inclusion and exclusion criteria depend on the specific research question and characteristics of the literature. First, researchers can specify relevant participant characteristics, such as age or gender. Second, researchers can identify the key variables that must be included in the study. Third, the language, date range, and types e.
Fourth, pertinent study characteristics, such as experimental design, can be defined. Eligibility criteria should be clearly documented and relevant to the research question. Specifying the eligibility criteria prior to conducting the literature search allows the researcher to perform a more targeted search and reduces the number of irrelevant studies.
Eligibility criteria can also be revised later, because the researcher may become aware of unforeseen issues during the literature search stage.
The next step is to identify, retrieve, and review published and unpublished studies. The goal is to be exhaustive; however, being too broad can result in an overwhelming number of studies to review. Online databases, such as PsycINFO and PubMed, compile millions of searchable records, including peer-reviewed journals, books, and dissertations. In addition, through these electronic databases, researchers can access the full text of many of the records. It is important that researchers carefully choose search terms and databases, because these decisions impact the breadth of the review.
Additional ways to identify studies include searching conference proceedings, examining reference lists of relevant studies, and directly contacting researchers. After the literature search is completed, researchers must evaluate each study for inclusion using the eligibility criteria.
At least a subset of the studies should be reviewed by two individuals i. It is vital that researchers keep meticulous records of this process; for publication, a flow diagram is typically required to depict the search and results.
Researchers should allow adequate time, because this step can be quite time consuming. Next, researchers calculate an effect size for each eligible study. The effect size is the key component of a meta-analysis because it encodes the results in a numeric value that can then be aggregated. The effect size metric is based on the statistical form of the results in the literature and the research question.
Because studies that include more participants provide more accurate estimates of an effect than those that include fewer participants, it is important to also calculate the precision of the effect size e. Meta-analysis software guides researchers through the calculation process by requesting the necessary information for the specified effect size metric.
I have identified some potentially useful resources and programs below. Although meta-analysis software makes effect size calculations simple, it is good practice for researchers to understand what computations are being used.
The effect size and precision of each individual study are aggregated into a summary statistic, which can be done with meta-analysis software. Researchers should confirm that the effect sizes are independent of each other i. Additionally, researchers must select either a fixed effects model i.
The random effects model is typically preferred when the studies have been conducted using different methodologies. Depending on the software, additional specifications or adjustments may be possible. During analysis, the effect sizes of the included studies are weighted by their precision e. This statistic is typically accompanied by an estimate of its precision e. Forest plots are a common way of displaying meta-analysis results.
Depending on the situation, follow-up analyses may be advised. Researchers can quantify heterogeneity e. Moderator variables, such as the quality of the studies or age of participants, may be included to examine sources of heterogeneity. Because published studies may be biased towards significant effects, it is important to evaluate the impact of publication bias e. Sensitivity analysis can indicate how the results of the meta-analysis would change if one study were excluded from the analysis.
If properly conducted and clearly documented, meta-analyses often make significant contributions to a specific field of study and therefore stand a good chance of being published in a top-tier journal. The biggest obstacle for most researchers who attempt meta-analysis for the first time is the amount of work and organization required for proper execution, rather than their level of statistical knowledge. Borenstein, M.
Introduction to meta-analysis. Hoboken, NJ: Wiley. Cooper, H. The handbook of research synthesis and meta-analysis 2nd ed. Lipsey, M. Practical meta-analysis. Thousand Oaks, California: Sage Publications. Rothstein, H.
Publication bias in meta-analysis: Prevention, assessment, and adjustments. It is nice to see the software we developed MetaXL being mentioned. However, the reason we developed the software and made publicly available for free is that we disagree with an important statement in the review.
We developed MetaXL because we think that the random effects model is seriously flawed and should be abandoned.
We implemented in MetaXL two additional models, the Inverse Variance heterogeneity model and the Quality Effects model, both meant to be used in case of heterogeneity.
More details are in the User Guide, available from the Epigear website. Thank you very much! The article really helped me to start understanding what meta-analysis is about. But I am still confused about how to remove quickly duplicates papers without wasting time if we more than 10 papers?
Not being one to blow my own horn all the time, but I would like to suggest that you may want to take a look at a web based application I wrote that conducts a Hunter-Schmidt type meta-analysis. The Meta-Analysis is very easy to use and corrects for sampling and error variance due to reliability. It also exports the results in excel format. You can also export the dataset effect sizes r, d, and z , sample sizes and reliability information in excel as well.
APS regularly opens certain online articles for discussion on our website. Effective February , you must be a logged-in APS member to post comments. By posting a comment, you agree to our Community Guidelines and the display of your profile information, including your name and affiliation. Comments will be moderated.
For more information, please see our Community Guidelines. Laura C. Her main area of expertise is post-trauma functioning, particularly in survivors of sexual violence or mass trauma e. She also has interest in predictors of violence and aggression, including psychophysiological and personality factors.
NIH has issued a Request for Information asking the community to weigh in on a number of questions related to basic behavioral science, and NIH needs to hear from individual scientists like you that basic human subjects research should not be classified as clinical trials. Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.