OpenMEE is open-source software for performing meta-analysis suited to the need of ecologist and evolutionary biologists. This program is made possible by the National Science Foundation (NSF), award numbers: DBI-1262402 (J. Gurevitch), DBI-1262442 (B. Wallace), DBI-1262545 (M.J. Lajeunesse).
Launch the program by clicking on LaunchOpenMEE.exe in the case of Windows or, on Mac, the OpenMEE app. This will open the main spreadsheet. Sometimes this can take a few seconds.
You will enter and interact with your data via this spreadsheet. From this spreadsheet you can enter your data in the following ways:
The latter two are accessible via the File menu.
The OpenMEE spreadsheet is very flexible. It has an unlimited number of rows and columns. The spreadsheets starts off with a fixed (large) size. As your data approaches the limits, the spreadsheet automatically resizes itself to accomodate more data. You also don't have to worry about formatting your data beforehand. Simply enter your data in whatever format you have it. For example, After typing in some two-by-two contingency table data we see the screen below.
Along the top of the spreadsheet we see columns labelled 'A', 'B', 'C', etc., along with '(cat)'. When you enter data into a new column, OpenMEE assumes by default that said column is a new categorical variable and assigns it a default name. When you are done typing in new data, you have to tell OpenMEE what kind of data type each column contains (e.g., continuous). You also must mark one column as the study labels column.
In addition to telling OpenMEE the types of each column, you may also assign your own custom labels (names) to each column. All this is done by right-clicking on column headers and choosing the desired option as shown below:
We now have to tell OpenMEE what type of data these columns contain. Again, this is done by right-clicking and selecting a target format from the Change format submenu as shown in Figure 4. Since this is two-by-two data, the four numerical columns are 'counts' while 'Country' and 'Year' are categorical variables, and 'study_name' is the variable we will mark as containing study labels (by selecting 'Mark as label column' from the context menu). Finally out of pedagogical necesssity, let's add a continuous column after country called 'Avg. Rainfall (cm)'. After assigning all these formats, we see
Note that for each variable format has its own color scheme as reminder when you're working with the spreadsheet. If you can't stand the default color scheme you can change it by clicking on the Table menu and then Table Preferences:
Here, you can change the color schemes of the variables as well as the font used on the spreadsheet as well as the number of digits displayed for continuous variables.
Before we can run an analysis, we need effect sizes and associated variances. These can be input in one of two ways:
To do this, simply create two new continuous variables using the method described above. Let's call them "my effect size" and "my variance". Now we need to tell OpenMEE that these are effects. Right-click on the header for "my effect size" and choose Change format → Trans. Effect. Likewise, choose 'Change Format → Trans. Var' for "my variance" as shown in the Figure below.
We will then see the screen below
Here we need to discuss the issue of 'transformed' scale vs. 'raw' scale. To perform a meta-analysis, we need an effect size and a measure of dispersion for each study. For example, if our metric is the odds ratio, we need the effect size on the transformed (log) scale because confidence intervals for odds ratios are generally not symmetric in the raw (untransformed) scale. Fortunately, OpenMEE can transform an effect size and confidence interval (or effect size and variance) from one scale another.
To calculate an effect size from raw data, click the 'Calculate Effect Size' button on the toolbar, or select 'Calculate Effect Size' from the analysis menu. This will show the wizard in Figure 10. First, you have to select a data type and a metric.
Next tell OpenMEE the locations of the necessary variables as show in Figure 11. You will be given the option to link the raw data columns to the resulting effect size columns (selected by default). This makes it so that when you change a value in the raw data, the effect column will update automagically.
In Figure 12, we see the calculated effect size and variance in the 'transformed' (natural log) scale. Also, note that on the bottom of the window we see a red bar. This provides a visual cue that the four raw data columns cont. #events, #cont #noevents, etc are 'linked' to the effect and variance columns.
In Figure 13, we see the result of transforming the effect size. Three new columns have been added containing the 'raw scale' effect size and upper and lower confidence bounds given the current confidence level shown in the toolbar. Changing the confidence level in the toolbar will cause the confidence interval to be recalculated.
In some cases, one may already have data that one would like to import for analysis. OpenMEE supports importing comma (and otherwise) delimited data. To achieve this, select Import CSV from the File menu. You will be prompted with a screen asking for details about your file format and its location, like so:
Once a file to be imported has been selected, a preview of how it will be read is shown in the small window. Note the field for special strings that denote missing data (you may want to use "NA" here, if importing from R).
Exporting data follows the reverse of the above (select Export CSV rather than Import CSV). Note that all calculated effects and variances will also be exported. Most programs, including Excel (for example), can read CSV's.
All analyses are performed similiarly. You can choose to perform an analysis by clicking its icon in the toolbar or by selecting it from the Analysis menu. More exotic analyses (such as 'bootstrapped' meta regression, meta-regression-based conditional means and boostrapped meta regression-based conditional means) are only accessible from the menu bar.
Here we describe the pages one sees on the wizards for each analysis (this is similar for all analyses).
Here we refer to analyses that look to estimate a 'grand mean', i.e., with no covariates. The pages you will see are as follows.
Cumulative meta-analysis is a meta-analytic approach in which studies are added one at a time, and the change in the cumulative effect size is observed. To perform a cumulative meta-analysis in OpenMEE, simply select Cumulative Meta-Analysis from the Analysis menu. We show a sample forest plot generated from a cumulative meta-analysis below. The left-hand side displays the usual study estimates and confidence intervals; the right hand side shows the effect on the overall (summary) estimate as studies are included in the meta-analysis.
Leave-one-out meta-analysis is conceptually similar to cumulative meta-analysis, except that instead of adding studies one at a time, one holds each study out in turn. This is an exploratory exercise that may highlight influential studies (summary estimates that leave influential studies out will differ substantially than those that include them). An example of output from a leave-one-out analysis is shown below.
In subgroup meta-analysis, one partitions studies into disjoint groups (e.g., studies conducted in China versus all others) and runs separate meta-analyses over these groups of studies. This is an exploratory exercise that may highlight differences between groups. The options for subgroup meta analysis are the same as for standard meta analysis except for a categorical variable must be selected. For example, if the selected variable is state, then subgroup analyses will be run on studies from each observed state. In the example shown below, this includes Massachusetts (MA) and Rhode Island (RI).
Meta-regression allows one to explore the relationship between covariates and study outcomes. OpenMEE supports mulitple regression and both continuous and categorical predictors (covariates).
On the 'Select covariates for regression page', select the covariates for regression. On this same page, one can choose whether you want to use a random or fixed effects model and the confidence level to use in the analysis. An example of this is shown below; here we have selected "Avg. Rainfall" to be included in the regression.
For any categorical covariates that have been selected to be included in the meta-regression, one needs to specify the value that will serve as its reference value (i.e., the intercept). We show an example below where the covariate is Country and the reference value has been specified as USA.
We note that one can also assess the variance of regression coefficient estimates via bootstrapping (see below). To do so, simply select 'bootstrap' rather 'parametric' under 'Type of Analysis' when specifying meta-regression options. We show an example of bootstrap output for the rainfall predictor below.
OpenMEE can compare two regression-models in terms of model-fit statistics and a likelihood ratio test. This is a strategy to assess the informativeness of a given (set of) predictors.
The process to do this model building comparison is very similiar to that of meta-regression. The only differences are that the covariates you select on the covariate select screen will be the covariates used to construct the Full model. You then select a subset of these covariates/interactions to use for the reduced model as shown in the figure.
Bootstrapping is a non-parametric approach for assessing the variance of a parameter estimate. OpenMEE allows for bootstrapped estimates both for meta-regressions and standard meta-analysis.
In general, the options for boostrapped meta-analysis are the same as those for standard meta-analysis, with the addition that one needs to provide bootstrap-specific parameter values. These include the number of bootstrap replicates to perform, for example. We show sample output from a boostrapped meta-analysis below.
When running bootstrap operations in OpenMEE, the program tries to generate results using the selected number of bootstrap replicates. We note that depending on the dataset and the meta-anaytic (pooling) model being used, a meta-analysis of a specific sample of the dataset may fail. OpenMEE hands this in a straight-forward way: if one particular replicate attempt fails, another is tried, and so on until 5 times the selected of bootstrap replicates have been attempted, at which point an error message is returned.
Sometimes data elements are missing from studies. OpenMEE supports basic imputation of covariates for meta-regression. To use this, select Multiple Imputations Meta-Regression form the Analysis drop-down menu. You will be prompted with the following screen
These parameters specify the imputation method to use and the number of multiple imputations to perform. Providing the methodological details of the imputation is beyond the scope of this document, but we refer the interested reader to the R MICE documentation for more details: we use this package 'under the hood' to perform the imputation.
OpenMEE offers various tools to help you visually explore your data including scatterplots and histograms.
OpenMEE includes the standard tools for publication bias diagnostics, including funnel plots. These can be generated by selecting the Funnel Plot option under the Publication Bias sub-menu of Data Exploration. Example output is shown below. We caution users to interpret these with care, and to read this criticism of funnel plots.
Under the same Publication Bias sub-menu, one can find an option to calculate fail-safe N
.One may wish to export the results from analyses for input into different programs (e.g., Microsoft Word). OpenMEE facilitates this in a few different ways. For images (e.g., forest plots), perhaps the easiest thing to do is to right-click and select "save png image as..." from the context menu, as shown below.
Using this approach, one may save the image to wherever is convienent and then incorporate it into documents. Note that we also support saving as PDF; this is in many cases preferable to PNG, because PDF's are vector graphics. These are to be preferred for publications.
Concerning textual output, such as tables, one can similarly use the contextual menu that is accessible via right-click. Specifically, this involves right-clicking on the textbox of interest and choosing "select all", and then right clicking again and selecting "copy". Results can then be pasted into, e.g., a Word document. Alternatively, one may instead use the "Export Results as Text Button" button:
This will save all displayed textual output to a file of the user's choice.
OpenMEE relies on R to perform its analyses. The casual user need not ever be aware of this in the course of his analyses. However, If desired you can see the commands being sent to R in real-time and even interact with the R console. You can also log the R commands to a text file. All this can be done by selecting 'R output viewer' in from the 'Through the looking glass menu'. Then, as you interact with program, the commands being sent to R will appear in the window. This can be useful to automate reproduction of results.
You can interact with objects in the R workspace by typing commands in to the 'Command Injection' area of the 'R output viewer' window and then pressing enter or pushing the 'Inject Command' button.