### General responsibilities

The entire review team is responsible for making a thoroughly described protocol publicly available before starting the systematic review. This is clearly recommended in The Cochrane Handbook of Systematic Reviews of Interventions [2]. The following sections should be described in full in the protocol with reference to relevant studies that substantiates the decision made during the protocol writing phase.

Remember the certainty of the evidence from a meta-analysis will always depend on more than a Trial Sequential Analysis (TSA) showing that the required information size has been reached.

## Trial Sequential Analysis

Clearly state that TSA will be used in the systematic review meta-analysis and what version of the software that is used. Describe all details about the analysis in the protocol (see below).

If the `R`

version of TSA is used, one can find the version via:

```
library(RTSA)
packageVersion("RTSA")
#> [1] '0.2.2'
```

## Outcome

All outcomes intended to be analysed with TSA should be clearly defined in the protocol. At this point it is advisable to consider if these are continuous or dichotomous outcomes. Please be aware that the TSA cannot be calculated with standardised mean differences (SMD).

For dichotomous and continuous outcomes, the following needs to be defined:

### alpha-level

Normally, meta-analyses will operate with the conventional 5% significance limit. Multiple testing of outcomes is at risk of increasing the type I error as seen when performing multiple interim analysis in randomised clinical trials [3]. Furthermore, if systematic reviews are investigating more than one outcome or investigating the same outcome at different time points, the risk of type-I-errors increases [4, 5].

Therefore, it is recommended to adjust the alpha level according to the number of outcomes going to be analysed. There are different ways to do this, with the Bonferroni correction being the most conservative. Jakobsen and colleagues [4] suggests dividing the conventional alpha level with the number between no correction (1) and the Bonferroni correction (number of outcomes). So, when three primary outcomes are defined the conventional alpha level should be divided by 1.5, if four outcomes exist divide by 2 and so on [4].

In `RTSA`

the alpha level is set in the function by the argument alpha. For a significance level of 5%, which is also the default value, set:

`RTSA(alpha = 0.05, ...)`

The alpha-level chosen for each outcome should be clearly described in the protocol.
### Level of power

Traditionally, systematic reviews are not concerned with the level of power [6]. As new recommendations in the GRADE system emerges, it is evident that an optimal required information size is needed to judge imprecision and the power of the analysis [7–9]. Normally, a power of 80% is accepted but for a more precise estimate a 90% power or higher is recommended.

In `RTSA`

the power is set in the function by the argument beta, where \(Power = 1 - beta\). Hence for 90% power, which is also the default value, set:

`RTSA(beta = 0.1, ...)`

The level of power should be clearly described for each outcome in the protocol.

### Heterogeneity

Heterogeneity can be present in meta-analysis, which means that there is between-trial variation. This changes the interpretation of the meta-analysis and the sample size and trial size requirements for the meta-analysis to reach the wanted level of power.

For traditional meta-analysis it has been custom to use the inconsistency (\(I^2\)) for estimating heterogeneity [2]. This can also be used in the TSA, but we recommend to calculate and adjust for diversity (\(D^2\)) when using the original stand-alone TSA software [10]. Diversity is a more conservative estimate for heterogeneity affecting the required information size if heterogeneity is present. The TSA will then present a diversity adjusted required information size (DARIS) [10]. For meta-analysis where the required information size is not reached and a 0% heterogeneity is found, a predefined sensitivity analysis adjusting for diversity between 25% to 50% could be relevant. Including more future trials will increase the risk of an increase in heterogeneity.

It has been found in [16], that it is not always sufficient to adjust for Diversity as it only increases the number of participants. In `RTSA`

the meta-analysis is per default adjusted by the estimated heterogeneity, if the meta-analysis is a retrospective meta-analysis and the meta-analysis is using a random-effects model.

`RTSA(fixed = FALSE, tau2 = 0.05, ...) # adjusted by a specific size of heterogeneity`

If one wants to explore different levels of diversity, say 50%, the following can be achieved by:

`RTSA(fixed = FALSE, D2 = 0.5, ...) # adjusted by diversity`

How heterogeneity is going to be incorporated in the calculation of random-effect meta-analysis should be clearly described in the protocol.

## Dichotomous outcomes

The following variables should be predefined in the protocol before conducting a TSA for each dichotomous outcome.

### Proportion of events in the control group (Pc)

For calculating the TSA for a dichotomous outcome, a proportion of events in the control group must be defined. Ideally this is derived from a previous well conducted and large trial or systematic review with low risk of bias [1]. If such a trial does not exist a large observational study at low risk of bias can be used. Data from the control group of the conducted meta-analysis can also be used if the mentioned data are not accessible.

In `RTSA`

the proportion of events in the control group is set using the argument `pC`

:

`RTSA(pC = 0.1, ...)`

How the probability of event in the control group was decided upon should be predefined in a protocol.

### Relative risk reduction (RRR)

The relative risk is an estimate of the effect of reducing the risk in the intervention group over the risk in the control group [6, 11]. If data from previous trials or systematic reviews at low risk of bias exists, these trials’ can be used to estimate the RRR. Usually in health research relatively small benefits are gained from one intervention. But very small RRR requires very large required information sizes (RIS). On the other hand, a large RRR can underestimate the power needed in the meta-analysis and increase the risk of random errors. Nevertheless, RRR should always be a realistic estimate.

In `RTSA`

for relative risks, both the argument `RRR`

or the argument `mc`

can be used. For a relative risk reduction of 20%, set `RRR = 0.2`

or `mc = 0.8`

(RR of 80%):

```
RTSA(RRR = 0.2, ...)
RTSA(mc = 0.8, ...)
```

How one decides on the size of RRR should be specified in the protocol.

### Zero events

In dichotomous outcomes zero events may occur in one or more groups of the included trials. In the protocol it is important to define how these events are handled. RevMan 5.1 [12] does by default replace a zero event with a 0.5 value. In the TSA in Java, higher or lower values can be defined by the authors using continuity adjustment techniques such as constant, reciprocal or empirical [13].

In `RTSA`

the default zero adjustment is 0.5:

`RTSA(zero_adj = 0.5, ...)`

How to handle zero events should be specified in the protocol.

## Continuous outcomes

For continuous outcomes the following should be predefined in the protocol besides the alpha-level and level of power.

### Minimally relevant difference (MIREDIF)

The minimally relevant difference is the change on a continues scale that is clinically relevant for the patient to experience. TSA takes the MIREDIF into account when calculating a RIS for a continuous outcome. MIREDIF should be set to the minimal clinical relevant difference. If this is not of interest, MIREDIF can be derived from previously conducted randomised clinical trials on patients with the same conditions.

In `RTSA`

the argument to specify MIREDIF is `mc`

, for a MIREDIF of 5:

`RTSA(mc = 5, ...)`

MIREDIF should be specified in the protocol.

### Variance

The variance is the spread of numbers in a dataset. Again, the variance of an outcome can be derived from previous studies examining the same outcome in the same group of patients. If this is not possible, variance can be calculated as the squared standard deviation.

In `RTSA`

the standard deviation is used:

`RTSA(sd_mc = X, ...)`

The expected variance should be specified in the protocol.

## Preparing the TSA

### Calculating the TSA-adjusted confidence interval (CI)

The TSA-adjusted CI gives a wider CI adjusted for the variables mentioned above. TSA-adjusted CI is, along with the DARIS and HARIS tools for examining imprecision in the analysis. State clearly in the protocol if these will be applied.

## Primary, secondary, and sub-group analysis

Trial Sequential Analysis is typically performed on the more decisive outcomes in the systematic review such as primary or secondary outcomes chosen to investigate benefits and harms of an intervention. If predefined sub-groups are planned (e.g. low compared to high risk of bias) and the authors wish to control these outcomes for type I and type II errors, a thorough description of this should be given in the protocol.