Supplementary material 4 to: Focused review format prototype

Flemyng E, Mitchell D
https://doi.org/null

The material in this section has been supplied by the author(s) for publication under a Licence for Publication and the author(s) are solely responsible for the material. Cochrane has reviewed this material, but Cochrane has not copyedited, formatted or proofread. Cochrane accordingly gives no representations or warranties of any kind in relation to, and accepts no liability for any reliance on or use of, such material.

Back to top

Risk of bias

Table of contents

Risk of bias

Risk of bias for analysis 1.1 Treatment failure: need for systemic corticosteroids (all randomised participants)
StudyBias
Randomisation processDeviations from intended interventionsMissing outcome dataMeasurement of the outcomeSelection of the reported resultsOverall
Authors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgement

FitzGerald 2004

Low risk of bias

Patients were randomised to treatment groups at visit 2 according to a blocked computer generated randomisation list for each centre. As a randomisation list was generated for each centre, this sounds like it was centrally generated.
Baseline characteristics not shown for the full population, only those who had an exacerbation and started the study inhaler. Paper states that "there were no differences in patient characteristics between those who
experienced an exacerbation and those who did not".

High risk of bias

Methods state 'double blind' so assume this means participants.
The participants themselves delivered the intervention.
For the purposes of the review, the denominator for the analysis was all randomised participants which assumes no one in the study deteriorated to the point of needing oral steroids without first meeting the criteria for initiating the study inhaler and doing so. Most people did not start the study inhaler, so there is potential for the assumption to have a substantial impact on the result.

Some concerns

Number of participants lost to folLow risk up were larger than the number of events and only participants who started study inhaler were included in the analyses.
No analysis methods that correct for bias due to missing outcome data mentioned.
Patients who discontinued who did not use the study inhaler might have done so for reasons relating to their disease worsening, but the CONSORT diagram does not suggest that this was the case although the most common reason is 'other' in both groups.

Low risk of bias

Method of measuring the outcome was appropriate.
Measurement of the outcome was the same between intervention groups.
The study participant (who was blinded) entered details into an electronic diary with thorough pre-defined criteria on the outcome

Some concerns

No study protocol or trial registration record is available.
Clear definition of outcome and timepoints.

High risk of bias

Used the tool algorithm.

Garrett 1998

Low risk of bias

Only information about the allocation is that the children were randomised by the hospital pharmacist and the investigators were blinded to this allocation.
Baseline characteristics were not provided by intervention group.

High risk of bias

Used a blinded inhaler delivered by the participant.
10/28 randomised participants not included in the analyses as they didn't have an exacerbation in both arms - therefore a per protocol analysis.
A large proportion of those randomised were exlcuded from the analyses.

High risk of bias

Only those with completed pairs of exacerbations were included in the final analysis - 10/28 excluded from the analyses as they only had an exacerbation in one arm.
No anlyses were reported to address bias from missing outcome data.
Reported reason for excluding from analyses was that they had good health status (no exacerabtions).

Low risk of bias

Method for measuring the outcome was appropriate, and unlikely to differ between groups.
Reported that investigators were blinded to the allocation.

Some concerns

No trial registration, study protocol or SAP (even though article mentions a study protocol).
The numerical result is not likely to have been selected on the basis of the results from multiple outcome measurements or analyses.

High risk of bias

Used algoritm.


Domain S = Low risk RISK OF BIAS
SQ1 = NI on whether the number of participants allocated to each of the two sequences equal or nearly equal.
SQ2 = Low risk - Cross-over data were included in the review by obtaining two-by-two data and applying a formula to account for inter-correlation of matched pairs (Elbourne 2002).
SQ3 = Low risk - Sufficient time was given for carryover effects to have disappeared before outcome assessment in the second period - folLow risked for two weeks and study tated that spirometry returned to normal within 1 week

Harrison 2004

Low risk of bias

An independent pharmacist randomly allocated individuals using computer generated random number tables.
Statistical tests for imbalances not presented but group differences at baseline do not suggest a problem with randomisation.

Low risk of bias

Study report does not explicitly state "double‐blind", but a placebo was used which implies that it was not open label, and text mentions "A range of active and placebo study inhalers was available to enable the type of inhaler and daily dose to be matched to patients’ regular inhaled corticosteroid, type of inhaler, and dose". Participants delivered the intervention themselves.



Used a IIT analysis.

Low risk of bias

Nearly all data available and number withdrawn much smaller than the number of events observed. 8.9 and 10.1% withdrew from the intervention and placebo groups respectively which is relatively Low risk and balanced, although the study does not report whether and how missing data were imputed.


Low risk of bias

Method for measuring the outcome was appropriate. Participants were given a 10-day course of prednisolone (30 mg per day) to be taken if their asthma control deteriorated to the point that they would usually start oral corticosteroids, or if their peak fLow risk fell by 40% from the mean run-in value.
Unlikely that the measurement of Measurement of the outcome was the same between intervention groups. Oral steroids were patient initiated and they were blinded to the contents of the study inhalers.
Outcome assessors were the study participants.

Some concerns

No publicly available trial registration, study protocol or SAP (SAP is mentioned in the study report).
Clear definition of outcome and timepoints

Low risk of bias

Overall risk of bias assessed as Low risk as the only 'some concerns' was due to no trial registration or study protocol details, but this is an old trial when making these available was not standard.

Jackson 2018

Some concerns

Method of sequence generation and allocation concealment not described (in paper, supplementary materials of protocol). However, the report mentions stratification which implies the sequence was computer generated.
Statistical tests for imbalances not presented but group differences at baseline do not suggest a problem with randomisation.

Low risk of bias

Participants and their carers/parents were blinded.
"The primary intention-to-treat analysis compared the overall efficacy of
each treatment strategy, regardless of whether any yelLow risk-zone episodes actually occurred."

Low risk of bias

Number of participants lost to folLow risk up is similar to the number of events; however, most participants were included in the study for most of the duration [mean folLow risk-up among all 127 participants in each group was 42.5 weeks and 40.3 weeks, respectively (i.e. most of the full 48 week folLow risk-up)].

Low risk of bias

Method of measuring the outcome was appropriate.
Measurement of the outcome was the same between intervention groups.
The outcome assessor was the study participant (who was blinded). However, systematic glucocorticoids were started after consultation with the trial clinician and it is unclear whether they were blind to the outcome.

Low risk of bias

Protocol dated February 2014 and trial enrolment began in August 2014.
The numerical result is not likely to have been selected on the basis of the results from multiple outcome measurements or analyses.

Low risk of bias

Insufficient reporting warrants an assessment of 'some concerns' for the randomisation process, but the overall reporting and conduct of the trial suggests that rigorous procedures were folLow risked to minimize bias.

Martinez 2011

Low risk of bias

The Data Coordinating Center generated the random allocation sequence via computer software.The DCC had no contact with participants.
No obvious imbalances between groups.

Low risk of bias

States double blind and placebo interventions were used.
Those delivering the intervention were the participants.



Number of treatment failures measured in the whole population

Low risk of bias

Withdrawals from the study were relatively Low risk and even between the two groups included in this review: 11.3% in the intervention group (double ICS) and 12.5% in the control group (stable ICS). However, in some groups the number of events was similar to the number of participants with missing data, therefore their outcomes could impact the estimated effect of intervention.

Authors didn't correct or assess for bias from missing outcome data.
Reasons for missing data were accounted for and most were not related to the ouctome of interest (only two participants across the study were lost because they were unsatisfied with asthma control, both in the rescue beclomethasone group).

Low risk of bias

Method for measuring the outcome was appropriate
Measurement of the outcome would not have differed between groups.
Drugs were masked A to D to mask statisticians to treatment group.

Some concerns

Trial registered before participant enrolment began (https://clinicaltrials.gov/ct2/show/study/NCT00394329). However, registry record contains no prospective information about statistical analyses and no protocol or SAP available.
The numerical result is not likely to have been selected on the basis of the results from multiple outcome measurements or analyses.

Low risk of bias

Overall risk of bias assessed as Low risk as the only 'some concerns' was due to no study protocol or SAP available. However, it was registered and no other details from the trial for this outcome gives us reason to believe that there are some concerns regarding risk of bias.

Oborne 2009

Low risk of bias

An allocation sequence of random permuted blocks of 10 was generated using a random number table by an independent pharmacist and implemented by one of the study investigators once participants were enrolled into the trial. Mentions an investigator providing the randomization schedule and concealed allocation of masked inhalers.
Inhalers were identical and participants blind to the intervention.
Baseline data provided for all participants and those starting the study inhaler - no issues noted.

Low risk of bias

Double-blind, placebo inhaler was used.
Participants were not aware of the intervention, which they adminstered.
An intention to trate analyses was used.

Low risk of bias

The number of events was less than the number of participants with missing. Thirty-eight drop-outs in active vs 39 dropouts in placebo vs 18 events in active vs 29 events in placebo. However, all participants contributed data for the intention-to-treat analysis up to the point at which they left the study. Only three were lost to folLow risk-up with no outcome data.

Low risk of bias

Method of measuring the outcome was appropriate.
Measurement of the outcome unlikely to have differed between intervention groups.
Doesn't state if outcome assessors were blineded (only investigators and participants in relation to how the inhaler looked)
Assessment of the outcome did not require judgement.

Some concerns

The trial was rgeistered (http://www.isrctn.com/ISRCTN46018181). It was registered after participant enrolment began but before recruitment was completed. The registry doesn't include anything about the anlyses and a protocol/SAP isn't available.
Clear definition of outcome and timepoints.

Low risk of bias

Overall risk of bias assessed as Low risk as the only 'some concerns' was due to no study protocol or SAP available. However, it was registered before recruitment completed and no other details from the trial from this outcome gives us reason to believe that there are some concerns regarding risk of bias.

Rice-McDonald 2005

Low risk of bias

Method of sequence generation not described but "order and allocation of treatment by concealed randomisation"
Does not report baseline characteristics for each intervention group.

High risk of bias

A double-blind, double-dummy study with order and allocation of treatment by concealed randomisation. The participants administered the intervention.
An intention to treat analysis was used, though only those with data for both periods were included. Of 35
randomized participants, 13 subsequently withdrew
prior to any asthma exacerbations.
A large proportion of those randomised were exlcuded from the analyses.

High risk of bias

Of the 35 people ranomised, 13 withdrew from the study (and unclear in what intervention or period they were in). Also, 11 participants had exacerbationsand went straight to oral corticosteroids (no study inhaler) as their symptoms were very bad (excluded from analysis and unsure which period and intervention each were in).
No analyses corrected for bias due to missing outcome data and a within-group change was assessed (not within person differences).
The reasons for withdrawal from the trial are not associated with the intervention or outcome. However, 11 participants who were randomised and had exacerbations that went straight to oral corticosteroids (no study inhaler) were not analysed and this missingness woudl depend on its true value.

Low risk of bias

Method for measuring the outcome was appropriate, and unlikely to differ between groups. Treatment failure well defined.
Outcome assessors were the particpants, who were blinded.

Some concerns

No trial registry, protocol or SAP available.
The numerical result is not likely to have been selected on the basis of the results from multiple outcome measurements or analyses.

High risk of bias

Algorithm used.

Domain S (for crossover trials) = Low risk
SQ S1 = the number of participants allocated to each of the sequences was nearly equal.
SQ S2 = N/A
SQ S3 = a four week washout period was used which is sufficient time for any carryover effects to have disappeared before outcome assessment in the second period.

Wainwright 2009

Low risk of bias

"Stratified block randomisation by age (3-5, 6-10, 11-14), gender, centre". "Sequential study number allocated from a list according to blocking details". "...blocking details emailed to Dept of Epidemiology and Preventative Medicine, Monash Med School, Melbourne". "Study puffer number was allocated. Pre-numbered puffers were held at RCH Brisbane pharmacy"
Imbalances noted in disease characteristics (High risker proportion of patients in the intervention group than the control group had persistent asthma and atopic history), but likely to be down to chance given sample size.

Some concerns

Placebo inhalers were used to presume this was to blind participants and personnel from the study medication, but no explicit definition of masking procedures



Outcome reported as percentages and unclear whether they related to the ITT or per protocol (those who started the inhaler) population. For the purposes of including data in the review, we assumed it was the latter and then used the full population numbers to include data for the primary outcome of the review. This assumes no one who didn't take the study inhaler needed oral steroids.
Most participants started the study inhaler but approx 25% of each group did not, so there is some potential for bias.

Low risk of bias

Outcome data accounted for and a much High risker number of events than those who withdrew. 11 in the intervention group (8.7%) an 10 in the control group (8.1%) withdrew from the study - Low risk and balanced. Only those having an exacerbation could be included in the main analyses.


Low risk of bias

The method of measuring the outcome was appropriate.
Measurment of the outcome likely to be the same between the groups. Clear instructions for what to do in the event of worsening symptoms and called weekly by study nurse - all events likely to have been recorded.
The outcome assessor was the study participant (who was blinded).

Some concerns

Trial registered (https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=630) but after participant enrollment was completed. No details about the analyses available. No published journal report.
The numerical result is not likelyt to have been selected on the basis of the results from multiple outcome measurements or analyses.

Some concerns

Overall risk of bias from tool algorithm.

Risk of bias for analysis 1.2 Treatment failure: need for systemic corticosteroids (of those starting inhaler)
StudyBias
Randomisation processDeviations from intended interventionsMissing outcome dataMeasurement of the outcomeSelection of the reported resultsOverall
Authors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgement

FitzGerald 2004

Low risk of bias

Patients were randomised to treatment groups at visit 2 according to a blocked computer generated randomisation list for each centre. As a randomisation list was generated for each centre, this sounds like it was centrally generated.
Baseline characteristics not shown for the full population, only those who had an exacerbation and started the study inhaler. Paper states that "there were no differences in patient characteristics between those who
experienced an exacerbation and those who did not".

Low risk of bias

Methods state 'double blind' so assume this means participants.
The participants themselves delivered the intervention.
Participants used a computerised diary to track doses. Authors state that compliance to the intervention was High risk for both groups.

High risk of bias

Statistical analysis used the "all patients treated" (APT) approach but most did not meet the threshold to start the study inhaler and so were not included in the analysis. In the full population, Of the 148 randomised to the control group, 115 completed the study (22% dropout), and 117/142 in the intervention group completed the study (17.6% dropout) which is High risker than the % who experienced an event in each group.
Patients who discontinued who did not use the study inhaler might have done so for reasons relating to their disease worsening, but the CONSORT diagram does not suggest that this was the case although the most common reason is 'other' in both groups.

Low risk of bias

Method of measuring the outcome was appropriate.
Measurement of the outcome was the same between intervention groups.
The study participant (who was blinded) entered details into an electronic diary with thorough pre-defined criteria on the outcome.

Some concerns

No study protocol or trial registration record is available.
Clear definition of outcome and timepoints.

High risk of bias

Used the tool algorithm.

Garrett 1998

Low risk of bias

Only information about the allocation is that the children were randomised by the hospital pharmacist and the investigators were blinded to this allocation.
Baseline characteristics were not provided by intervention group.

Some concerns

Used a blinded inhaler delivered by the participant.
Participants' were familiarised with the intervention and recording during the run-in period. Authors state that 95% of all the diary entries during an exacerbation period were completed and the study protocol was folLow risked correctly in all but five exacerbations. Authors included the data from the five exacerbations where the protocol was not folLow risked in the analysis.
Five deviations from the protocol involved the study inhaler being used for an incorrect number of days, which could have affected the outcome (additional active ingredient)
Authors state that the approach was appropriate as it reflects the reality of asthma treatment and adherence. In the review, cross-over data were included by obtaining two-by-two data and applying a formula to account for inter-correlation of matched pairs (Elbourne 2002).

High risk of bias

Only participants who had two matching pairs of exacerbations were analysed, which was 18 out of 28 enrolled. Those who had 1 or 3+ exacerbations during the study were not included.
No anlyses were reported to address bias from missing outcome data.
Reported reason for excluding from analyses was that they had good health status (no exacerabtions).
Those who had 1 or 3+ exacerbations during the study were not included.

Low risk of bias

Method for measuring the outcome was appropriate, and unlikely to differ between groups.
Reported that investigators were blinded to the allocation.

Some concerns

No trial registration, study protocol or SAP (even though article mentions a study protocol).
The numerical result is not likely to have been selected on the basis of the results from multiple outcome measurements or analyses.

High risk of bias

Used alogorithm.

Domain S = Low risk RISK OF BIAS
SQ1 = NI on whether the number of participants allocated to each of the two sequences equal or nearly equal.
SQ2 = Low risk - Cross-over data were included in the review by obtaining two-by-two data and applying a formula to account for inter-correlation of matched pairs (Elbourne 2002).
SQ3 = Low risk - Sufficient time was given for carryover effects to have disappeared before outcome assessment in the second period - folLow risked for two weeks and study tated that spirometry returned to normal within 1 week

Harrison 2004

Low risk of bias

An independent pharmacist randomly allocated individuals using computer generated random number tables.
Statistical tests for imbalances not presented but group differences at baseline do not suggest a problem with randomisation.

High risk of bias

Study report does not explicitly state "double‐blind", but a placebo was used which implies that it was not open label, and text mentions "A range of active and placebo study inhalers was available to enable the type of inhaler and daily dose to be matched to patients’ regular inhaled corticosteroid, type of inhaler, and dose". Participants delivered the intervention themselves.

There is no information on how the trialists ensured adherence to the intervention.

Clear criteria given to participants as part of an action plan to guide how they should measure their symptoms and at what point they should initiate the study inhaler. However, no details about how they monitored or increased adherence to the intervention so unsure whether there were failures in implementing it.

Report doesn't detail methods used to derive an unbiased estimate of the effect of adhering to intervention but the trial measured treatment failure only in those participants who started the study inhaler.


Low risk of bias

8.9 and 10.1% withdrew from the total intervention and placebo groups respectively which is relatively Low risk and balanced, although the study does not report whether and how missing data were imputed.


Low risk of bias

Method for measuring the outcome was appropriate. Participants were given a 10-day course of prednisolone (30 mg per day) to be taken if their asthma control deteriorated to the point that they would usually start oral corticosteroids, or if their peak fLow risk fell by 40% from the mean run-in value.
Unlikely that the measurement of Measurement of the outcome was the same between intervention groups. Oral steroids were patient initiated and they were blinded to the contents of the study inhalers.
Outcome assessors were the study participants.

Some concerns

No publicly available trial registration, study protocol or SAP (SAP is mentioned in the study report).
Clear definition of outcome and timepoints

High risk of bias

Overall risk of bias assessed as Low risk as the only 'some concerns' was due to no trial registration or study protocol details, but this is an old trial when making these available was not standard.

Martinez 2011

Low risk of bias

The Data Coordinating Center generated the random allocation sequence via computer software.The DCC had no contact with participants.
No obvious imbalances between groups.

Low risk of bias

States double blind and placebo interventions were used.
Those delivering the intervention were the participants.
Measures were taken to ensure adherence, inc. electronic measurement, participant diaries and pump canister checks.

Low risk of bias

The same raw data were used in the ITT and per protocol anlayses for this review because the study's design implies all participants would have used their study inhaler. Withdrawals from the study were relatively Low risk and even between the two groups included in this review: 11.3% in the intervention group (double ICS) and 12.5% in the control group (stable ICS). However, in some groups the number of events was similar to the number of participants with missing data, therefore their outcomes could impact the estimated effect of intervention.
Authors didn't correct or assess for bias from missing outcome data.
Reasons for missing data were accounted for and most were not related to the ouctome of interest (only two participants across the study were lost because they were unsatisfied with asthma control, both in the rescue beclomethasone group).

Low risk of bias

Method for measuring the outcome was appropriate
Measurement of the outcome would not have differed between groups.
Drugs were masked A to D to mask statisticians to treatment group.

Some concerns

Trial registered before participant enrolment began (https://clinicaltrials.gov/ct2/show/study/NCT00394329). However, registry record contains no prospective information about statistical analyses and no protocol or SAP available.
The numerical result is not likely to have been selected on the basis of the results from multiple outcome measurements or analyses.

Low risk of bias

Overall risk of bias assessed as Low risk as the only 'some concerns' was due to no study protocol or SAP available. However, it was registered and no other details from the trial for this outcome gives us reason to believe that there are some concerns regarding risk of bias.

Oborne 2009

Low risk of bias

An allocation sequence of random permuted blocks of 10 was generated using a random number table by an independent pharmacist and implemented by one of the study investigators once participants were enrolled into the trial. Mentions an investigator providing the randomization schedule and concealed allocation of masked inhalers.
Inhalers were identical and participants blind to the intervention.
Baseline data provided for all participants and those starting the study inhaler - no issues noted.

High risk of bias

Placebo inhaler was used.
Participants were not aware of the intervention, which they adminstered.

Clear criteria give to participants as part of an action plan to guide how they should measure their symptoms and at what point they should initiate the study inhaler. However, no details about how they monitored or increased adherence to the intervention so unsure whether there were failures in implementing it.

Report doesn't detail methods used to derive an unbiased estimate of the effect of adhering to intervention but the trial measured treatment failure only in those participants who started the study inhaler.

High risk of bias

The per protocol analysis based on only participants who started their study inhaler represents a relatively small, non-randomized proportion of the full cohort, and there was an imbalance in the number of people in each group who did so.
No analyses corrected for bias.
Patients who discontinued who did not use the study inhaler might have done so for reasons relating to their disease worsening. The CONSORT diagram includes changes to inhaler therapy and other medical reasons for reasons for discontinuing the intervention.

Low risk of bias

Method of measuring the outcome was appropriate.
Measurement of the outcome was the same between intervention groups.
Investigators blinded and they were likely the assessors.

Some concerns

The trial was rgeistered (http://www.isrctn.com/ISRCTN46018181). It was registered after participant enrolment began but before recruitment was completed. The registry doesn't include anything about the anlyses and a protocol/SAP isn't available.
Clear definition of outcome and timepoints

High risk of bias

Used algorithm.

Rice-McDonald 2005

Low risk of bias

Method of sequence generation not described but "order and allocation of treatment by concealed randomisation"
Does not report baseline characteristics for each intervention group.

Low risk of bias

A double-blind, double-dummy study with order and allocation of treatment by concealed randomisation. The participants administered the intervention.

Compliance to the intervention was monitored by symptom and medication diaries and a four week run in tested participants compliance to the protocol as part of the inclusion criteria.

Participants were contacted regularly by a nurse to judge when to initiate the study inhaler

High risk of bias

Of 35 randomized participants, 13 subsequently withdrew prior to any asthma exacerbations.
There is no evidence that the result was not biased by missing outcome data
The reasons for withdrawal from the trial are not associated with the intervention or outcome. However, 11 participants who were randomised and had exacerbations that went straight to oral corticosteroids (no study inhaler) were not analysed and this missingness woudl depend on its true value.

Low risk of bias

Method for measuring the outcome was appropriate, and unlikely to differ between groups. Treatment failure well defined.

Outcome assessors were the particpants, who were blinded.

Some concerns

No trial registry, protocol or SAP available.
The numerical result is not likely to have been selected on the basis of the results from multiple outcome measurements or analyses.

High risk of bias

Used algorithm.

Domain S (for crossover trials) = Low risk
SQ S1 = the number of participants allocated to each of the sequences was nearly equal.
SQ S2 = N/A
SQ S3 = a four week washout period was used which is sufficient time for any carryover effects to have disappeared before outcome assessment in the second period.

Wainwright 2009

Low risk of bias

"Stratified block randomisation by age (3-5, 6-10, 11-14), gender, centre". "Sequential study number allocated from a list according to blocking details". "...blocking details emailed to Dept of Epidemiology and Preventative Medicine, Monash Med School, Melbourne". "Study puffer number was allocated. Pre-numbered puffers were held at RCH Brisbane pharmacy"
Imbalances noted in disease characteristics (High risker proportion of patients in the intervention group than the control group had persistent asthma and atopic history), but likely to be down to chance given sample size.

High risk of bias

Placebo inhalers were used to presume this was to blind participants and personnel from the study medication, but no explicit definition of masking procedures

No details about how they monitored or increased adherence to the intervention so unsure whether there were failures in implementing it.

No details about methods to derive an unbiased estimate of the effect of adhering to intervention.

Low risk of bias

Have assumed that all participants who used oral steroid rescue had an exacerbation. Outcome data accounted for and a much High risker number of events than those who withdrew.


Low risk of bias

The method of measuring the outcome was appropriate.
Measurment of the outcome likely to be the same between the groups. Clear instructions for what to do in the event of worsening symptoms and called weekly by study nurse - all events likely to have been recorded.
The outcome assessor was the study participant (who was blinded).

Some concerns

Trial registered (https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=630) but after participant enrollment was completed. No details about the analyses available. No published journal report.
The numerical result is not likelyt to have been selected on the basis of the results from multiple outcome measurements or analyses.

High risk of bias

Used algorithm

Risk of bias for analysis 1.3 Serious and non-serious adverse events
StudyBias
Randomisation processDeviations from intended interventionsMissing outcome dataMeasurement of the outcomeSelection of the reported resultsOverall
Authors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgement
Subgroup 1.3.1 Serious

Martinez 2011

Low risk of bias

The Data Coordinating Center generated the random allocation sequence via computer software.The DCC had no contact with participants.
No obvious imbalances between groups.

Low risk of bias

States double blind and placebo interventions were used.
Those delivering the intervention were the participants.
Number adverse events measured in the whole population

Low risk of bias

Withdrawals from the study were relatively Low risk and even between the two groups included in this review: 11.3% in the intervention group (double ICS) and 12.5% in the control group (stable ICS). Raw numbers are available for the review's analyses. However, in some groups the number of events was similar to the number of participants with missing data, therefore their outcomes could impact the estimated effect of intervention.
No analyses on this outcome.
Authors kept good accounts of reasons for missing data and reported the SAEs.

Some concerns

Study report mentions a data and safety monitoring board which implies adverse event measuring procedures were folLow risked.
Measurement of the outcome would not have differed between groups.
The outcome assessor was the study participant (who was blinded) and AEs were overseen by a safety monitoring board, though unsure if they were blinded.
Assessment of the outcome may have been influenced by knowledge of intervention received
It was unlikely that assessment of the outcome was influenced by knowledge of intervention received given the small events numbers and oversight from the safety monitoring board.

Some concerns

Trial registered before participant enrolment began (https://clinicaltrials.gov/ct2/show/study/NCT00394329). However, registry record contains no prospective information about analyses and no SAP or protocol available.
The numerical result is not likely to have been selected on the basis of the results from multiple outcome measurements or analyses.

Some concerns

Used tool algorithm to generate overall risk of bias.

Wainwright 2009

Low risk of bias

"Stratified block randomisation by age (3-5, 6-10, 11-14), gender, centre". "Sequential study number allocated from a list according to blocking details". "...blocking details emailed to Dept of Epidemiology and Preventative Medicine, Monash Med School, Melbourne". "Study puffer number was allocated. Pre-numbered puffers were held at RCH Brisbane pharmacy"
Imbalances noted in disease characteristics (High risker proportion of patients in the intervention group than the control group had persistent asthma and atopic history), but likely to be down to chance given sample size.

High risk of bias

Placebo inhalers were used to presume this was to blind participants and personnel from the study medication, but no explicit definition of masking procedures
Outcome reported as as counts on a bar chart so unclear whether participants or events are the unit of analysis, and whether the events were counted in the full population or only those starting the inhaler. For the purposes of including data in the review, data were included as people with full population denominators.
Most participants started the study inhaler but approx 25% of each group did not, so there is some potential for bias.

High risk of bias

Number of events similar to the number that withdrew, and no reasons for withdrawal specified.
No evidence that the analysis corrected for bias.
Withdrawal from the study could be related to participants’ health status so it
is possible that missingness in the outcome was influenced by its true value. The circumstances of the trial make it likely that missingness in the outcome depends on its true
value, e.g. participants might drop out if their asthma symptioms get worse or they had an adverse reaction.

Low risk of bias

No information about the method of measuring the outcome.
Measurment of the outcome likely to be the same between the groups.
No information on whether the assessors were aware of the intervention.
The outcome assessment is not subjective.

Some concerns

Trial registered (https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=630) but after participant enrollment was completed. No details about the analyses available. No published journal report.
The numerical result is not likelyt to have been selected on the basis of the results from multiple outcome measurements or analyses.

High risk of bias

Used algorithm

Subgroup 1.3.2 Non-serious

Foresi 2000

Low risk of bias

Described as randomized but no details to assess sequence generation or allocation concealment.
Some imbalances across groups (statistical tests not presented) but may be expected for sample size and do not systematically favour one group over another.

High risk of bias

Not described fully but described as double-blind - placebo used to blind intervention in the event of an exacerbation.
Only those who had an exacerbation were included in the analysis - so the result relates to adverse events of the delivered intervention rather than allocation to it.
Adverse events were not reported for those who did not have an exacerbation, which was the majority (84%, 82% and 68% across groups), and event rates were Low risk so could be very different if measured in the full population.

High risk of bias

Per-protocol analysis was performed on 175 out of 213 randomized patients (191 who completed minus 38 with protocol violations), and adverse events were only reported for those who had an exacerbation (minority of population). Those included in the analysis were those who had an exacerbation, for which rates differed across groups and relate to asthma severity and effectiveness of the intervention. Withdrawal rates fromthe study also likely to be related to participants’ health status, as these include adverse events and other reasons.

Low risk of bias

No information about how adverse events were measured or managed. However, all participants visited the clinic once a month to discuss progress, symtpoms and adherence, and all kept daily symptom/management diaries.
Symptoms were recorded in daily dairies by the participant, who was blinded.

High risk of bias

No trial registration, study protocol or SAP.
No clear definitions for non-serious adverse events.
Outcome was not analysed.

High risk of bias

Used algorithm.

Oborne 2009

Low risk of bias

An allocation sequence of random permuted blocks of 10 was generated using a random number table by an independent pharmacist and implemented by one of the study investigators once participants were enrolled into the trial. Mentions an investigator providing the randomization schedule and concealed allocation of masked inhalers.
Inhalers were identical and participants blind to the intervention.
Baseline data provided for all participants and those starting the study inhaler - no issues noted.

Some concerns

"double-blind, placebo-controlled trial" "Drug groups were labelled as A, B, C, and D to mask statisticians to treatment group during the first complete run-through of data analyses." "Active and placebo inhalers were…identical apart from the presence or absence of inhaled corticosteroid, to achieve allocation concealment and blinding of investigators and pariticipants".
Only those who had an exacerbation were included in the analysis - so the result relates to adverse events of the delivered intervention rather than allocation to it.
Adverse events were only reported for the subset of participants who started the study inhaler, which was 23% and 28% in the active and placebo groups, respectively. There was a small number of events so the result might have been very different had adverse events been measured in the full population.

High risk of bias

Only 94 out of 403 participants (23%) started the study inhaler, so most of the population were not included in the analysis of adverse events
No evidence that the analysis was not biased by focusing on those who had an exacerbation.
Those included in the analysis were those who had an exacerbation, for which rates differed across groups and relate to asthma severity and effectiveness of the intervention.

High risk of bias

No information about how adverse events were measures or monitored. The participant's management plan only requested the daily diary of symptoms was completed if they felt their asmtha was deterioating. As there were more exacerbations in the active group we can assume more patients deteriotaed. Therefore, more participants in the active group would have been capturing other non-series adverse events in their diaries (and possibly why the active group had 9 events and the placebo 3).

High risk of bias

The trial was rgeistered (http://www.isrctn.com/ISRCTN46018181). It was registered after participant enrolment began but before recruitment was completed. The registry doesn't include anything about the anlyses and a protocol/SAP isn't available.
No clear definitions for non-serious adverse events.
Outcome was not analysed.

High risk of bias

Used tool algorithm.

Risk of bias for analysis 1.4 Unscheduled physician visits
StudyBias
Randomisation processDeviations from intended interventionsMissing outcome dataMeasurement of the outcomeSelection of the reported resultsOverall
Authors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgement

FitzGerald 2004

Low risk of bias

Patients were randomised to treatment groups at visit 2 according to a blocked computer generated randomisation list for each centre. As a randomisation list was generated for each centre, this sounds like it was centrally generated.
Baseline characteristics not shown for the full population, only those who had an exacerbation and started the study inhaler. Paper states that "there were no differences in patient characteristics between those who
experienced an exacerbation and those who did not".

High risk of bias

Methods state 'double blind' so assume this means participants. The participants themselves delivered the interventin.



Used an 'all patients treated' approach and only those who had exacerbations being included in the analysis (this means they were grouped according to the intervention they received and not the assigned intervention).
Those excluded from the analysis in the trial should not have had a substantial impact on the result as we (the review) have access to the raw numbers for our analysis.

Some concerns

Number of participants lost to folLow risk up were larger than the number of events.
Mo analysis methods that correct for bias due to missing outcome data mentioned.
About a third of participants lost to folLow risk up was due to disease not improving and adverse effects, so it is possible that missingness in the outcome was influenced by its true value.
The analysis does not account for participant characteristics that are likely to explain the relationship between missingness in the outcome and its true value but neither does it meet any of the five reasons that make it likely that missingness in the outcome depended on its true value.

Low risk of bias

Method of measuring the outcome was appropriate and the same between groups.

Participants (who were blinded) reported this outcome.

Some concerns

No study protocol or trial registration record is available.
The study participant (who was blinded) entered details into an electronic diary with pre-define criteria criteria for the outcome
It's unlikely that the result being assessed was selected from multiple eligible analyses

High risk of bias

Used algorithm.

Harrison 2004

Low risk of bias

An independent pharmacist randomly allocated individuals using computer generated random number tables.
Statistical tests for imbalances not presented but group differences at baseline do not suggest a problem with randomisation.

Low risk of bias

Study report does not explicitly state "double‐blind", but a placebo was used which implies that it was not open label, and text mentions "A range of active and placebo study inhalers was available to enable the type of inhaler and daily dose to be matched to patients’ regular inhaled corticosteroid, type of inhaler, and dose". Participants delivered the intervention themselves.



Stated an ITT analysis would be used for the primary objective, and have handled this outcome similarly.

Low risk of bias

Nearly all data available though number withdrawn similar to the number of events (31 in intervention, 28 in placebo). 8.9 and 10.1% withdrew from the intervention and placebo groups respectively which is relatively Low risk and balanced, although the study does not report whether and how missing data were imputed.


Low risk of bias

Method for measuring the outcome was appropriate.

Report doesn't mention who assessed this outcome but it's objective so would not have been influenced by the knowledge of the intervention received.

Some concerns

No trial registration, study protocol or SAP.
Th result being assessed was not selected from mutiple outcome measurements.
It's unlikely that the result being assessed was selected from multiple eligible analyses - though the trialists did use both an ITT and per protocol analysis on the outcomes.

Low risk of bias

Overall risk of bias assessed as Low risk as the only 'some concerns' was due to no study protocol or SAP available. However, it was registered and no other details from the trial for this outcome gives us reason to believe that there are some concerns regarding risk of bias.

Wainwright 2009

Low risk of bias

"Stratified block randomisation by age (3-5, 6-10, 11-14), gender, centre". "Sequential study number allocated from a list according to blocking details". "...blocking details emailed to Dept of Epidemiology and Preventative Medicine, Monash Med School, Melbourne". "Study puffer number was allocated. Pre-numbered puffers were held at RCH Brisbane pharmacy"
Imbalances noted in disease characteristics (High risker proportion of patients in the intervention group than the control group had persistent asthma and atopic history), but likely to be down to chance given sample size.

Some concerns

Placebo inhalers were used to presume this was to blind participants and personnel from the study medication, but no explicit definition of masking procedures



Outcome reported as percentages and unclear whether they related to the ITT or per protocol (those who started the inhaler) population. For the purposes of including data in the review, we assumed it was the latter and then used the full population numbers to include data for the primary outcome of the review. This assumes no one who didn't take the study inhaler had the outcome.
Most participants started the study inhaler but approx 25% of each group did not, so there is some potential for bias.

Low risk of bias

Outcome data accounted for and a much High risker number of events than those who withdrew. 11 in the intervention group (8.7%) an 10 in the control group (8.1%) withdrew from the study - Low risk and balanced.


Low risk of bias

The method of measuring the outcome was appropriate.
Measurment of the outcome likely to be the same between the groups. Clear instructions for what to do in the event of worsening symptoms and called weekly by study nurse - all events likely to have been recorded and unlikely to have differed between groups

No information on whether the assessors were aware of the intervention.
The outcome assessment is not subjective.

Some concerns

Trial registered (https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=630) but after participant enrollment was completed. No details about the analyses available. No published journal report.
The numerical result is not likelyt to have been selected on the basis of the results from multiple outcome measurements or analyses.

Some concerns

Used algorithm

Risk of bias for analysis 1.5 Unscheduled acute care, emergency department visit or hospital admission
StudyBias
Randomisation processDeviations from intended interventionsMissing outcome dataMeasurement of the outcomeSelection of the reported resultsOverall
Authors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgement

Garrett 1998

Low risk of bias

Only information about the allocation is that the children were randomised by the hospital pharmacist and the investigators were blinded to this allocation.
Baseline characteristics were not provided by intervention group.

High risk of bias

Used a blinded inhaler delivered by the participant.



10/28 randomised participants not included in the analyses as they didn't have an exacerbation in both arms - therefore a per protocol analysis.
No hospitalisations reported in either group but there is a large proportion of those randomised were exlcuded from the analyses.

High risk of bias

Only those with completed pairs of exacerbations were included in the final analysis - 10/28 excluded from the analyses as they only had an exacerbation in one arm. No mention of any children dropping out of the trial.
No anlyses were reported to address bias from missing outcome data.
Reported reason for excluding from analyses was that they had good health status (no exacerabtions).
Those who had 1 or 3+ exacerbations during the study were not included.

Low risk of bias

The method for measuring the outcome as appropriate.
Measurement of the outcome would not have differed between groups.
Investigators were blinded.

Some concerns

No trial registration, study protocol or SAP (even though article mentions a study protocol).
The numerical result is not likely to have been selected on the basis of the results from multiple outcome measurements or analyses.

High risk of bias

Used algorithm.

Domain S = Low risk RISK OF BIAS
SQ1 = NI on whether the number of participants allocated to each of the two sequences equal or nearly equal.
SQ2 = Low risk - Cross-over data were included in the review by obtaining two-by-two data and applying a formula to account for inter-correlation of matched pairs (Elbourne 2002).
SQ3 = Low risk - Sufficient time was given for carryover effects to have disappeared before outcome assessment in the second period - folLow risked for two weeks and study tated that spirometry returned to normal within 1 week

Jackson 2018

Some concerns

Method of sequence generation and allocation concealment not described (in paper, supplementary materials of protocol). However, the report mentions stratification which implies the sequence was computer generated.
Statistical tests for imbalances not presented but group differences at baseline do not suggest a problem with randomisation.

Low risk of bias

Participants and their carers/parents were blinded.
"The primary intention-to-treat analysis compared the overall efficacy of
each treatment strategy, regardless of whether any yelLow risk-zone episodes actually occurred."

Low risk of bias

Number of participants lost to folLow risk up is similar to the number of events; however, most participants were included in the study for most of the duration [mean folLow risk-up among all 127 participants in each group was 42.5 weeks and 40.3 weeks, respectively (i.e. most of the full 48 week folLow risk-up)].

Low risk of bias

Method of measuring the outcome was appropriate.
Measurement of the outcome was the shame between intervention groups.
The outcome person who decided on this outcome was the study participant and carer (who were blinded).

Low risk of bias

Protocol dated February 2014 and trial enrolment began in August 2014.
The numerical result is not likelyt to have been selected on the basis of the results from multiple outcome measurements or analyses.

Low risk of bias

Insufficient reporting warrants an assessment of 'some concerns' for the randomisation process, but the overall reporting and conduct of the trial suggests that rigorous procedures were folLow risked to minimize bias.

Martinez 2011

Low risk of bias

The Data Coordinating Center generated the random allocation sequence via computer software.The DCC had no contact with participants.
No obvious imbalances between groups.

Low risk of bias

States double blind and placebo interventions were used.
Those delivering the intervention were the participants.



Report states an IIT analysis was used.

Low risk of bias

Withdrawals from the study were relatively Low risk and even between the two groups included in this review: 11.3% in the intervention group (double ICS) and 12.5% in the control group (stable ICS). Raw numbers are available for the review's analyses. However, for this outcome there were 0 events reported so missing participant data could impact the estimated effect of intervention.
No analyses on this outcome.
Authors kept good accounts of reasons for missing data.

Low risk of bias

Method for measuring the outcome was appropriate and would not have differed between groups.

Report doesn't mention who assessed this outcome but it's objective so would not have been influenced by the knowledge of the intervention received.

Some concerns

Trial registered before participant enrolment began (https://clinicaltrials.gov/ct2/show/study/NCT00394329). Registry record contains no prospective information about analyses and no SAP or protocol available. However, there was no analyses for SAEs.
The result being assessed was not selected from mutiple outcome measurements.
It's unlikely that the result being assessed was selected from multiple eligible analyses

Low risk of bias

Overall risk of bias assessed as Low risk as the only 'some concerns' was due to no study protocol or SAP available. However, it was registered and no other details from the trial for this outcome gives us reason to believe that there are some concerns regarding risk of bias.

Wainwright 2009

Low risk of bias

"Stratified block randomisation by age (3-5, 6-10, 11-14), gender, centre". "Sequential study number allocated from a list according to blocking details". "...blocking details emailed to Dept of Epidemiology and Preventative Medicine, Monash Med School, Melbourne". "Study puffer number was allocated. Pre-numbered puffers were held at RCH Brisbane pharmacy"
Imbalances noted in disease characteristics (High risker proportion of patients in the intervention group than the control group had persistent asthma and atopic history), but likely to be down to chance given sample size.

Some concerns

Placebo inhalers were used to presume this was to blind participants and personnel from the study medication, but no explicit definition of masking procedures



Outcome reported as percentages and unclear whether they related to the ITT or per protocol (those who started the inhaler) population. For the purposes of including data in the review, we assumed it was the latter and then used the full population numbers to include data for the primary outcome of the review. This assumes no one who didn't take the study inhaler had the outcome.
Most participants started the study inhaler but approx 25% of each group did not, so there is some potential for bias.

High risk of bias

Outcome data accounted for but a much Low risker number of events than those who withdrew.
There is no evidence that the result was not biased by missing outcome data.
Patients who discontinued who did not use the study inhaler might have done so for reasons relating to their disease worsening.
Reasons for withdrawal not presented.

Low risk of bias

The method of measuring the outcome was appropriate. Clear instructions for what to do in the event of worsening symptoms and called weekly by study nurse - all events likely to have been recorded.
Measurment of the outcome likely to be the same between the groups.
No information on whether the assessors were aware of the intervention.
The outcome assessment is not subjective.

Some concerns

Trial registered (https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=630) but after participant enrollment was completed. No details about the analyses available. No published journal report.
The numerical result is not likelyt to have been selected on the basis of the results from multiple outcome measurements or analyses.

High risk of bias

Used algorithm

Risk of bias for analysis 1.6 Duration of exacerbation
StudyBias
Randomisation processDeviations from intended interventionsMissing outcome dataMeasurement of the outcomeSelection of the reported resultsOverall
Authors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgement

Harrison 2004

Low risk of bias

An independent pharmacist randomly allocated individuals using computer generated random number tables.
Statistical tests for imbalances not presented but group differences at baseline do not suggest a problem with randomisation.

Low risk of bias

Study report does not explicitly state "double‐blind", but a placebo was used which implies that it was not open label, and text mentions "A range of active and placebo study inhalers was available to enable the type of inhaler and daily dose to be matched to patients’ regular inhaled corticosteroid, type of inhaler, and dose". Participants delivered the intervention themselves.



Used a log rank test.

Low risk of bias

8.9 and 10.1% withdrew from the intervention and placebo groups respectively which is relatively Low risk and balanced. However, the study does not report whether and how missing data were imputed and the outcome can only be measured in those who had an exacerbation which was a subset of the randomised population.


Low risk of bias

Method for measuring the outcome was appropriate.
Unlikely that the measurement of the outcome would have differed between the groups.
Study participants were outcome assessors (so blinded).

Some concerns

No publicly available trial registration, study protocol or SAP (SAP is mentioned in the study report).
Clear definition of outcome and timepoints

Low risk of bias

Used tool algorithm for overall risk of bias.

Risk of bias for analysis 1.7 Duration of exacerbation (days to symptom recovery)
StudyBias
Randomisation processDeviations from intended interventionsMissing outcome dataMeasurement of the outcomeSelection of the reported resultsOverall
Authors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgement

Harrison 2004

Low risk of bias

An independent pharmacist randomly allocated individuals using computer generated random number tables.
Statistical tests for imbalances not presented but group differences at baseline do not suggest a problem with randomisation.

Low risk of bias

Study report does not explicitly state "double‐blind", but a placebo was used which implies that it was not open label, and text mentions "A range of active and placebo study inhalers was available to enable the type of inhaler and daily dose to be matched to patients’ regular inhaled corticosteroid, type of inhaler, and dose". Participants delivered the intervention themselves.



Used a log rank test.

Low risk of bias

8.9 and 10.1% withdrew from the intervention and placebo groups respectively which is relatively Low risk and balanced. However, the study does not report whether and how missing data were imputed and the outcome can only be measured in those who had an exacerbation which was a subset of the randomised population.


Low risk of bias

Method for measuring the outcome was appropriate.
Unlikely that the measurement of the outcome would have differed between the groups.
Study participants were outcome assessors (so blinded).

Some concerns

No publicly available trial registration, study protocol or SAP (SAP is mentioned in the study report).
Clear definition of outcome and timepoints

Low risk of bias

Used tool algorithm for overall risk of bias.

Risk of bias for analysis 1.8 Duration of exacerbation (days to lung function recovery)
StudyBias
Randomisation processDeviations from intended interventionsMissing outcome dataMeasurement of the outcomeSelection of the reported resultsOverall
Authors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgementAuthors' judgementSupport for judgement

Harrison 2004

Low risk of bias

An independent pharmacist randomly allocated individuals using computer generated random number tables.
Statistical tests for imbalances not presented but group differences at baseline do not suggest a problem with randomisation.

Low risk of bias

Study report does not explicitly state "double‐blind", but a placebo was used which implies that it was not open label, and text mentions "A range of active and placebo study inhalers was available to enable the type of inhaler and daily dose to be matched to patients’ regular inhaled corticosteroid, type of inhaler, and dose". Participants delivered the intervention themselves.
Used a log rank test.

Low risk of bias

8.9 and 10.1% withdrew from the intervention and placebo groups respectively which is relatively Low risk and balanced. However, the study does not report whether and how missing data were imputed and the outcome can only be measured in those who had an exacerbation which was a subset of the randomised population.

Low risk of bias

Method for measuring the outcome was appropriate.
Unlikely that the measurement of the outcome would have differed between the groups.
Study participants were outcome assessors (so blinded).

Some concerns

No publicly available trial registration, study protocol or SAP (SAP is mentioned in the study report).
Clear definition of outcome and timepoints

Low risk of bias

Used tool algorithm for overall risk of bias.