Catedras.arcadiaconsulting.es

13 Walker (JB/D) 9/11/01 1:04 pm Page 435 HEALTH POLICY AND PLANNING; 16(4): 435–443
How to do (or not to do) . . .
Allowing for uncertainty in economic evaluations: qualitative
sensitivity analysis
D WALKER AND JA FOX-RUSHBYHealth Policy Unit, London School of Hygiene and Tropical Medicine, London, UK Sensitivity analysis allows analysts to explore the impact of uncertainty on their findings. It is an importantpart of any economic evaluation, and a lack of analysis is evidence of a poor quality study. Sensitivity analysishelps the analyst evaluate the reliability of conclusions for the context of the evaluation and can also facilitateconsideration of the generalizability of results to other settings. The variety of one and multi-way sensitivityanalyses offer simple and complementary approaches to evaluating the impact of uncertainty on the resultsand conclusions of economic evaluations. The paper begins with a brief discussion of the types of uncertaintythat can arise in economic evaluation, and follows with suggestions of how to plan a justified sensitivityanalysis. A number of specific techniques are worked through with examples, followed by a discussion ofwhen it is best to use them. The main weakness associated with sensitivity analysis is the control that theanalyst retains over three parts of the process: the choice of which variables to vary and which to treat asknown or fixed; the amount of variation around the base value of the parameter that is considered clinicallymeaningful or policy-relevant; and the determination of what constitutes a sensitive or robust finding. It istherefore essential that the approach of the analyst is clear and justified. It is likely that the future will seefurther developments in the approaches and training of statistical analysis. But in the meantime, an increasein the number of evaluators undertaking a wider range of sensitivity analysis would improve the quality ofevidence for, and outcomes of, decision-making.
1.0 Introduction
no recognition of the inherent uncertainty. Sensitivity analy-sis allows analysts to explore the impact of uncertainty on the Structured methodological reviews of published economic evaluations have consistently pointed to inadequacies in theway that uncertainty is treated (e.g. Udvarhelyi 1992; Briggs This paper begins with a brief description of the types of and Sculpher 1995; Agro et al. 1997; Walker and Fox-Rushby uncertainty that can arise in economic evaluation, and follows 2000). The most common finding is that research results are with suggestions about how to plan a justified sensitivity not subject to any form of sensitivity analysis and, when analysis. A number of specific techniques are worked through sensitivity analysis has been undertaken, the dominant with examples, followed by a discussion of when it is best to approach is an unjustified one-way analysis. Multi-way sensitivity analyses and statistical analyses are undertakenmuch less frequently.
2.0 Types of uncertainty
Briggs and Sculpher (1995) attributed the state of analysis to The two main taxonomies of uncertainty currently used were the fact that ‘. . . few guidelines offer details on how exactly presented in the mid 1990s by Manning et al. (1996) and sensitivity analysis should be carried out’, although, more Briggs et al. (1994). Manning et al. (1996) distinguished two recently, Mullins and Ogilvie (1998) found that different types of uncertainty: parameter and modelling. Parameter pharmacoeconomic guidelines recommend quite different uncertainty is ‘. . . uncertainty about the true numerical values approaches. Brown (1999) has also suggested that researchers of the parameters used as inputs’. They argue it arises for may feel unable to express doubts and uncertainties them- selves because decision-makers may view uncertainty as asign of weakness. Whatever the reason, the impact is thatthere can be a misplaced confidence in the results, which leads • the size of key inputs (either their quantity or value of the to bad decisions if point estimates of results are reported with quantity) in the economic evaluation are unknown or not 13 Walker (JB/D) 9/11/01 1:04 pm Page 436 observable, e.g. how future technology may change or how All these types of uncertainty can be addressed through sensi- prices of one input relative to another may change in the tivity or statistical analysis. This paper focuses on sensitivity analysis, which has been the main focus of activity in assess- • there is not a consensus about what value an input para- ing the impact of uncertainty. Statistical analysis of uncer- meter should take. e.g. what the appropriate rate for social tainty in economic evaluations incorporates sampling time preference is or which approach should be used to variability in parameter estimates and, whilst it has been the subject of much less research and fewer empirical appli- • there is uncertainty about the process behind variables, e.g.
cations in the economic evaluation of health interventions, factors explaining utilization of services or aspects of the interest is increasing, so we point readers to further literature • sampling variability of parameters exists, e.g. estimates of 3.0 Getting started: planning the sensitivity
• it is unclear how estimates relate to different populations, analysis1
e.g. extrapolating costs or effects to/from a random, ratherthan convenience, sample.
At this stage there are several steps that need to be performedprior to undertaking any type of sensitivity analysis. For each Modelling uncertainty is broken down into ‘model structure type of uncertainty (for costs and consequences) outlined in uncertainty’ and ‘modelling process uncertainty’ (Manning et al. 1996). Model structure uncertainty concerns doubtsabout the correct method for combining the parameters of (1) Identify all the parameters or approaches to modelling the costs, consequences and/or combinations of costs and that could be subjected to sensitivity analysis (in princi- consequences. This could include debates about: whether pal the model and all parameters are potential candi- particular types of costs or effects should be included, e.g.
productivity costs or decisions to include/exclude particular (2) Choose the input parameters or approaches to modelling types of adverse reactions; and the functional form associ- that you feel are most important to subject to a sensitivity ated with effectiveness (e.g. the impact on disease of cover- analysis from the list of possibilities, and justify the age of a population by a vaccine) or cost (e.g. the impact of choices made. For example, you may consider those vari- scale of production on the various inputs costed) or the ables (for the quantity or price/value of costs and effects) relationship between costs and effects. In all instances, ques- tions of whether the parameters assume multiplicative or additive forms can influence the results. Modelling process • have the greatest sampling variability; uncertainty is the uncertainty introduced by the combination • are most in the control of policymakers; of decisions made by the analyst. The analyst retains the • influence the largest percentage of total cost/effects; most influence over choosing what variable to include and • are more likely to differ from published data; how. For example, Busulwa et al. (2001) showed that the • are subject to greatest disagreements amongst method- training and working pattern of economic evaluation students affects both the variables selected for analysis as • are key to explaining how costs and/or effects vary Analysts should also justify why some parameters, if any, The taxonomy of uncertainty provided by Briggs et al.
or different types of models, have not been subjected to (1994) was based around four causes of uncertainty: vari- sensitivity analysis. Reasons might include that parame- ability in sample data on costs and effects of interventions ter estimates are known with certainty2 or that it only has within a population; methods used to measure and value a minimal impact on results3 (Drummond et al. 1997).
costs and effects; extrapolation of results from inter- (3) Choose the range of alternative values or models that you mediate outcomes to final outcomes or over time for the will substitute into the base-case analysis, justifying all same population; and the extent to which the results are choices made. The range of values adopted may be drawn generalizable to other populations. Where these two tax- from the literature, expert opinion accessed through onomies meet is a moot point. Ideas from both categoriz- consensus building techniques, sampling variation in the ations are contained within each and both provided original data, or the researcher’s own views. For parame- guidance on methodological approaches for investigation, ter uncertainty, the following might be considered: although Briggs et al. (1994) made a closer linkage • for deterministic data – high and low values of each key between types of uncertainty and methods for analyzing uncertainty. On the whole Manning et al. (1996) raise more • for stochastic data – the range, plus or minus one direct issues concerning the functional form and processes standard deviation of sampling error from clinical data, lying behind variables, although arguably this is implicit in or the most often used 95% confidence intervals for Briggs et al.’s issues concerning generalizability if one key parameters to determine a plausible range for assumes production processes change between settings.
Briggs et al. (1994) explicitly discuss the issue of methodo- For modelling uncertainty, the following might be logical uncertainty, which Briggs (2000) worked through later in his call for greater methodological comparability • using alternative functional forms for key variables; • including/excluding particular types of costs/effects; 13 Walker (JB/D) 9/11/01 1:04 pm Page 437 • asking another person/group to undertake the analysis base-case incremental cost-effectiveness5 of $39 per disabil- ity-adjusted life year (DALY) averted to $53 per DALY (4) Choose which techniques to use to analyze uncertainty averted. The high estimate produced an 18% divergence from (see section 4) and apply the sensitivity analysis to the the base-case estimate resulting in an incremental cost per evaluation. We suggest beginning with one-way analyses DALY averted of $32. These results are illustrated graphi- as a route to understanding the impact of individual vari- ables/models prior to moving to multivariate analyses.
(5) The final step in a sensitivity analysis is to interpret the A second type of one-way analysis is a ‘threshold analysis’.
findings. The analyst must determine how much change This concept is drawn from decision analysis, where the from the base-case result is acceptable or constitutes a analyst varies the size of an input parameter over a range and robust finding and/or the combination of parameter determines the level above or below which the conclusions values required to achieve pre-determined incremental change, and hence the ‘threshold’ point at which neither of cost-effectiveness ratios (see section 5.2).
the alternatives are favoured over the other – where thedecision is a ‘toss-up’ (Kassirer and Pauker 1981). This These steps show how much control the analyst retains.
concept can be applied to economic evaluation. For example, Whilst quantitative analysis is required, it remains an essen- analyses of antiretroviral therapies to reduce mother-to-child tially qualitative analysis because of the element of choice in transmission of HIV might identify the ‘threshold’, or ‘break- deciding what to vary, by how much, which technique to use even’, or ‘switching’, price of a new drug where it is and is not and in determining what constitutes robust findings or not. It cost-effective to introduce it. However, we will also show that also shows how essential it is for the evaluator to justify all the the decision rules used in cost-effectiveness analysis make choices involved before others can accept conclusions.
threshold analyses more difficult to interpret relative to cost-minimization or cost-benefit analysis (Briggs et al. 1994; 4.0 Techniques of sensitivity analyses
This section focuses on the techniques available for selection Figure 2 illustrates a threshold analysis in which the costs, in the fourth step outlined above. The predominant focus is effects and cost-effectiveness of short-course antiretroviral on approaches to estimating the impact of parameter uncer- therapy (using zidovudine) have been compared with the tainty in one-way and multi-way sensitivity analysis using cost, effects and cost-effectiveness of ultra short-course anti- worked examples. All examples focus on treating pregnant retroviral therapy (using nevirapine), for a given population.
women with antiretroviral therapy to reduce mother-to-child In the first instance, ultra short-course antiretroviral therapy transmission of HIV and are illustrative rather than any (USC1) dominates the programme of short-course antiretro- reflection of reality.4 Section 4.3 focuses briefly on viral therapy (SC1), i.e. it is more effective and less costly.
approaches to assess modelling uncertainty.
However, it is possible to vary the price of zidovudine, until 4.1 One-way (univariate) sensitivity analysis
The traditional approach to sensitivity analysis is to examineone variable at a time; one-way or univariate sensitivityanalysis. The process is simple; after calculating the base-casescenario, the incremental cost-effectiveness ratio is re-calcu-lated holding all parameters constant apart from the oneparameter chosen which is varied over the specified, and justi-fied, range. This process is repeated for as many parametersas desired, and ideally all of the model parameters.
Table 1 illustrates the results of a one-way sensitivity analysis,in which the HIV seroprevalence among pregnant women hasbeen varied, from a base-case estimate of 20%, between 15%and 25%, representing low and high estimates that mighthave been obtained from the literature. Using a low estimate Figure 1.
An illustrative example of one-way sensitivity analysis of seroprevalence resulted in a 36% divergence from the An illustrative example of one-way sensitivity analysis of incremental cost-effectiveness of short over long-course antiretroviral 13 Walker (JB/D) 9/11/01 1:04 pm Page 438 antiretroviral therapy is at point USC2, a point at which bothinterventions accrue the same amount of effects, it is possible to identify the cost of short-course antiretroviral therapyrequired to result in this intervention being more cost-effec-tive than ultra short-course antiretroviral therapy – essen- tially, this problem becomes one of cost-minimization. Forexample, say that the current cost of short-course antiretro- Costs (US$)
viral therapy is $20 per person. The cost for 100 individuals istherefore $2000 (SC1). If however, the cost of the therapy could be reduced to $15 per person, the cost for 100 indi-viduals would be $1500 and in this instance a decision-maker would be indifferent between the two alternatives, given that they both cost the same amount and produce the same quantity of effects. Hence, we have identified the threshold Figure 2.
An illustrative example of threshold analysis value of short-course therapy, $15, above which USC therapyis more cost-effective and below which short-course therapyis more cost-effective. A similar position exists concerning both alternatives have the same average cost-effectiveness cost-benefit analysis, where the specific focus is the point ratio, i.e. SC1 moves to point SC2 and an average cost-effec- where a technology offers a net benefit (Briggs et al. 1994).8 tiveness ratio of $10 per unit of effect ($1000/100). It may betempting, on the basis of this evidence, to suggest that neither 4.2 Multi-way (multivariate) sensitivity analysis9
of the options being compared would be favoured over theother as the average cost-effectiveness of each regimen is There are several ways to deal with multiple sources of uncer- equal. However, decisions should be made based on incre- tainty or variability: two-way, three-way, n-way and scenario mental cost-effectiveness ratios, which in this case is also $10 analyses (Briggs et al. 1994; Genugten et al. 1996; Petitti per unit of effect, the same as the average cost-effectiveness 2000). A two-way analysis varies two parameters, both of ratios of the two interventions. Unfortunately, this infor- which are common to the interventions assessed, at the same mation still does not provide sufficient information to guide time, and assesses the impact on the incremental cost-effec- decision-makers. To determine which intervention should be tiveness ratios of two mutually exclusive interventions, e.g.
implemented based on cost-effectiveness, either a fixed short-course over long-course antiretroviral therapy. The first budget or a price per unit of effectiveness must be introduced step is to construct a two-by-two matrix reflecting the incre- as the decision rule (Karlsson and Johannesson 1999). That is mental cost-effectiveness for every combination of the two to say, if a budget of $1000 were available, decision-makers variables of interest, in this case the price of AZT and HIV would choose either part of USC1 or SC2,7 whereas if a budget prevalence among pregnant women (holding all other par- of $1500 were available, a decision-maker would prefer to ameters constant at their base-line values) (see Table 2). The introduce USC1 as it results in greater effects, even though second step is to identify the pairs of values that equalize a the average cost-effectiveness ratios are equal. In fact a pre-determined willingness-to-pay for a unit of effect ($60 per decision-maker might prefer to introduce an intervention DALY averted in this example), i.e. the values of the two vari- with a higher average cost-effectiveness ratio with a larger ables of interest at which the decision is a ‘toss-up’ given the budget because of a desire to maximize effects given a fixed threshold value chosen.10 Next, all those combinations of budget, e.g. USC3 were a budget of $2 000 available.
price and prevalence that result in these threshold cost-effec-tiveness ratios are identified and presented graphically.11 However, if the same problem was a cost-minimization analy- Figure 3 shows that the region to the right of the line sis, it is possible to perform a threshold analysis that can be represents combinations of parameter values for which more readily interpreted. For example, if ultra short-course short-course antiretroviral therapy would be considered An illustrative example of two-way sensitivity analysis of incremental cost-effectiveness of short- over long-course antiretroviral 13 Walker (JB/D) 9/11/01 1:04 pm Page 439 value. To help understand how to interpret this graph, takethe case where the probability of breast-feeding is 100%. Allthe space below the line represents the case when a short-course regimen would be considered cost-effective relative tolong-course therapy given a threshold value of $60 per DALYaverted, and the area above the line is where long-coursetreatment would be considered cost-effective.
It is also possible to perform n-way sensitivity analyses, inwhich the expected cost-effectiveness is determined ‘. . . forevery possible combination of every reasonable value ofevery variable’ (Petitti 2000). This type of analysis is difficultto undertake and difficult to interpret; we will not be illus-trating how to do this type of analysis in this paper.
Figure 3.
An illustrative example of two-way sensitivity analysis of short- over long-course antiretroviral therapy The fourth type of multi-way sensitivity analysis is ‘scenarioanalysis’, of which there are many examples. There are also avariety of approaches that can be used to develop scenarios cost-effective; and the region to the left illustrates combi- that encompass the researchers thinking through possible nations for which long-course therapy would be considered scenarios themselves, through to scenarios developed with consensus group techniques. We note three types of scenariosthat might be used: In three-way sensitivity analysis, as the name suggests, theincremental cost-effectiveness is determined for combi- • Analysis of the set of extreme circumstances across par- nations of estimates of three parameters (holding all other ameters, also known as a ‘max-min’ analysis or ‘worst/best’ parameters constant at their baseline levels). This time a case analysis (Briggs et al. 1994). In this case the parameter choice is made to hold one of the three variables at a particu- values that yield the worst (highest) and the best (lowest) lar level and to identify the combination of the other two vari- cost-effectiveness ratios are combined. For the purposes of ables that equal a pre-determined willingness-to-pay per unit illustration, we base this on two parameters (although in of effect. This analysis is repeated according to the number of practice any number of parameters can be used). Using the levels the analyst wants to hold the first choice variable at same example, an HIV prevalence of 10% and a price of and/or number of different willingness-to-pay values the $1.13 per dose of AZT might produce the worst scenario analyst wants to explore. Again, the interpretation of three- ($200 per DALY averted), and a combination of 40% and way sensitivity analyses is difficult without a graph. Figure 4 price of $0.17 the best scenario ($5 per DALY averted) (see shows the case where the decision-maker’s willingness-to-pay per unit of effect is $60 per DALY averted and three par- • Use of an agreed ‘reference case’ of methods by analysts.
ameters have been varied: price of AZT; HIV prevalence The most well-known reference case is described by Gold among pregnant women; and the percentage of women who et al. (1996)12 who set out the methodological guidance subsequently breast-feed their children. Five lines are shown from the report of the Panel on Cost-Effectiveness and for five values representing the probability that women Medicine in the United States. It is particularly aimed at breast-feed their children. For each of these values, the line increasing the quality and comparability of results across shows the combination of the price of AZT and HIV preva- interventions and reducing what Briggs et al. (1994) call lence among pregnant women that would result in an incre- mental cost-effectiveness ratio equal to the pre-determined • Use of the ‘null’ set (Genugten et al. 1996). A case calling Figure 4. An illustrative example of three-way sensitivity analysis of short- over long-course antiretroviral therapy. Note: an asterisk
signifies the percentage of women who subsequently breast-feed their children
13 Walker (JB/D) 9/11/01 1:04 pm Page 440 for evaluating all cost-effectiveness ratios alongside a Looking at one source of uncertainty at a time in the model scenario assuming no interventions at all was recently set provides an incomplete and under-estimate of how uncertain out by Murray et al. (2000). Using Murray et al.’s approach the estimated overall cost-effectiveness ratio actually is to defining a scenario for the null set would involve the (Agro et al. 1997). There are three related problems: development of natural history models to estimate theimpact of disease without any formal sector health care • the incremental cost and effectiveness depend on multiple interventions and redefining all interventions considered with respect to this null set. In particular it is argued that • the interaction of particular factors may imply that the total using a null set scenario will increase the generalizability of effect could be something quite different from the simple results across regions of the world.
• the cost-effectiveness ratio is a ratio of two uncertain numbers, with the result that the uncertainty in the ratio 4.3 Functional form sensitivity analyses
may be substantially larger than that of either of its One-way and multi-way sensitivity analyses focus on the choice of values for parameters. They do not question the waythat parameters are assumed to be related to each other in the The various forms of multi-way analyses allow these aspects underlying model. Computing incremental cost-effectiveness to be taken into account to some degree. Of these, possibly ratios using different types of models and comparing the the ‘max-min’ is least useful, unless the results are insensitive impact on the final ratios is the only approach recommended to the extreme combination of parameter values considered to date (Manning et al. 1996). The two main approaches to (Agro et al. 1997). If the results are sensitive to the extremes, this are either for the analyst to run alternative models or for the results are not very useful bounds on the uncertainty in different analysts or groups of analysts to run their own the cost-effectiveness ratio for two reasons: it is highly models on the same data. Examples of some of the structural unlikely that all of the extreme values of key parameters will occur in any particular setting; and, under some circum-stances, two or more sources of uncertainty may partially • comparing simple and more complex models (e.g. judging offset each other, due to the inherent structure of the the impact of increasing the ability to distinguish different problem. Two- and three-way sensitivity analyses can be helpful to identify the best scenario likely to appeal to • comparing the effect of using multiplicative or additive decision-makers with a note of the reliability of such a situ- models of diseases, interventions evaluated and co- ation, but they also suffer from some of the same problems of morbidities when calculating age-sex specific hazard func- one-way sensitivity analyses; namely, that they may be diffi- cult to interpret if the variables used are dependent on each • changing the relationship between costs and number of other (Agro et al. 1997). In addition, these types of analyses become cumbersome if more than two inputs are variedsimultaneously.
5.0 Discussion
The ‘reference case’ as a type of scenario analysis may stimu-late an improvement in the comparability and methodologi- Having set out why sensitivity analysis is needed, and how it cal quality of economic evaluations. However, the remaining might be planned and executed, it is important to reflect on uncertainty associated with the effects of applying different when the alternative approaches might be used. Secondly, we parameter estimates can only be handled using different consider how the results of sensitivity analyses might be inter- tools, and the reference case requires that additional sensi- preted. Finally, we indicate the value of statistical approaches tivity analysis be undertaken. It is important to note also, that that might be used to evaluate uncertainty, and provide a the reference case (as with the null set) has not yet been vali- dated for low- and middle-income countries.
The variety of univariate and multivariate sensitivity analyses 5.1 What are the advantages and disadvantages of the
provides a range of complementary techniques for dealing different types of sensitivity analysis?
with uncertainty. For this reason, we urge practitioners of Relative to the other techniques described, one-way sensi- economic evaluation of health care programmes to tivity analyses are easy to use and provide flexibility in para- strengthen their research by performing a range of sensitivity meter choice. They are a logical, easy to grasp place to start analyses in order to best capture the extent to which uncer- to understand the structure of a particular cost-effectiveness tainty is present in their findings, and hence the robustness of analysis and provide the natural building blocks to do multi- their results and recommendations. The rather cursory way sensitivity analyses. They can shed light on whether any section on functional form sensitivity analysis was a reflection piece(s) of research could improve the outcome from a policy of our desire for completeness in covering approaches to decision and whether it is worth waiting for this additional sensitivity analysis, and the paucity of methodological and data. However, although insightful, one-way sensitivity empirical work in this area. We hope it encourages more analyses (including threshold analyses) by themselves are people to consider how to undertake assessment of un- 13 Walker (JB/D) 9/11/01 1:04 pm Page 441 5.2 How should the results of sensitivity analysis be
are that all parameters can be varied simultaneously and, as interpreted?
it allows point estimates of cost-effectiveness ratios to begiven confidence intervals, the likelihood of particular cost- Following any sensitivity analysis, the first step is to note effectiveness ratios occurring can be judged. A growing which variables cause the greatest and least change in the number of publications have addressed the application of sta- incremental cost-effectiveness ratio. The two main difficulties tistical methods to pharmacoeconomics. Most of the litera- with this are deciding: what constitutes a large/small change, ture is related to assessing the variability of cost-effectiveness and how likely the change is to be. With a sensitivity analysis ratios, calculation of confidence intervals and formal hypoth- both these decisions are the analyst’s own judgement and the esis testing with cost-effectiveness ratios (Mullins and Ogilvie basis of such decisions need to be open for readers (and 1998). This body of literature is likely to continue to evolve policymakers) to assess and consider changing according to and be debated. Manning et al. (1996), Briggs (2000), Petitti different views about the future. For example, threshold (2000) and Hutubessy et al. (2001) all provide further details analyses of the price of antiretrovirals can help in identifying on methods for the interested reader.
prices at which different therapies might be cost-effective insub-Saharan Africa given knowledge of the size of the budget 6.0 Conclusions
available or a decision-maker’s willingness to pay per unit ofeffect. The analyst makes a judgement of how likely this is to Sensitivity analysis is an important part of any economic be and therefore how robust conclusions about the base-case evaluation, and a lack of analysis is evidence of a poor quality study. Sensitivity analysis helps the analyst evaluate the reli-ability of conclusions for the context of the evaluation and can The implications of the results of the sensitivity analysis can also facilitate consideration of the generalizability of results be considered in terms of recommendations for policy and/or to other settings. The variety of one-way and multi-way sensi- tivity analyses offers simple and complementary approachesto evaluating the impact of uncertainty on the results and con- • results of a sensitivity analysis may show that collecting one clusions of economic evaluations. However, the main weak- type of data may make conclusions far more robust, and ness associated with sensitivity analysis is the control that the thus a decision may be better delayed until data are analyst retains over three parts of the process: the choice of which variables to vary and which to treat as known or fixed; • decision-makers may take results from one type of sensi- the amount of variation around the base value of the para- tivity or scenario analysis dealing, for example, with a vari- meter that is considered clinically meaningful or policy- able more in their control to set policy; relevant; and the determination of what constitutes a • decision-makers in different time periods or countries may sensitive or robust finding (Mullins and Ogilvie 1998). It is also be able to draw alternative conclusions provided ana- therefore essential that the approach of the analyst is clear lysts have undertaken sensitivity analyses. For example, if and justified. It is also likely that the future will see further the decision-maker’s willingness-to-pay for a unit of effect developments in the approaches and training of statistical was $60, the breast-feeding rate was only 30%, price of analysis, but in the meantime an increase in the number of AZT was $1.0/dose and HIV prevalence was 0.3, then the evaluators undertaking a wider range of justified sensitivity illustrative example provided in Figure 4 would suggest, analysis would improve the quality of evidence for, and out- provided all other things were equal, that long-course treat- • estimates of the maximum willingness-to-pay by decision- Endnotes
makers for a unit of effect can be used to identify decisions.
1 Prior to undertaking a sensitivity analysis, it is important to For example, $50 per DALY averted has been adopted have completed the base-case analysis of the evaluation and checked arbitrarily, by the World Bank (Jamison et al. 1993; World for potential errors. Detection of errors in the base-case analysis Bank 1993),13 as the threshold below which public-health during the sensitivity analysis will mean that not only will the base- interventions are deemed to be cost-effective in low- case analysis have to be re-calculated, but also that the sensitivity 2 Although if authors are considering the generalizability of results and models across space and time, they may still wish to Finally, as Manning et al. (1996) state, it is important that examine the impact of such uncertainty and choose a range of policy-makers understand that any ‘. . . particular analysis presented is but one sampled from a universe of possible 3 This could, of course, be difficult to claim without having 4 See Marseille et al. (1999), Söderlund et al. (1999) and Stringer et al. (2000) for examples of applications of sensitivity analysis 5.3 What other types of techniques exist to evaluate
to economic evaluations of strategies to reduce mother-to-childtransmission of HIV among pregnant women in sub-Saharan Africa.
uncertainty?
5 This is calculated as the difference in cost of two competing In recent years, there has been an increased interest in interventions divided by the difference in effectiveness of the sametwo competing interventions, e.g. long- and short-course antiretro- developing and undertaking statistical analyses of uncer- tainty in the estimated incremental cost-effectiveness ratio.
6 When an intervention is both more effective and less costly It is particularly aimed at evaluating uncertainty due to than the alternative, a state of dominance occurs, i.e. there is never a sampling variation of the input parameters. The advantages switching point at which an intervention is and is not cost-effective.
13 Walker (JB/D) 9/11/01 1:04 pm Page 442 Issues of dominance are relevant only when interventions are Manning WG, Fryback DG, Weinstein MC. 1996. Reflecting uncer- mutually exclusive and can only be discussed after sensitivity tainty in cost-effectiveness analysis. In: Gold MR, Siegel JE, analyses have been performed because initial decisions made on the Russell LB, Weinstein MC (eds). Cost-effectiveness in health basis of point estimates of cost-effectiveness may suggest that an and medicine. New York: Oxford University Press.
intervention is dominant, when in fact this relationship may not hold Marseille E, Kahn JG, Mmiro F et al. 1999. Cost effectiveness of true for other values of parameters.
single-dose nevirapine regimen for mothers and babies to 7 This assumes constant returns to scale and perfect divisibility decrease vertical HIV-1 transmission in sub-Saharan Africa.
The Lancet 354: 803–9.
8 Nevertheless, it will still be necessary to apply decision rules Miller M, McCann L. 2000. Policy analysis of the use of hepatitis B, to identify those interventions that should be implemented on the Haemophilus influenzae type b, Streptococcus pneumoniae con- jugate and rotavirus vaccines in national immunization sched- 9 If an intervention dominates the alternative, it is not possible ules. Health Economics 9: 19–35.
to perform a multi-way sensitivity analysis.
Mullins CD, Ogilvie S. 1998. Emerging standardization in pharma- 10 It is also possible to identify the pairs of values that result in coeconomics. Clinical Therapeutics 20: 1194–202.
an incremental cost-effectiveness ratio of zero (i.e. the average cost- Murray CJL, Evans DB, Acharya A, Baltussen RMPM. 2000.
effectiveness ratios of the two alternatives are equal), but as illus- Development of WHO guidelines on generalised cost-effec- trated above, this provides no aid to decision-makers.
tiveness analysis. Health Economics 9: 235–51.
11 It is difficult to interpret the results of a two-way sensitivity Petitti D. 2000. Meta-analysis, decision analysis and cost-effectiveness analysis: methods for quantitative synthesis in medicine. New 12 See their Appendix A and applications of the reference case Söderlund N, Zwi K, Kinghorn A, Gray G. 1999. Prevention of ver- 13 Alternative benchmarks have been suggested as well. For tical transmission of HIV: analysis of cost effectiveness of example, an intervention that results in a life-year saved for less than options available in South Africa. British Medical Journal 318:
the per capita GNP is sometimes considered to be cost-effective (Miller and McCann 2000). Also note that benchmarks in developed Stringer J, Rouse DJ, Vermund SH, Goldenburg RL, Sinkala M, nations will usually be several orders of magnitude greater than Stinett AA. 2000. Cost-effective use of nevirapine to prevent vertical HIV transmission in sub-Saharan Africa. Journal of
Acquired Immune Deficiency Syndromes 24: 369–77.
Udvarhelyi IS, Colditz GA, Rai A, Epstein AM. 1992. Cost-effec- References
tiveness and cost-benefit analyses in the medical literature. Arethe methods being used correctly? Annals of Internal Medicine Agro KE, Bradley CA, Mittmann N, Iskedjian M, Llerisch AL, 116: 238–44.
Einarson TR. 1997. Sensitivity analysis in health economics and Walker D, Fox-Rushby J. 2000. Economic evaluation of communi- pharmacoeconomic studies. Pharmacoeconomics 11: 75–88.
cable disease interventions in developing countries: a critical Briggs A, Sculpher M, Buxton M. 1994. Uncertainty in the economic review of the published literature. Health Economics 9:
evaluation of health care technologies: the role of sensitivity analysis. Health Economics 3: 95–104.
WHO. 1996. Investing in health research and development: report of Briggs A, Sculpher M. 1995. Sensitivity analysis in economic evalu- the ad hoc committee on health research relating to future inter- ation: a review of published studies. Health Economics 4:
vention options. TDR/Gen/96.1. Geneva: World Health Briggs AH. 2000. Handling uncertainty in cost-effectiveness models.
World Bank. 1993. World Development Report: Investing in Health.
Pharmacoeconomics 17: 479–500.
Brown R. 1999. How misused decision aids mislead policy makers.
Decision Societies Website. http://faculty.fuqua.duke.edu/daweb(accessed 8th Feb 2001).
Busulwa WR, Buyse D, Mulligan JA, Walker D, Fox-Rushby JA.
Acknowledgements
2001. Modelling process uncertainty in cost-effectiveness: can itbe measured? Paper presented at the 3rd International Health We would like to acknowledge David Wonderling of the London Economics Association Conference, July 22–25, 2001.
School of Hygiene and Tropical Medicine and two anonymous Fox-Rushby JA (forthcoming) Disability-Adjusted Life Years reviewers for their helpful comments on an earlier draft of this paper.
(DALYs) for decision-making? Office of Health Economics.
Damian Walker and Julia Fox-Rushby are members of the Health Genugten MLL, van Rutten FFH, Jager JC. 1996. Scenario develop- Economics and Financing Programme, which is supported by funds ment and costing in health care: methodological accomplish- from the UK Department for International Development (DFID).
ments and practical guidelines. Utrecht: Foundation for FutureHealth Sciences STG, International Books.
Hutubessy RCW, Baltussen RMPM, Evans DB, Barendregt JJ, Biographies
Murray CJL. 2001. Stochastic league tables: communicating
results to decision-makers. Health Economics 10: 473–7.
Damian Walker, MSc, is a Research Fellow in Health Economics in Jamison D, Mosley W, Measham A, Bobadilla JL (eds). 1993.
the Health Economics and Financing Programme of the London Disease control priorities in developing countries. New York: School of Hygiene and Tropical Medicine. His main area of research is the economic evaluation of health care programmes in developing Karlsson G, Johannesson M. 1999. The decision rules of cost-effec- countries, with particular interest in HIV/AIDS and TB prevention tiveness analysis. In: Mallarkey G (eds). Economic evaluation in strategies, safe motherhood initiatives and the introduction of new healthcare. Hong Kong: Adis International Limited.
vaccines to routine immunization programmes.
Kassirer JP, Pauker SG. 1981. The toss up. New England Journal of Medicine 305: 1467–9.
Mandelblatt JS, Fryback DG, Weinstein MC, Russell LB, Gold MR, Julia Fox-Rushby, PhD (Economics), is a Senior Lecturer at the Hadorn DC. 1996. Assessing the effectiveness of health inter- London School of Hygiene and Tropical Medicine. She has written ventions. In: Gold MR, Siegel JE, Russell LB, Weinstein MC numerous academic papers on the cost-effectiveness of health inter- (eds). Cost-effectiveness in health and medicine. New York: ventions across the world, specializing particularly in maternal and child health, malaria and, more recently, vaccine-preventable 13 Walker (JB/D) 9/11/01 1:04 pm Page 443 disease. She has also been involved over the past 15 years in develop- Correspondence: Damian Walker, Health Economics and Financing ing a number of non-disease-specific measures of health-related Programme, Health Policy Unit, Department of Public Health and quality of life as a member of the EuroQol group, advisor to the Policy, London School of Hygiene and Tropical Medicine, Keppel WHOQOL group and as principal investigator of the KENQOL Street, London WC1E 7HT, UK. Tel. +44 (0) 20 7927 2104. Fax. +44 (0) 20 7637 5391. Email: damian.walker@lshtm.ac.uk

Source: http://catedras.arcadiaconsulting.es/sites/default/files/docs/how_to_do_or_not_to_do_walker_2001.pdf