# Confidence in estimates

Since the MCMC approach for Bayesian models use repeated random samples from a probability distribution, they provide ideal data to assess the uncertainty associated with the estimates generated by the model. The mapped estimate is the median value of these repeated random samples, chosen over the mean value to reduce any impact of extreme values.

We supply two different aspects of the confidence in the estimates; first the confidence that the area’s estimate represents a true difference from the national average, and second, the credible interval, which is a measure of precision of the median estimate.

- The probability that the area’s estimate differs from the Australian average. This is displayed in the V-plots, and also on the maps (default view).

This is calculated based on the posterior probability (PP) that the estimate is either above or below 1 (as 1 is the Australian average). The calculation involves comparing each monitored MCMC iteration against 1. So, for our posterior probability of being above 1, any MCMC iteration that has an SIR or EHR above 1 is given a value of 1. These are then summed together for each area and divided by the total number of monitored iterations, giving the probability that the estimate for this area is above 1 (the Australian average). For area \(i\) given the number of monitored MCMC iterations \(M\), this can be expressed through an equation as:

$$PP_{i,\text{high}}=\frac{1}{M}\sum_{m=1}^M=\mathbb{I}\left(SIR_i^{(m)}>1\right)$$

If \(PP_{i,\text{high}} \) denotes the posterior probability of area \(i\)‘s estimate being above 1, then \(PP_{i,\text{low}}=1-PP_{i,\text{high}}\) is the posterior probability of being less than 1. From this, a new measure is constructed, which represents the confidence that the area SIRS/EHRs are different from 1 (either above or below).

This measure of confidence is computed as \(\left| PP_{i,\text{high}} – PP_{i,\text{low}} \right|\).

According to Richardson et al. (2004), a \(PP_{i,\text{high}} \geq \) 80% suggests the area is genuinely above the Australian average. If \(PP_{i,\text{high}} = \) 80% then \(PP_{i,\text{low}} = \) 20%, so the equivalent in our measure of confidence is 60%. This 60% domain is delineated for clarity, and areas above this zone are likely to genuinely differ from the Australian average. Estimates on the right are genuinely above the Australian average, while those on the left are genuinely below the Australian average.

The default map also displays this confidence by shading any areas with a PP < 60% towards the Australian average colour. This confidence view can be toggled on or off.

- The 60% and 80% credible intervals

Here the focus is on how precise the estimate is. The interpretation is that there is a 60% (or an 80%) chance the true estimate lies between the lowest point of the credible interval and the highest point of the credible interval. The density of this interval is shown in the wave plot, and the mouse-over gives the exact values.

### References

Richardson, S., A. Thomson, N. Best and P. Elliott (2004). “Interpreting posterior relative risk estimates in disease-mapping studies.” Environmental Health Perspectives **112**(9): 1016-1025.