The Australian Cancer Atlas displays cancer statistics as ‘modelled’ or ‘smoothed’ estimates. Which simply means that it smooths out random variations in cancer rates so we can see a clearer pattern in the results. This visual explainer describes what data smoothing is, and how it is used in the Australian Cancer Atlas.

Smoothing of cancer data in the Atlas is applied to geographical areas. Within any given area, observed rates of cancer burden (such as diagnosis and survival) or influencing factors (such as screening and testing, risk factors, and hospital treatments) may fluctuate due to random variation. These random fluctuations can obscure our view of the real underlying cancer variations and therefore need to be minimised in order to understand the overall change in cancer rates. That is the reason that estimates of diagnosis and survival rates (including all other indicators) are smoothed in the Atlas.

To better understand why smoothing is applied to cancer data in the Atlas, let's take a look at a hypothetical example of cancer diagnosis, and apply the same basic concept of smoothing described in the bee example.

As shown in the figure below, the flight path of the bee can be smoothed to varying degrees, from very minimal smoothing (red line 1) to merely a straight line (red line 4).

The top-left example is considered too close to the raw data as it retains much of the random variation of the raw data and therefore does not make the data any easier to interpret. The examples in the bottom row have too much smoothing and we begin to lose information about the bee’s true flight path. To ensure the smoothed results are balanced, meaningful and useful, the Atlas team identified a statistical approach that provided the right balance between showing too much variability (with minimal smoothing) and showing too little variability (with excessive smoothing). In the approach used in the Atlas, information is borrowed from neighbouring geographical areas to increase the amount of data available to generate reliable statistics. The logic behind this is that populations in nearby areas tend to have similar characteristics, such as lifestyles, although this may not always be the case in practice. To account for this, the statistical approach pulls estimates for each area toward the overall average, which, in the case of the Australian Cancer Atlas, was the Australian average.

Among other factors, the amount of smoothing that occurs in an area depends on the size of its population. Statistics in areas with larger populations are less influenced by those in neighbouring areas. Conversely, in areas with smaller populations, the smoothing process has a greater effect, which includes pulling estimates closer to the overall (Australian) average.

All statistical estimates have some degree of uncertainty. The more uncertainty there is, the less confident we are in the estimate. Statistics from areas with smaller populations tend to have more uncertainty because there are less data to base the statistic on, making it less precise and less reliable. Smoothing helps reduce uncertainty because it borrows information from neighbouring areas and from the national average, particularly in those situations where the population is small and estimates are less reliable. Let’s explore further this advantage of smoothing in reducing uncertainty with the help of an image:

1. Reducing random variation from data: Results using raw, original data represent the observed values without any modifications, while smoothing helps reveal the underlying patterns by reducing the impact of random variation.

2. Data privacy: By removing the unnecessary details, data smoothing also hides the personal identifiers of the people whose cancer diagnosis and survival has been recorded and ensures data privacy. Privacy of all data relating to people diagnosed with cancer is of utmost importance and data smoothing ensures this privacy is maintained in the Australian Cancer Atlas. This is especially important for areas with small numbers of observations (for example, small numbers of diagnoses). Smoothing protects data privacy by borrowing additional information from neighbouring areas.

3. Handling missing data: Data smoothing can help to accommodate missing or incomplete data by estimating values based on surrounding data points. This can provide a more complete and more accurate picture of any underlying patterns.