Why you should dismiss the CARRS-Q bicycle helmet research
Monash University Research Used by CARRS-Q
Image Credit: Dr P Martin
This is the fourth post in a series (Read Part One, Part Two, Part Three) looking at the non-peer reviewed CARRS-Q publication entitled “Bicycle Helmet Research“, published in 2011 and widely relied upon to support mandatory helmet laws for cyclists. This publication was commissioned by the State Government to support its policy of mandatory helmet laws in Queensland, Australia, in the face of criticisms of its lack of support for Brisbane’s public bike hire scheme.
The second post discussed the failings of the Cochrane Review, a review of pre-existing research used to support bicycle helmet efficacy, which was in turn heavily relied upon by the CARRS-Q publication to support the Queensland Government’s policy of mandatory helmet laws.
The third post looked at other bicycle injury related research utilised by the CARRS-Q publication, highlighting some of the failings of that research and how that original research has been misinterpreted by CARRS-Q to support mandatory helmet laws. This post looks at more of that research, focusing on a series of Victorian-based studies conducted by Monash University.
According to CARRS-Q, these Victorian studies:
“… found that the number of bicyclists admitted to hospital with head injuries decreased following the introduction of compulsory helmet wearing.” (CARRS-Q, 2011, page 17)
The CARRS-Q publication goes on to state:
“These decreases were associated with an increase in helmet wearing rates, coupled with reduced levels of cycling.” (CARRS-Q, 2011, page 18)
The Victorian studies were undertaken by the Monash University Accident Research Centre. These studies actually found that head injuries were significantly decreasing well before the introduction of helmet laws. (Cameron et al: Evaluation of bicycle helmet wearing law in Victoria during its first 12 months, Report 32, Monash University Accident Research Centre, 1992).
Below is a graph from the Victorian study (the arrow indicates when helmet laws were introduced):
(Cameron, M., Heiman, L., Neiger, D., Evaluation of bicycle helmet wearing law in Victoria during its first 12 months, (1992), Report 32, Monash University Accident Research Centre, figure 10.)
The first study in this series found that the rate of head injuries recorded for Melbourne cyclists did decrease, but found that the amount of decrease of head injury due to helmet use, when compared to rates of helmet use prior to making laws mandatory, identified some anomalies which:
“ … suggests that the number of cyclists [in Melbourne] involved in crashes with motor vehicles has decreased during the post-law period, either due to a reduction in bicycle use or a reduction in the risk of crash involvement.” (Cameron et al, 1992, page 8)
This series of studies found that there was a reduction of 37% (based on Transport Accident Commission data) or 51% (based on hospital admissions data) in the number of cyclists killed or suffering head injuries during the 12 months following the introduction of mandatory helmet laws in Victoria. However the studies also found that there was also a substantial reduction of 21% (Transport Accident Commission data) or 24% (hospital admissions data) in the number of severely injured cyclists who did not sustain head injury. In short, the study found there appeared to be a general injury reduction in all crashes involving cyclists, not just head injuries.
This finding is also supported by other research that shows that injuries for all road users including pedestrians have significantly decreased, and therefore such decreases cannot be attributed to bicycle helmet laws.
The Victorian studies also found that the reduction in death or injury for cyclists did not match the expected reduction, when compared to pre-legislation helmet wearing rates. In other words, if previous reductions could be attributed to voluntary helmet wearing, these reductions were too great to be attributed to compulsory helmet wearing rates.
These findings were supported by the second study in the series, which confirmed that:
[t]he proportion of all injured cases with a head injury in 1992 was significantly less than that projected on the basis of continuing pre-law trends. The mechanisms by which this reduction was achieved seem to be twofold: a reduction in the number of bicyclists involved in crashes resulting in severe injury and a reduction in the risk of head injury for bicyclists who were severely injured.” (Cameron et al 1992, page 10)
So something else, other than helmets, may be responsible for the decrease in head injuries over and above general decreases in all types of injuries.
The answer may actually lie in this same series of studies. These studies also found that while the rate of helmet wearing increased as a result of the laws, the total cycling rates for children significantly decreased as a result of the introduction of helmet laws. The first Victorian study actually found that cycling decreased by 15% for 5-11 year olds, and by 44% for teenagers. The study also concluded that there may also have been an increase in adult cyclists, however such a finding is not as well documented:
“A comparison of the 1991 survey results for adults with a similar survey in 1987/88 (while a less direct comparison because of the nearly four year gap and different times of the year) showed that adult bicycle use had increased by 58%. The increase in adult cycling appears to have off-set the decrease in child cycling. Preliminary results from the 1992 survey conducted by MUARC [Monash University Accident Research Centre] indicate that these trends have been maintained.” (Cameron et al 1992, page i)
In fact, despite this conclusion, this same study identifies a similar survey carried out in 1990, carried out in the same locations and at the same time of year. A comparison of the 1991 survey with this similar survey carried out just one year prior paints a very different picture: it shows a drop in total cyclist numbers (including adults) of 45%. (Cameron et al 1992, pages 5 and 6).
Firstly based on the other research discussed above and in a previous post, it is clear that children and teenagers are far more likely to be involved in an accident than adult cyclists. In short, the decrease in head injuries could be explained by different riding patterns between children (who significantly decreased in riding rates during the relevant periods of the studies) and adults who are inherently less likely to crash or sustain serious injury in the event of a crash (who may have increased during the relevant periods).
Secondly, the Victorian studies utilised head injury data from two sources, the Transport Accident Commission and hospital admissions data. Transport Accident Commission data included all types of head injuries: fractures to the skull, brain injuries and lacerations to the head or scalp. (Cameron et al 1992, page 8). Hospital data included all acute admissions sustaining head injury. (Cameron et al 1992, page 13). ‘Acute’ in this sense does not mean severe, it only means immediate or sudden.
Injuries were also simplistically divided into ‘head injuries’ (which included admissions for multiple injuries) and other injuries (where there was any type of injury but no head injury). (Finch et al, Head injury reductions in Victoria two years after the introduction of mandatory bicycle helmet use, Report 51, Monash University Accident Research Centre, 1993, page 5)
Hospital admissions data was also not limited to admissions because of the head injuries (for example, it may include someone with very mild, superficial head injuries but also other, non-head related severe injuries).
These studies also included all types of head injuries no matter how superficial, including minor cuts and bruises. These studies may have also included neck injuries proactively counted in favour of the efficacy of helmet laws, not just ignored from the equation.
Thirdly, the studies rely heavily on the assumption that consistent reduction in cyclists’ head injuries prior to the introduction of mandatory helmet laws was the result of a consistent increase in the voluntary uptake of helmets. If this assumption is incorrect, then the subsequent findings of the studies about the additional effect of greater helmet use as a result of mandatory laws are flawed. The studies don’t appear to rely on any evidence for this assumption other than a loose correlation between reduced head injuries and helmet wearing rates, the latter ascertained from observational surveys. The studies also admit some possible failings of these observational surveys:
“It is possible that student surveys could be biased towards higher wearing rates (Heiman, 1987) as it is known that some students only wear helmets when leaving home and on approaching school.” (Finch et al, 1993, page 2)
Notably, this possible bias also relates to the same age groups of cyclists (aged 5-11 years, and aged 12 to 17 years) identified in the reports as suffering significant reduction in general cycling numbers as a result of helmet laws, making the correlation between helmet wearing rates and head injury rates after the introduction of helmet laws even more dubious.
Surprisingly, the third report in the series found that, three years after the introduction of mandatory helmet laws in Victoria, head injuries amongst cyclists involved in crashes with motor vehicles had actually increased. (Newstead et al: Bicyclist head injuries in Victoria three years after the introduction of mandatory helmet use, Report 75, Monash University Accident Research Centre, 1994)
This finding resulted in the conclusion that:
“The effect of the bicycle helmet law in reducing head injury rates below pre-law trend predictions for bicyclists injured in motor vehicle involved crashes was not clear.” (Newstead et al, 1994, Executive Summary)
But this was considered by the researchers to be an anomaly, and was subsequently addressed in a later report. (Carr et al: Evaluation of the bicycle helmet wearing law in Victoria during its first four years, Report 76, Monash University Accident Research Centre, 1995)
In fact, perhaps as a result of some post-publication editing, the third report states:
A subsequent MUARC report [Carr et al, 1995], which should be read in conjunction with this report, investigates bicycle injury data and the effect of the bicycle helmet law four years after its introduction. This report has revealed biases in the bicyclist injury data, certainly affecting the analysis and results presented here. This is shown by the analysis [Carr et al, 1995] which adjusts for the bias in the bicyclist injury data and reaches different conclusions to those drawn here.” (Newstead et al, 1994, Executive Summary)
The fourth study in the series found that:
“… [Newstead et al, 1994] noted that some differences between the two data sources used in their study – the Hospital Admissions data based on Health Department records of admissions to public hospitals, and the claims data from the Transport Accident Commission which is responsible for “no fault” injury compensation in the case of crashes involving motor vehicles. In addition there was also an apparent increase in head injuries in the third year of the legislation.”
“…. and it was concluded that a substantial part of the apparent increase in head injuries amongst crash-involved bicyclists related to a change in admissions policy in the Victorian hospital system (Casemix Funding), resulting from the use of incentives for hospitals based on their throughput and the number of injuries treated.” (Carr et al, 1995, Executive Summary)
The researchers concluded that changes to the way hospitals were funded meant that hospitals were more likely to formally record minor injuries, and more likely to admit for observation a patient with a suspected head injury instead of simply keep them in emergency for several hours. (Carr et al, 1995, page 2)
The study did not explain why this change in funding would affect head injury records more than any other form of injury records, against which head injury records were compared.
The researchers then ‘corrected’ the data after comparing all types of head injury (not just road related) for the period preceding the introduction of the new funding system, with the period assumed to be affected by the new funding system. The researchers then ‘corrected’ the reported number of head injuries for cyclists for each year by a ‘correction factor’ determined by comparing the number of total head injuries (not including cyclists or pedestrians) for each year against the number of head injuries (not including cyclists or pedestrians) for the base year of 1986/1987. Head injuries for pedestrians were used as a ‘surrogate variable’ for general road safety improvements.
The ‘corrected’ numbers of head injured cyclists were:
(Carr et al, 1995, table 2.2)
For 1993/94, this correction appears to have reduced the number of cyclists suffering head injury by 16%.
Only by use of data ‘corrected’ according to the above assumption, and then through heavy manipulation designed to accentuate the significance of the available data, does the fourth study overcome the findings of the third study and reiterate the findings of the first two studies in the series, arguing that increased helmet use has led to significant decreases in head injuries, but only when compared to the expected increases if helmet laws were not introduced.
However, far from there being clearly obvious reductions in injury such as those that followed the introduction of other road safety measures such as seatbelts and random breath testing, the results appear far from clear. For example, differences between actual head injuries before helmet laws were introduced and after the laws were introduced, are far from significant:
(Carr et al, 1995, figure 2.6)
The finding that head injuries were reduced because of helmet wearing is strongly based on a hypothesized increase in head injuries if helmet laws were not introduced. However, as demonstrated by the graph replicated at the top of this post, the first study in this series clearly identified that head injuries were actually decreasing before the laws were introduced.
The CARRS-Q publication cites this fourth study as finding a 39.5% reduction in head injury hospital admissions in Victoria after helmet laws were introduced, but fails to clarify the hypothesis upon which this figure is based. (CARRS-Q page 18). This fourth study actually found no reduction in the level of critical head injuries, only a reduction in ‘serious and severe’ head injuries. (Carr et al, 1995, page 22).
The CARRS-Q publication misleadingly states that:
“Helmet legislation was shown to account for a large proportion of the reduction, although the researchers acknowledge some of the reduction may be a result of a reduction of exposure to crash risk.” (CARRS-Q page 18)
While the actual report does conclude helmet laws has had a positive effect on head injuries, what the report actually concludes is:
“This evaluation has shown that substantial reductions have been achieved in both number and severity of head injuries to bicyclists admitted to hospital since the introduction of the mandatory helmet wearing law in mid- 1990. As there has been numerous other improvements in road safety in Victoria in this time frame, as well as increased availability of bicyclist paths etc., it is paramount that a cautious approach is taken when attributing specific reductions to particular countermeasures.”
“In this analysis of numbers of head injured, we were unable to include any reliable measures of exposure, and thus it is impossible to distinguish between reductions due to helmet wearing and reductions solely due to possible reductions in exposure. There has been some evidence recently (Mead,1993; Ratcliffe, 1992) that the introduction of mandatory helmet wearing laws has led to decreases in overall numbers of bicyclists; Finch et al [the second in this series of Victorian studies discussed above] showed that exposure for teenagers was reduced in Victoria as a result of the helmet wearing law.” (Carr et al, 1995, page 21)
Additionally, it appears this report also included all types of head injury, including superficial scalp injuries, and like the other reports discussed above and in a previous post, also excluded neck injuries.
Despite CARRS-Q’s claims, what a closer analysis of the actual reports shows is that there is no clear, unequivocal evidence that wearing a bicycle helmet provides any significant protection from serious or life-threatening injuries including neck injuries. It is only clear that helmets provide some increased protection from minor injuries, such as non-life threatening cuts, bumps and bruises, particularly for children cyclists.
The next post in this series will consider CARRS-Q’s claims that helmets laws do not discourage cycling, and that there are no significant economic benefits to repealing helmet laws. It will also consider more recent research produced since the CARRS-Q publication was released, that contradicts CARRS-Q’s findings.
If you are prone to head injuries or suffering from one right now. It would always be better to be prepared and be knowledgeable about the illness. Here are some good tips on how to treat head injuries.Don’t drink alcohol for at least 24 hours, don’t smoke.