2012 April

CARRS-Q Research – Part Four

Why you should dismiss the CARRS-Q bicycle helmet research

Monash University Research Used by CARRS-Q

Image Credit: Dr P Martin

This is the fourth post in a series (Read Part OnePart Two, Part Three) looking at the non-peer reviewed CARRS-Q publication entitled “Bicycle Helmet Research“, published in 2011 and widely relied upon to support mandatory helmet laws for cyclists.  This publication was commissioned by the State Government to support its policy of mandatory helmet laws in Queensland, Australia, in the face of criticisms of its lack of support for Brisbane’s public bike hire scheme.

The second post discussed the failings of the Cochrane Review, a review of pre-existing research used to support bicycle helmet efficacy, which was in turn heavily relied upon by the CARRS-Q publication to support the Queensland Government’s policy of mandatory helmet laws.

The third post looked at other bicycle injury related research utilised by the CARRS-Q publication, highlighting some of the failings of that research and how that original research has been misinterpreted by CARRS-Q to support mandatory helmet laws. This post looks at more of that research, focusing on a series of Victorian-based studies conducted by Monash University.

According to CARRS-Q, these Victorian studies:

“…  found that the number of bicyclists admitted to hospital with head injuries decreased following the introduction of compulsory helmet wearing.” (CARRS-Q, 2011, page 17)

The CARRS-Q publication goes on to state:

“These decreases were associated with an increase in helmet wearing rates, coupled with reduced levels of cycling.” (CARRS-Q, 2011, page 18)

The Victorian studies were undertaken by the Monash University Accident Research Centre.  These studies actually found that head injuries were significantly decreasing well before the introduction of helmet laws. (Cameron et al: Evaluation of bicycle helmet wearing law in Victoria during its first 12 months, Report 32, Monash University Accident Research Centre, 1992).

Below is a graph from the Victorian study (the arrow indicates when helmet laws were introduced):

(Cameron, M., Heiman, L., Neiger, D., Evaluation of bicycle helmet wearing law in Victoria during its first 12 months, (1992), Report 32, Monash University Accident Research Centre, figure 10.)

The first study in this series found that the rate of head injuries recorded for Melbourne cyclists did decrease, but found that the amount of decrease of head injury due to helmet use, when compared to rates of helmet use prior to making laws mandatory, identified some anomalies which:

“ … suggests that the number of cyclists [in Melbourne] involved in crashes with motor vehicles has decreased during the post-law period, either due to a reduction in bicycle use or a reduction in the risk of crash involvement.” (Cameron et al, 1992, page 8)

This series of studies found that there was a reduction of 37% (based on Transport Accident Commission data) or 51% (based on hospital admissions data) in the number of cyclists killed or suffering head injuries during the 12 months following the introduction of mandatory helmet laws in Victoria.  However the studies also found that there was also a substantial reduction of 21% (Transport Accident Commission data) or 24% (hospital admissions data) in the number of severely injured cyclists who did not sustain head injury.  In short, the study found there appeared to be a general injury reduction in all crashes involving cyclists, not just head injuries.

This finding is also supported by other research that shows that injuries for all road users including pedestrians have significantly decreased, and therefore such decreases cannot be attributed to bicycle helmet laws.

The Victorian studies also found that the reduction in death or injury for cyclists did not match the expected reduction, when compared to pre-legislation helmet wearing rates.  In other words, if previous reductions could be attributed to voluntary helmet wearing, these reductions were too great to be attributed to compulsory helmet wearing rates.

These findings were supported by the second study in the series, which confirmed that:

[t]he proportion of all injured cases with a head injury in 1992 was significantly less than that projected on the basis of continuing pre-law trends.  The mechanisms by which this reduction was achieved seem to be twofold: a reduction in the number of bicyclists involved in crashes resulting in severe injury and a reduction in the risk of head injury for bicyclists who were severely injured.” (Cameron et al 1992, page 10)

So something else, other than helmets, may be responsible for the decrease in head injuries over and above general decreases in all types of injuries.

The answer may actually lie in this same series of studies.  These studies also found that while the rate of helmet wearing increased as a result of the laws, the total cycling rates for children significantly decreased as a result of the introduction of helmet laws. The first Victorian study actually found that cycling decreased by 15% for 5-11 year olds, and by 44% for teenagers. The study also concluded that there may also have been an increase in adult cyclists, however such a finding is not as well documented:

“A comparison of the 1991 survey results for adults with a similar survey in 1987/88 (while a less direct comparison because of the nearly four year gap and different times of the year) showed that adult bicycle use had increased by 58%.  The increase in adult cycling appears to have off-set the decrease in child cycling.  Preliminary results from the 1992 survey conducted by MUARC [Monash University Accident Research Centre] indicate that these trends have been maintained.” (Cameron et al 1992, page i)

In fact, despite this conclusion, this same study identifies a similar survey carried out in 1990, carried out in the same locations and at the same time of year. A comparison of the 1991 survey with this similar survey carried out just one year prior paints a very different picture: it shows a drop in total cyclist numbers (including adults) of 45%. (Cameron et al 1992, pages 5 and 6).

Firstly based on the other research discussed above and in a previous post, it is clear that children and teenagers are far more likely to be involved in an accident than adult cyclists. In short, the decrease in head injuries could be explained by different riding patterns between children (who significantly decreased in riding rates during the relevant periods of the studies) and adults who are inherently less likely to crash or sustain serious injury in the event of a crash (who may have increased during the relevant periods).

Secondly, the Victorian studies utilised head injury data from two sources, the Transport Accident Commission and hospital admissions data.  Transport Accident Commission data included all types of head injuries: fractures to the skull, brain injuries and lacerations to the head or scalp. (Cameron et al 1992, page 8). Hospital data included all acute admissions sustaining head injury. (Cameron et al 1992, page 13).  ‘Acute’ in this sense does not mean severe, it only means immediate or sudden.

Injuries were also simplistically divided into ‘head injuries’ (which included admissions for multiple injuries) and other injuries (where there was any type of injury but no head injury). (Finch et al, Head injury reductions in Victoria two years after the introduction of mandatory bicycle helmet use, Report 51, Monash University Accident Research Centre, 1993, page 5)

Hospital admissions data was also not limited to admissions because of the head injuries (for example, it may include someone with very mild, superficial head injuries but also other, non-head related severe injuries).

These studies also included all types of head injuries no matter how superficial, including minor cuts and bruises. These studies may have also included neck injuries proactively counted in favour of the efficacy of helmet laws, not just ignored from the equation.

Thirdly, the studies rely heavily on the assumption that consistent reduction in cyclists’ head injuries prior to the introduction of mandatory helmet laws was the result of a consistent increase in the voluntary uptake of helmets.  If this assumption is incorrect, then the subsequent findings of the studies about the additional effect of greater helmet use as a result of mandatory laws are flawed.  The studies don’t appear to rely on any evidence for this assumption other than a loose correlation between reduced head injuries and helmet wearing rates, the latter ascertained from observational surveys.   The studies also admit some possible failings of these observational surveys:

“It is possible that student surveys could be biased towards higher wearing rates (Heiman, 1987) as it is known that some students only wear helmets when leaving home and on approaching school.” (Finch et al, 1993, page 2)

Notably, this possible bias also relates to the same age groups of cyclists (aged 5-11 years, and aged 12 to 17 years) identified in the reports as suffering significant reduction in general cycling numbers as a result of helmet laws, making the correlation between helmet wearing rates and head injury rates after the introduction of helmet laws even more dubious.

Surprisingly, the third report in the series found that, three years after the introduction of mandatory helmet laws in Victoria, head injuries amongst cyclists involved in crashes with motor vehicles had actually increased. (Newstead et al: Bicyclist head injuries in Victoria three years after the introduction of mandatory helmet use, Report 75, Monash University Accident Research Centre, 1994)

This finding resulted in the conclusion that:

“The effect of the bicycle helmet law in reducing head injury rates below pre-law trend predictions for bicyclists injured in motor vehicle involved crashes was not clear.” (Newstead et al, 1994, Executive Summary)

But this was considered by the researchers to be an anomaly, and was subsequently addressed in a later report. (Carr et al: Evaluation of the bicycle helmet wearing law in Victoria during its first four years, Report 76, Monash University Accident Research Centre, 1995)

In fact, perhaps as a result of some post-publication editing, the third report states:

A subsequent MUARC report [Carr et al, 1995], which should be read in conjunction with this report, investigates bicycle injury data and the effect of the bicycle helmet law four years after its introduction.  This report has revealed biases in the bicyclist injury data, certainly affecting the analysis and results presented here.  This is shown by the analysis [Carr et al, 1995] which adjusts for the bias in the bicyclist injury data and reaches different conclusions to those drawn here.” (Newstead et al, 1994, Executive Summary)

The fourth study in the series found that:

“… [Newstead et al, 1994] noted that some differences between the two data sources used in their study – the Hospital Admissions data based on Health Department records of admissions to public hospitals, and the claims data from the Transport Accident Commission which is responsible for “no fault” injury compensation in the case of crashes involving motor vehicles.  In addition there was also an apparent increase in head injuries in the third year of the legislation.”

“…. and it was concluded that a substantial part of the apparent increase in head injuries amongst crash-involved bicyclists related to a change in admissions policy in the Victorian hospital system (Casemix Funding), resulting from the use of incentives for hospitals based on their throughput and the number of injuries treated.” (Carr et al, 1995, Executive Summary)

The researchers concluded that changes to the way hospitals were funded meant that hospitals were more likely to formally record minor injuries, and more likely to admit for observation a patient with a suspected head injury instead of simply keep them in emergency for several hours. (Carr et al, 1995, page 2)

The study did not explain why this change in funding would affect head injury records more than any other form of injury records, against which head injury records were compared.

The researchers then ‘corrected’ the data after comparing all types of head injury (not just road related) for the period preceding the introduction of the new funding system, with the period assumed to be affected by the new funding system.  The researchers then ‘corrected’ the reported number of head injuries for cyclists for each year by a ‘correction factor’ determined by comparing the number of total head injuries (not including cyclists or pedestrians) for each year against the number of head injuries (not including cyclists or pedestrians) for the base year of 1986/1987.  Head injuries for pedestrians were used as a ‘surrogate variable’ for general road safety improvements.

The ‘corrected’ numbers of head injured cyclists were:

(Carr et al, 1995, table 2.2)

For 1993/94, this correction appears to have reduced the number of cyclists suffering head injury by 16%.

Only by use of data ‘corrected’ according to the above assumption, and then through heavy manipulation designed to accentuate the significance of the available data, does the fourth study overcome the findings of the third study and reiterate the findings of the first two studies in the series, arguing that increased helmet use has led to significant decreases in head injuries, but only when compared to the expected increases if helmet laws were not introduced.

However, far from there being clearly obvious reductions in injury such as those that followed the introduction of other road safety measures such as seatbelts and random breath testing, the results appear far from clear. For example, differences between actual head injuries before helmet laws were introduced and after the laws were introduced, are far from significant:

(Carr et al, 1995, figure 2.6)

The finding that head injuries were reduced because of helmet wearing is strongly based on a hypothesized increase in head injuries if helmet laws were not introduced. However, as demonstrated by the graph replicated at the top of this post, the first study in this series clearly identified that head injuries were actually decreasing before the laws were introduced.

The CARRS-Q publication cites this fourth study as finding a 39.5% reduction in head injury hospital admissions in Victoria after helmet laws were introduced, but fails to clarify the hypothesis upon which this figure is based. (CARRS-Q page 18). This fourth study actually found no reduction in the level of critical head injuries, only a reduction in ‘serious and severe’ head injuries. (Carr et al, 1995, page 22).

The CARRS-Q publication misleadingly states that:

“Helmet legislation was shown to account for a large proportion of the reduction, although the researchers acknowledge some of the reduction may be a result of a reduction of exposure to crash risk.” (CARRS-Q page 18)

While the actual report does conclude helmet laws has had a positive effect on head injuries, what the report actually concludes is:

“This evaluation has shown that substantial reductions have been achieved in both number and severity of head injuries to bicyclists admitted to hospital since the introduction of the mandatory helmet wearing law in mid- 1990.  As there has been numerous other improvements in road safety in Victoria in this time frame, as well as increased availability of bicyclist paths etc., it is paramount that a cautious approach is taken when attributing specific reductions to particular countermeasures.”

“In this analysis of numbers of head injured, we were unable to include any reliable measures of exposure, and thus it is impossible to distinguish between reductions due to helmet wearing and reductions solely due to possible reductions in exposure. There has been some evidence recently (Mead,1993; Ratcliffe, 1992) that the introduction of mandatory helmet wearing laws has led to decreases in overall numbers of bicyclists; Finch et al [the second in this series of Victorian studies discussed above] showed that exposure for teenagers was reduced in Victoria as a result of the helmet wearing law.” (Carr et al, 1995, page 21)

Additionally, it appears this report also included all types of head injury, including superficial scalp injuries, and like the other reports discussed above and in a previous post, also excluded neck injuries.

Despite CARRS-Q’s claims, what a closer analysis of the actual reports shows is that there is no clear, unequivocal evidence that wearing a bicycle helmet provides any significant protection from serious or life-threatening injuries including neck injuries. It is only clear that helmets provide some increased protection from minor injuries, such as non-life threatening cuts, bumps and bruises, particularly for children cyclists.

The next post in this series will consider CARRS-Q’s claims that helmets laws do not discourage cycling, and that there are no significant economic benefits to repealing helmet laws.  It will also consider more recent research produced since the CARRS-Q publication was released, that contradicts CARRS-Q’s findings.

CARRS-Q Research – Part Three

Photo Credit: Dr Paul Martin

Why you should dismiss the CARRS-Q bicycle helmet research

Other Research CARRS-Q Used

This is the third post in a series (Read Part One, Part Two) looking at the non-peer reviewed CARRS-Q publication entitled “Bicycle Helmet Research“, published in 2011 and widely relied upon to support mandatory helmet laws for cyclists.  This publication was commissioned by the State Government to support its policy of mandatory helmet laws in Queensland, Australia, in the face of criticisms of its lack of support for Brisbane’s public bike hire scheme.

Part Two of this series discussed the failings of the Cochrane Review, a review of pre-existing research supporting bicycle helmet efficacy, which was in turn heavily relied upon by the CARRS-Q publication to support the Queensland Government’s policy of mandatory helmet laws for all cyclists.

This post considers other bicycle injury related research utilised by the CARRS-Q publication, highlighting some of the failings of that research and demonstrating how that original research has been misinterpreted by CARRS-Q to support mandatory helmet laws.

The Canadian Report

As well as the Cochrane Review, the CARRS-Q publication relies on Ontario-based research to conclude that helmets greatly reduce the likelihood of bicycle related head injuries (Rowe et al: Bicyclist and environmental factors associated with fatal bicycle-related trauma in Ontario, Canada Medical Association Journal 1995).

The CARRS-Q publication states that:

“Research from Canada examining bicycle fatalities found that the vast majority involved bicyclists not wearing a helmet (96%), although some fatalities did involve helmeted cyclists …” (CARRS-Q, 2011, pages 11 and 12)

Importantly, the CARRS-Q publication does not put this in context:  the Canadian report concerned only 212 cyclist deaths in Ontario when helmet wearing was not compulsory.  The report made no analysis of the helmet wearing rates of the general cycling community.  By quoting the research as finding 96% of fatalities were not wearing a helmet, CARRS-Q is misrepresenting this research as finding helmet use had some statistical relevance to the likelihood of injury, when in reality there is no indication in the Canadian report that wearing a helmet had any effect on the likelihood of death. The Canadian Report itself only concluded that:

“[b]icycle-related deaths result from factors that are generally avoidable.  Identifiable risk factors other than lack of helmet use suggest that additional research is required to determine the benefits of preventative interventions aimed at reducing the number of such deaths.  Age-specific strategies appear warranted.” (Rowe et al, 1995, page 45)

The Canadian report also stated that:

“In all, 193 (91%) of the events were collisions with motor vehicles… Such events were responsible for all the deaths involving children less than 10 years of age and 88% of the deaths involving those 10 to 19.” (Rowe et al, 1995, page 48)

It should be concluded from the Canadian report that a cyclist’s age has more to do with the likelihood of death than whether they are wearing a helmet.

The Canadian report also concluded that:

“Unfortunately, because of our rudimentary understanding of risk factors associated with fatal bicycle-related trauma, countermeasures or preventive interventions have been limited to the promotion of bicycle helmet use. In addition, most of the potential interventions have yet to be subjected to adequate scientific testing.” (Rowe et al, 1995, page 49)

The Canadian Report also found that:

“Most (91%) of the deaths resulted from collisions between motor vehicles and bicycles.  Age did play an important role in crash characteristics.” (Rowe et al, 1995, page 48)

Within a year of the Canadian study being published, Ontario introduced mandatory helmet laws only for children cyclists.  Not adult cyclists.  This was in accordance with those findings.

More recent research by CARRS-Q confirms that teenagers are risk takers.

The general effect of helmet use on injury is mentioned briefly in the Canadian report, with the Canadian report citing current estimates for head and brain injury reduction from helmet wearing.  However these estimates are attributed to earlier studies, specifically the third and fourth studies utilised in the Cochrane Review discussed in a previous post, including a report focusing solely on head injuries for children under the age of 15.

The Melbourne Study

The CARRS-Q publication also relies on a Melbourne study comparing head injury rates between helmeted and non-helmeted cyclists, and to the first object struck by cyclist in a crash.  (McDermott et al: Bicyclist head injury prevention by helmets and mandatory wearing legislation in Victoria, Australia, Annals of the Royal College of Surgeons of England, 1995)

According to the CARRS-Q assessment of this study:

“When fatalities were considered, helmet wearing cyclists had fatality rate of 0.4% while unhelmeted riders had a fatality rate of 0.9%. However this finding was not significant.” (CARRS-Q, 2011, page 12)

While noting that head injuries were less frequent, and less severe, for helmet-wearing bicyclists, the CARRS-Q publication also noted that the frequency of neck injuries was significantly increased by helmet wearing.

The Melbourne study relied on a previous study co-authored by the same author for its comparison of head injury rates between helmeted and non-helmeted cyclists. (McDermott et al: The effectiveness of bicyclist helmets: a study of 1710 casualties, J Trauma 1993).

This earlier study was one of the studies relied upon for the Cochrane Review and discussed in a previous post.  This previous study found helmeted cyclists were more likely to collide with motor vehicles than non-helmeted cyclists, and that there were no significant differences in mortality rates between helmeted and non-helmeted cyclists, and which also identified an increase in neck injuries attributable to helmet use.

The Melbourne study referenced by the CARRS-Q publication also found that total cyclist fatality numbers decreased from 77 for the three years prior to the introduction of mandatory helmet legislation in Victoria, to 41 for the first three years after the introduction.  However, importantly, this study does not take into account the reduced numbers of cyclists during this period, or which age groups had the greatest reductions in cycling rates. These reduced numbers are discussed in the next post.

The Melbourne study referenced by CARRS-Q also noted its results resembled the results of a Seattle study and a study focussing on child bicyclists.  However these studies were the very same studies already discussed in the previous post, the third and fourth studies relied upon for the Cochrane Review.

In short, as well as the actual findings of these studies differing somewhat from the conclusions drawn from them by CARRS-Q, a lot of the claims that these studies support each other’s findings are spurious given that the studies either reference each other or reference the same sources.

The New Zealand Report

The CARRS-Q publication also refers to a New Zealand study that indicates helmet wearing decreased head injuries for non-motor vehicle related crashes.  (Povey et al: Cycle Helmet Effectiveness in New Zealand, Accident Analysis and Prevention, 1999)

The CARRS-Q publication states of the New Zealand report that this reduction of head injuries:

“ … was evident across age groups, being strongest among children of primary school age.  This trend was also found for bicycle crashes involving a motor vehicle, although the effect was not as great.” (CARRS-Q, 2011, page 12)

This New Zealand study only differentiated age groups for non-motor vehicle related crashes. The study was also commissioned by the Land Transport Safety Authority (LTSA), which was the government agency responsible for the introduction of mandatory helmet laws in New Zealand 4 years prior.

This New Zealand study concluded that:

“The relatively large increase in helmet wearing associated with the passing of a compulsory helmet wearing law in 1994 [in New Zealand] reduced head injuries by between 24 and 32% in non-motor vehicle crashes, and by 20% in motor vehicle crashes.” (Povey et al, 1999, abstract)

This reduction is significantly lower than CARRS-Q’s claim that helmets reduce likelihood of head injury by 69%.  This New Zealand study also concluded that:

“No increase or decrease in the severity of head injuries for which cyclists were hospitalised over this period could be detected.  This may have been due to the small and highly variable number of ‘high severity’ injuries.” (Povey et al, 1999, abstract)

This New Zealand study also identified significant disparity in head injuries across different age groups:

“For an increase in 5 percentage points in the helmet wearing rate, the corresponding decreases in head injuries in non-motor vehicle accidents were estimated to be 10.2, 5.3 and 3.2% for children of primary school age (5-12 years), secondary school age (13-18 years), and adults respectively.” (Povey et al, 1999, abstract)

In line with the Canadian report discussed above, there were significant differences in injuries depending on age.

The findings of this New Zealand study has been heavily criticised due to anomalies between voluntary uptake in helmet use prior to mandatory laws, and mandatory helmet use:

“The pre-law increase in adults wearing helmets (from 30% in 1990 to 43% in 1993) was accompanied by a fall of 45 head injuries per 100 limb injuries (i.e. -3.47 for every 1% increase in helmet wearing) compared to a fall of just 11 head injuries when helmet wearing increased from 43% to 93% with the law (-0.23 for every 1% increase in wearing). Unless voluntary wearing is 15 times more effective in reducing head injuries, it seems likely that the claimed 20% – 32% falls in head injuries with increased helmet wearing were, in reality, an artefact caused by failure to fit time trends in their model.” (Robinson: Changes in head injury with the New Zealand bicycle helmet law, Accident Analysis and Prevention, 2001).

A more recent study (covered here), also focusing on the effect of helmet use in New Zealand, found that with inclusion of neck injury, helmet use has no net protective effect on cyclists’ injury.  (Elvik: Publication bias and time-trend bias in meta-analysis of bicycle helmet efficacy: a re-analysis of Attewell, Glase and McFadden, Accident Analysis and Prevention, 2011).

This more recent study is also consistent with the Melbourne study discussed above, on the increased risk of neck injury from helmet wearing.

The Queensland Report

The CARRS-Q publication (p16) also claims mandatory helmet laws have been proven not to reduce head injuries simply because they reduce the average number of cyclists.

The CARRS-Q publication relies on research that compared the reduction of Queensland cyclists’ head injuries after the introduction of mandatory helmet laws, with the reduction of other types of injuries suffered by Queensland cyclists.[1] (King et al: Bicycle helmet legislation and enforcement in Queensland 1991-1993: effects on helmet wearing and crashes, Road User Behaviour Section, Road Transport and Safety Division, Queensland Transport, 1994).

According to CARRS-Q, this Queensland report found a decrease in the rate of cyclists’ head injury compared to other types of reported injury that:

“ … could not be explained as an effect of less cycling.” (CARRS-Q, 2011, page 16)

This Queensland report was written by two employees of the Queensland State Government’s Queensland Transport Department in 1996, including a manager of the Department’s Road Transport Safety Division.  That manager is also one of the four authors of the CARRS-Q publication.

The report was written for presentation at the Third International Conference on Injury Prevention and Control, held in Melbourne in February 1996.  This report was one of a number of pro-helmet presentations amongst a broader programme of social injury prevention presentations.

This report was not peer-reviewed.  It is also important to consider whether the report’s presentation at the Conference should give this report any academic authority. The Conference director has said of this Conference that:

” I saw our intentions to schedule only excellent and innovative papers come falling around our ears.  The reason? A large proportion of submitted papers consisted of injury incidence surveys.  Many had intervention recommendations bearing no relation to the actual study, simply tacked on to the end.  Others were program descriptive papers, many without rigorous evaluation.  … In the end fiscal pragmatism won out – we found a place in the program for all authors in the most creative way we could.”

The majority of the Queensland report focusses on documenting the introduction of mandatory helmet laws in Queensland, and the subsequent enforcement by police.  The reports provides data about the number and type of fines issued by police for infringements of the legislation, and the results of some observational surveys in and around schools and other areas, about helmet wearing rates.

Notably, the report identifies that the number of fines issued for failing to wear a bicycle helmet in 1993 were similar to the number of drink driving offences detected.  It also states that the penalty was set at $30 as this was the approximate cost of a helmet.

The report also presents police crash data from January 1991 to September 1993, broken down into two levels of severity: fatal or hospital treatment being classified as severe, and other medical treatment being classified as less severe. This data is also presented in three categories: before legislation, after legislation but before fines were introduced, and after fines were introduced.  Here is how the data was presented in the report:

The report concludes from this data that:

“… there was clearly a drop in crashes in the first quarter after helmet wearing became compulsory, followed by a rebound to what appears to be a lower quarterly crash rate than previously.”

If one charts the above report, the injury trends appear as follows (with the vertical lines indicating the introduction of helmet laws, and the later introduction of fines for enforcement):

Cyclist crashes by severity: Fatal or hospital treatment compared to other medical treatment (as per King et al)

However a different picture of helmet efficacy emerges if these 1990-1993 figures are merged with similar statistics presented on page 26 the CARRS-Q publication, categorising the latter in the same way as the Queensland publication:

Cyclist crashes by severity: Fatal or hospital treatment compared to other medical treatment (1990 to 1993 as per King et al); Fatalities and hospitalisations compared to medical treatment and minor injury (1994 to 2008 as per CARRS-Q)

Importantly, the above data does not differentiate between head injuries and other injuries.

The Queensland report agreed that it could be argued that the above decreases could be explained solely by reductions in cycling.  The report addressed this by presenting data collected by the Queensland Injury Surveillance and Prevention Program, from a sample of hospitals in the southern metropolitan area of Brisbane.  This data was also categorised as treated (being less serious) and admitted (assumed to be more serious) and also broken up into before legislation, after legislation but before fines were introduced, and after fines were introduced.  Actual data was not presented, just mean quarterly averages:

From this data, the report concludes that:

“The figures show that in every comparison the decrease in head injuries among cyclists was greater than the decrease among injuries to other parts of the body, which suggests that bicycle helmet legislation has been effective in reducing head injuries among cyclists.”

It is clear that the data was, at best, not broadly representative of the Queensland crash and injury statistics.  The CARRS-Q publication itself states that:

“When examining bicycle crash patterns using police-reported data, it is important to acknowledge some of the limitations associated with the data.  Research indicates that there is significant underreporting of bicycle crashes …” (CARRS-Q, 2011, page 25)

In fact the research that CARRS-Q goes on to quote estimates that police data could represent as little as 3.5% of crashes, and over-represents the more serious bicycle crashes.

The hospital data represented above in TABLE IV used by the Queensland report is also limited to only a handful of hospitals, all within the Brisbane metropolitan region.  Additional later data, comparing head injuries to other types of injuries per year since the introduction of helmet laws, was not used by CARRS-Q.  Instead, CARRS-Q only compared the injury type between helmeted and non-helmeted cyclists.  The many problems with this approach are discussed in the last post of this series.

The data used in this Queensland report is also limited to just the three year period before, during and after the introduction of mandatory helmet laws. As demonstrated by the above discussions, limiting data to such a small period can present an unrealistic picture of the efficacy of helmet laws.

It would also appear from the CARRS-Q publication that this report does not take into account the effect of changes to the demographic of cycling caused by helmet laws.  In other words, as evidenced by other studies discussed above and in the next post, mandatory helmet laws deter cycling the most amongst adolescents and young children, which are the age groups most likely to have a collision with a motor vehicle (which in turn are likely to result in more serious injuries or death) or most likely to suffer head injury in a non-vehicle related accident.

A number of other researchers have also been highly critical of this Queensland study.  It has been said with classic academic understatement that a number of State–based studies designed to evaluate the efficacy of helmet legislation, including this Queensland study, were:

“ … less sophisticated in a statistical sense.”

(Hendrie et al, Road Accident Prevention Research Unit, Department of Public Health, University of Western Australia: An economic evaluation of the mandatory bicycle helmet legislation in Western Australia, Insurance Commission of Western Australia Conference on Road Safety, 1999).

One academic has pointed out that the reduction in percentage of head injury compared to other types of injury in the Queensland report was similar across all road user types, including pedestrians, not just cyclists.  This strongly indicates that cyclists’ head injury reduction was due to other road safety measures, not helmets. (Robinson: Is there any reliable evidence that Australian helmet legislation works? Velo Australia International Bicycle Conference, Freemantle, Western Australia, 1996)

It is also not clear without reading the original study whether, like a number of other studies relied upon by CARRS-Q, ‘head injury’ included all types of head injury including minor cuts, bumps and bruises.

This Queensland report also failed to consider any increases in neck injury after the introduction of mandatory helmet laws.

It also appears that the Government Department influenced how the findings of the Queensland report were presented in the CARRS-Q publication. The original draft of the CARRS-Q publication did not rely on this Queensland report for any findings about helmet efficacy, only for findings related to the increase of helmet use as a result of the helmet laws.  It was only after review of the draft CARRS-Q publication by Queensland Transport that the CARRS-Q publication was amended to use the Queensland study in this way.

Looking at the original data of police and hospital data presented in the Queensland report, here is the pattern of injuries

Despite CARRS-Q’s claims, the actual research referenced in the CARRS-Q publication indicates significant differences in helmet efficacy for children and adults. It also identifies motor vehicles as the biggest factor in cyclists’ head injury rates. Even the most minor reduction to cyclists’ exposure to collision with motor vehicles, will prevent far more head injuries than bicycle helmets could ever achieve. This could be done through driver education, awareness and better cycling infrastructure/road layout changes, speed limit reviews and so on, all of which will follow from greater cycling participation.

The next post in this series will look at a series of Victorian-based studies conducted by Monash University, also relied upon by CARRS-Q to support mandatory helmet laws.


[1] King, M., Fraine, g., Bicycle helmet legislation and enforcement in Queensland 1991-1993: effects on helmet wearing and crashes, (1994), Road User Behaviour Section, Road Transport and Safety Division, Queensland Transport.

CARRS-Q Research – Part Two

Why you should dismiss the CARRS-Q bicycle helmet research

The Cochrane Review

This is the second post in a series (Read Part One) looking at the non-peer reviewed CARRS-Q publication entitled Bicycle Helmet Research, published in 2011 and widely relied upon to support mandatory helmet laws for cyclists.  This publication was commissioned by the State Government to support its policy of mandatory helmet laws in Queensland, Australia.  This post considers helmet efficacy research conducted by the Cochrane Collaboration, which was heavily relied upon by the CARRS-Q publication.

The CARRS-Q publication’s key finding is that:

“[c]urrent bicycle helmet wearing rates are halving the number of head injuries experienced by Queensland cyclists.”

In its publication CARRS-Q goes so far as to say that helmets reduce the risk of head injury by 69%, and serious brain injury by 74% (and these figures are being quoted, unquestioned, by the media).  While these figures are far in excess of the findings of much of the research it reviews, CARRS-Q presents these high figures by relying heavily on just the one report published by a company called the Cochrane Collaboration  (Thompson et al: Helmets for preventing head and facial injuries in bicyclists, Cochrane Database of Systematic Reviews 1999).

The Cochrane Collaboration is a UK based company, listed as a charity, that presents itself as an international network of volunteer contributors whose writings on issues of health and welfare are published through a medium called the Cochrane Library.  The Cochrane Collaboration is predominately supported by national and trans-national government funding. Typically a Cochrane Review is considered the ‘gold standard’ with respect to medical meta-analyses but this doesn’t imply that Cochrane Reviews are infallible or will be interpreted correctly.

The Cochrane review used data that compared bicyclists who crashed and sustained head injury against bicyclists who crashed and did not sustain head injury, and then assessed helmet wearing rates amongst both groups.

The authors of the Cochrane Review did not collect any new data but instead relied on pre-existing data.  Of all the academic data available about helmet use, the authors of the Cochrane Review found only seven studies that the authors claimed were:

“… conducted to the most rigorous scientific standards.” (CARRS-Q, 2011, page 11)

Four of the seven studies used were previous studies conducted by one of the authors themselves, and all the chosen studies supported bicycle helmet use.

The first study utilised in the Cochrane review related to 1,040 Emergency Department visits to a hospital in Cambridge, UK in 1992 (Maimaris et al: Injury patterns in cyclists attending an accident and emergency department: a comparison of helmet wearers and non-wearers, British Medical Journal 1994).

This study found that head injuries were suffered by:

  • 8.1% of non-helmeted cyclists who owned a helmet,
  • 9.2% of non-helmeted cyclists who didn’t own a helmet, and
  • 3.5% of helmeted cyclists.

Even though helmets were not mandatory in the environment where this research was gathered, no attempt was made to investigate or exclude any other factors influencing certain cyclists choosing to wear a helmet that may also influence the likelihood of injury. For example, with no mandatory laws requiring helmets, it is very likely that people who voluntary choose to wear helmets are more risk-averse or belong to a data sub-set less likely to have a cycling related injury (for example age, riding location, riding environment, etc.)

The second study utilised in the Cochrane review related to 1,710 cyclists treated in two Melbourne hospitals during a period spanning 1987 and 1989, prior to the introduction of mandatory helmet laws. (McDermott et al: The effectiveness of bicyclist helmets: a study of 1,710 casualties, Journal of Trauma 1993).

This second study concluded that, prior to the introduction of mandatory helmet laws, 21.1% of helmeted cyclists suffered head injury compared to 34.4% of unhelmeted cyclists.

However this study also found that helmeted cyclists were more likely to collide with motor vehicles than non-helmeted cyclists (17.6% to 14.5%), and importantly that there were no significant differences in mortality rates between helmeted and non-helmeted cyclists.

This Melbourne study also identified an increase in neck injuries that is attributable to helmet use, but this is not mentioned at all in the CARRS-Q publication. This finding is important, because it gels with other more recent research discussed elsewhere in this series of posts.

This Melbourne study is also discussed further in the next post. It is also relied upon by CARRS-Q to further support its findings, despite the Melbourne study being one of the studies utilised in the Cochrane Review itself.

The third study utilised by the Cochrane Review related to 445 children cyclists under 15 years old who were treated at two Brisbane hospitals during a period spanning 1991 and 1992. (Thomas et al: Effectiveness of bicycle helmets in preventing head injury in children: case control study, British Medical Journal 1994).

However this Brisbane-based study included any injury to the skull or scalp in its definition of head injury. Any scratch or bruise no matter how inconsequential chalked up a result in favour of helmets. Brain injury was determined as instances where the child reportedly lost consciousness.

The fourth study was conducted by one of the authors of the Cochrane Review, and related to 668 cyclists treated for head injuries at hospitals in Seattle, USA during a period spanning 1986 and 1987. (Thompson et al: A case control study on the effectiveness of bicycle safety helmets, New England Journal of Medicine 1989)

This Seattle-based study also included superficial contusions, abrasions and lacerations in its definition of head injury, as well as skull fractures and brain injuries.

The fifth study was also conducted by one of the authors of the Review, concurrently with the above study, and related to 531 cyclists treated for facial injuries at hospitals in Seattle, USA during the same period spanning 1986 and 1987. (Thompson et al: Effectiveness of bicycle safety helmets in preventing serious facial injury, Journal of the American Medical Association 1996)

This study related to facial injuries only, not head injuries.  The results of this study was that helmets had little or no protective effect on overall risk of facial injury, but did protect against serious upper facial injury.

The sixth and seventh studies, also conducted by the same author, were conducted concurrently from 1992 to 1994 and also related to cyclists treated at Seattle, USA hospitals for facial injuries and head injuries. (Thompson et al: Effectiveness of bicycle safety helmets in head injuries: a case control study, Journal of the American Medical Association 1996; Thompson et al: Effectiveness of bicycle safety helmets in preventing serious facial injury, Journal of the American Medical Association 1996)

Only one of the seven studies used in the Cochrane Review included deaths at the scene of the crash or those who were declared dead on arrival to the hospital. (Thompson et al: Effectiveness of bicycle safety helmets in head injuries: a case control study, Journal of the American Medical Association 1996).

Much like the first study, the third, fourth, fifth and sixth studies all included cyclists of all ages and also related to environments where helmets were not mandatory.  Similarly, these reports do not explore or compensate for any other differences between riders who voluntarily chose to wear helmets even though they are not compelled to by law, and riders who do not, such as age, riding conditions, or purpose of travel.  Nor do any of the reports explore the possibility of risk compensation: whether riders take more or less risks in their riding style or choice of where to ride, depending on whether they are wearing a helmet. For example, in environments where helmet wearing rates are extremely low and cycling rates (and safety) are high, such as The Netherlands, there is very unlikely to be any correlation between risky behaviour and not wearing helmets.  In the contrary, only a very small number of cyclists who are partaking in activities that are perceived to be of a higher risk, for example bike racing or sport-orientated cycling, are likely to wear a helmet. This is important.

All of the above raises three very major concerns with the Cochrane Review:

  1. The Cochrane Review mixes results from children-focussed research with all-age research.  It therefore cannot be relied upon if children have a different propensity for injuries compared to other types of cyclists. Other research discussed in other posts in this series indicates this is true.
  2. The Cochrane Review cannot be relied upon if, where helmets are not mandatory, there are any significant differences in riding patterns between cyclists who voluntarily choose to wear a helmet and those that do not.  For example, some people who choose to ride safely or in safer circumstances may also choose to wear a helmet.
  3. The Cochrane Review cannot be relied upon for any findings relating any effect that helmets may have on injuries serious enough to possibly cause death. Excluding the most serious injuries from the research will overemphasize the impact that superficial injuries will have on the findings.  It will increase the statistical relevance of scratches, bumps and bruises.

The Cochrane review concludes that helmet wearing reduces the likelihood of head injury by 69% where a cyclist was involved in a crash with a motor vehicle, and 68% in all other types of crashes.  Despite other research, available to CARRS-Q, discussed in other posts in this series arriving at significantly smaller numbers, this headline-grabbing statistic is the one CARRS-Q relies upon.

The Cochrane Review also states that over 90% of cyclist deaths are caused by collisions with motor vehicles.

From the reports above, it could equally be determined that people who are likely to own a helmet are the type of people less likely to have an accident. It could also be determined that if one wears a helmet, one’s risk of a collision with a motor vehicle is increased.

Very recent research by CARRS-Q has even confirmed this, finding that cyclists, notably teenagers, who choose to not wear helmets (despite mandatory laws) may elicit riskier behaviour and are ten times more likely to be involved in a ‘conflict’, such as swerving or emergency braking to avoid a collision.

While such findings may appear spurious, the potential for the Cochrane Review findings to support such conclusions is important. As will be discussed in the other posts in this series, there is additional significant evidence that reductions in head injury have more to do with helmet laws reducing the number of cyclists particularly risk-taking teenagers, than any actual protection helmets offer in the event of a crash.  There is also additional significant evidence that wearing a helmet makes cyclists feel safer, thereby taking more risks, and that motorists take more risks when driving amongst helmeted cyclists.

One thing is extremely clear from the research used in the Cochrane Review, that is not highlighted in the CARRS-Q publication:  these reports all support the contention that many, many more deaths and injuries can be prevented by taking steps to ensure motor vehicles don’t collide with cyclists, than could ever be prevented by helmets.

The CARRS-Q publication relied on other research as well as the Cochrane Review. Other posts in this series critically discuss much of this other research, and also highlights some of the research ignored by CARRS-Q.

CARRS-Q Research – Part One

Why you should dismiss the CARRS-Q bicycle helmet research

In 2011 the Centre for Accident Research & Road Safety – Queensland (CARRS-Q) produced a publication called Bicycle Helmet Research. The publication relies on some new research carried out by CARRS-Q, but mostly reviews and re-analysed selected existing research in order to support mandatory helmet laws in Queensland. It has not been published in any peer reviewed journal and has only been published by CARRS-Q itself. Despite this lack of academic rigour, many politicians and government policy makers opposed to repealing or amending mandatory bicycle helmet laws now rely on this publication to dismiss any criticism of this law. If you have recently written a letter to the Queensland Government (or even other Australian state governments) on this subject you may be wondering about this new CARRS-Q reference.

In the publication CARRS-Q make the following claims:

  • that mandatory helmet laws reduce the likelihood of head injury by 69%, and serious brain injury by 74%,
  • that helmet laws do not reduce the number of people choosing to cycle in Queensland
  • that the economic benefits of encouraging cycling do not outweigh the medical costs involved with cyclists’ head injuries.

In order to reach these headline grabbing conclusions, CARRS-Q disregards some of the actual findings of the original research it re-analyses, ignores a significant amount of research that discredits helmet laws, and uses a significant amount of unreliable data.

If this sounds a bit too conspiratorial, then consider the backdrop to this report being commissioned (we have touched on this before, here and here):

  • CARRS-Q is an initiative of the Motor Accident Insurance Commission (MAIC). The MAIC is the regulatory authority that oversees Queensland’s motor vehicle insurance scheme, and has the responsibility to pay compensation to victims of motor vehicle crashes.  The MAIC’s key objectives are motor vehicle focused.
  • The CARRS-Q publication was commissioned by the Queensland State Government’s Transport and Main Roads Department in response to media reports about the failure of public bicycle hire schemes, introduced by an opposing political party’s local government and failing due to oppressive State government helmet laws.
  • Due to diligent use of Right to Information legislation (RTI), cycling advocates obtained documents proving that the report was reviewed no fewer than three times by the State Government’s Transport and Main Roads Department before its public release, with some significant changes made to ‘strengthen’ the supposedly academic findings.
  • Documents obtained through RTI also showed that, in response to the cycling advocates’ RTI application, the public release of the report was carefully managed through the media to promote the mandatory helmet laws.
  • Annastacia Palaszczuk, the Minister responsible for Transport in Queensland at the time the report was commissioned and subsequently reviewed, has openly declared that she will never repeal helmet laws.

CARRS-Q’s claims are far in excess of the findings of most of the actual, peer reviewed research CARRS-Q relies upon.

CARRS-Q argues mandatory helmet laws are necessary by:

  • Selectively choosing only some of the available research and ignoring research that does not support helmet laws.
  • Putting its own pro-helmet spin on the findings of the research it reviews.
  • Dismissing claims that the health benefits of cycling outweigh the health costs associated with cyclists’ head injuries.
  • Claiming that helmet laws don’t discourage cycling.
  • Applying some very dubious assumptions to incomplete hospital and crash data, to claim the data shows a marked decrease in head injury rates since the introduction of helmet laws.
  • Dismissing the findings of Voukelatos and Rissel (2010) on the grounds that it makes too many assumptions, and uses incomplete data.

This post is the introduction to a series of posts looking beyond the findings of CARRS-Q, and looking critically at the source data and original research CARRS-Q used.

Part Two will look at helmet efficacy research conducted by the Cochrane Collaboration, a Government-funded UK charity that conducts a broad range of medical research.  This helmet research is relied upon very heavily by the CARRS-Q publication, however the Cochrane research included data relating solely to children cyclists, and excluded injuries serious enough to cause death thereby overemphasizing the statistical relevance of minor injuries such as scratches, bumps and bruises.

Part Three post will look at a significant amount of other research used by CARRS-Q, to support mandatory helmet laws.

Part Four will examine the Monash University research also used by CARRS-Q.

Part Five will deal with CARRS-Q’s claim that there are no significant economic benefits to repealing helmet laws.

Part Six will consider CARRS-Q’s claims that helmets laws do not discourage cycling.

The Final Post in this series will review CARRS-Q’s use of existing police and hospital data and available bicycle fatality data.

The analyses of the source data and original research discussed in this series of posts will show that, contrary to the claims of CARRS-Q:

  • Young children are much more likely to suffer ‘head injury’* that can be prevented by helmet wearing, and are much more likely to be discouraged from cycling because of mandatory helmet laws.  This means any research that compares head injury rates to other types of injuries (for example upper limb injuries) will be misrepresent the efficacy of helmets if it does not differentiate between age groups.
  • Mandatory helmet laws reduce the amount of cycling in the community (see here and here).
  • The net social economic advantages from cycling far outweighs the possible economic costs of reduced helmet use. (We have covered this briefly before).
  • There is no clear, unequivocal evidence that wearing a bicycle helmet provides any significant protection from serious or life-threatening injuries including neck injuries.  It is only clear that helmets provide some increased protection from minor injury, such as non-life threatening cuts, bumps and bruises.
* The use of the term ‘head injury’ is misleading to the public without further qualification, despite being technically correct. Most people assume this is means ‘serious brain injury’. The reality is that it includes everything from minor cuts & bumps and dental trauma to brain injury at the extreme…

It’s for your own safety!

This poster was created by an Australian supporter who clearly ‘gets it’ and while we don’t normally post such things, we really liked this one. Thanks for the support!

Helmet Freedom on TV

Our new television advertisements will begin airing in Brisbane, Australia, over the next few weeks, starting on Channel Go this evening from 19:30.

The purpose of the ads is to highlight just how isolated Australia and New Zealand are with respect to their all-age mandatory bicycle helmet laws – being the only two countries in the world to have such laws and enforce them.

Many good ideas have spread rapidly from Australia (for example seatbelts, black box flight recorders and so on), yet the countries with the highest cycling rates in the world know that forcing all cyclists to wear a bicycle helmets at all times while riding *any* bicycle is counterproductive. It is about the cheapest and laziest thing that a Government could do in the name of ‘cycling safety’, particularly considering it hasn’t improved cyclist safety at all!

The Northern Territory realised that this law was flawed and, soon after their introduction, added exemptions for adults on footpath & cyclepaths. They enjoy a higher rate of cycling – particularly for transport, not sport – and yet have a safety record which is no different to the rest of the country. There are more exemptions in the law for seatbelt use (or non-use) than for bicycle helmets (paying pedicab passengers being the only common exemption across states). We don’t even have an exemption for our crippled public bike hire schemes in Melbourne & Brisbane, despite calls from a prominent cycling journalist for a rethink. Suggestions to the previous Queensland Government of even a trial exemption for bike share were ignored.

It’s time for change.
Take action today and start enjoying the ride! Read more