A critical examination of a program or system is an invaluable tool for an agency seeking to strengthen the quality of the service being delivered and improve outcomes.
Our trained evaluators have designed and implemented numerous quantitative and qualitative evaluations of flood education programs, emergency warning dissemination, emergency plan implementation and community response, providing the analysis and results for agencies to improve effectiveness and decision making.
Some examples of our projects include:
Review of Total Flood Warnings for the Victoria Floods Review.
Between September 2010 and February 2011 there was extensive flooding across almost all of Victoria with 400 towns being flooded, many two or three times. Record flooding occurred on many of the river systems, thousands of residents were advised to evacuate but many were caught unawares by the rapidly rising floodwaters and others did not appreciate just how high the water could come. The Victoria Floods Review was established to investigate all aspects of the floods and make recommendations to the State Government on how the communities and emergency service agencies could be better prepared and changes needed to be made in flood warning and recovery to reduce the extent of damages in the future. Molino Stewart was engaged to prepare a report on all aspects of flood warning from data collection, through forecasting and warning dissemination to community response to warnings. A sample of events across several catchments from September 2010 to February 2011 were investigated.
Molino Stewart developed a methodology for evaluating the accuracy and timeliness of Bureau of Meteorology flood forecasts and applied it to several key gauges across the sample events investigated. We identified how data collection networks and flood forecasting methodologies contributed to warning outcomes. We interviewed key stakeholders involved in the warning process, including forecasters, incident controllers, intelligence officers and local commanders to understand their experiences. We reviewed community feedback and media reporting of events to gauge how well messages were received, understood and acted upon. We reported our findings to the Victoria Floods Review.
Post Flood Review Tool for Bureau of Meterology
Molino Stewart was engaged by the Bureau of Meteorology to conduct research into and make recommendations regarding the development of a post-flood performance review process for the Bureau’s forecasting and warning services and products.
Molino Stewart interviewed representatives from direct clients in all of the Bureau’s seven regions – the States and Territories of Australia – to identify what might be the best post-flood performance review approaches. It also researched approaches used by other hazard-related agencies in Australia and overseas.
Review to enhance Total Flood Warning Systems Development in Victoria. (Victoria State Emergency Service).
Molino Stewart reviewed the role of the Victoria SES (VICSES) in flood education and flood warning with recommendations to the VICSES Board on appropriate actions and resources to improve both through VICSES initiatives. In relation to education, Molino Stewart carried out a literature search of community flood education programs in Victoria, elsewhere in Australia and overseas. We analysed these programs and VIC SES’s FloodSmart program in relation to best practices ascertained by research. Based on findings, Molino Stewart recommended approaches to VICSES to position it as the ‘champion’ of community flood education in Victoria. In relation to flood warning, Molino Stewart reviewed roles and responsibilities in the various components of the total flood warning system through interviewing the Bureau of Meteorology and Councils, Catchment Management Authorities and VICSES regional staff across the whole of Victoria. We prepared a report on where there were gaps in the system for which the VICSES would be best placed to take responsibility. Molino Stewart prepared a business case for a VICSES submission to government which resulted in increased funding to the VICSES and the creation of additional roles within the organisation for emergency planning, flood warning and community education.
Operations Component of the Flood Review
Heavy rain fell across north-east Victoria during late February and early March 2012 causing record flooding in some parts of the Goulburn-Broken river system. The most flood-affected watercourse was Broken Creek and its associated tributaries such as Boosey Creek, Muckatah Creek and Major Creek. The extent of flooding in the Broken Creek catchment was 70,000 hectares. Townships in this catchment include Numurkah, Nathalia, Tungamah and Katamatite.
Emergency agencies, including the Victorian State Emergency Service (VICSES), the Country Fire Authority (CFA), the Department of Sustainability and Environment (DSE) and Victoria Police, worked with local councils in the area to conduct a series of incident management actions to warn communities and help ensure their safety during the flood.
As part of its role to provide assurance on the effectiveness of Victoria’s emergency management arrangements, the Office of the Emergency Services Commissioner (OESC) coordinated an independent review of the 2012 March North East Victoria Flood.
Molino Stewart was engaged to help conduct the review. We designed a review plan in consultation with the agency and examined aspects of prediction, interpretation, message construction and communication and response related to warnings issued. Data was collected and analysed from a sample of catchments and we interviewed a broad range of stakeholders including from VICSES, BoM, DSE, CFA, CMAs, local councils and water authorities. The findings were compared against the recommendations of the Review of the 2010-11 Flood Warning and Response which led to major reforms of how flood warnings are managed across the state.
Review of the May 2009 East Coast Low Warning and Public Information.
Molino Stewart evaluated the warning and public information performance related to the May 2009 flood event on the NSW North Coast, across three main towns; Grafton, Kempsey and Lismore. A community survey was designed and implemented and Molino Stewart facilitated multiple focus groups across these areas. Molino Stewart also project managed a supporting technical review of the warning system performance and wrote the final report for the NSW SES.
Maitland and Newcastle Flood Warning and Evacuation Evaluation
Between Friday June 8th 2007 and Monday June 11th 2007, the New South Wales Hunter and Central Coast regions experienced severe weather conditions resulting in substantial flooding in some areas, the “Pasha Bulker” storm. Over 4,000 people were evacuated in the Maitland area and other evacuations were made in the Newcastle and Lake Macquarie areas. Molino Stewart conducted community surveys to evaluate warning and evacuation performance during the severe weather and gauge community attitudes and behaviours in response to these warnings and evacuations. A total of 168 surveys were completed in Maitland and 318 in Newcastle. Molino Stewart analysed the survey responses and provided recommendations to the SES in relation to future flood education and warning systems and messaging.
Review of Nathalia Flood Levees
There are a number of temporary flood barrier systems in use, under test or generally available throughout Australia. Every system meets a certain demand for flood protection, and may be suitable across a range of conditions, subject to the particular circumstances of a location.
Molino Stewart conducted a review of the Temporary Flood Levees utilised at Nathalia during the flooding from the Broken River in February/March 2012 to provide a detailed response of its performance and to consider what options there are for the provision of temporary levees for flood management.
Cost Benefit Analysis of Natural Hazards Education
Molino Stewart prepared an economic analysis to support an SES business case to increase its role in community education in relation to floods, tsunami, storms and earthquakes. This included analysis of existing data on damage costs associated with each of the natural hazards and quantification of reductions to these costs that might ensue from the implementation of better preparedness, education and warning.
Review of information prepared for council FRM officers and community engagement resources
Molino Stewart undertook a review of answers to written questions asked by Sydney local council officers at the Flood Risk Management (FRM) workshop facilitated previously by Molino Stewart for the NSW Office of Environment and Heritage. Molino Stewart also reviewed relevant FRM community engagement resources of potential use to council officers and presented those of worth in an annotated bibliography.
