Testing the International Standards Organization Verification and Validation protocol for evacuation simulations-An application to the FDS+Evac model

Abstract

Nowadays, many evacuation models exist in the market, and new models are continuously released with new features. How to assess the usability and reliability of model results becomes an issue for both developers and users. Therefore, verification and validation (V&V) protocols were introduced in the assessment process of evacuation models. To conduct V&V of a model, a widely accepted test procedure should be defined and followed by the tester. However, there are no comprehensive and globally accepted V&V test procedures available to conduct V&V of evacuation models. Therefore, the International Standard Organization Verification and Validation Standard (ISO V&V Standard) (ISO, 2019) was proposed as a new benchmark of V&V for building evacuation models. This ISO document (ISO, 2019) includes carefully designed V&V tests based on accepted knowledge of human behaviour in fire and building features. Notably, a most comprehensive validation test procedure is provided with a list of recommended experimental data for the first time, along with methods to conduct the mode results evaluation. This thesis goes through the whole ISO V&V test procedure including a global validation test by applying it to a simulator named FDS+Evac (Korhonen, T., & Hostikka, 2009). The benefits of the application of the ISO test procedure in V&V were identified and discussed in comparison with existing test procedures. Based on the simulated results except test 26, the issues and challenges of FDS+Evac and current evacuation models were analysed and summarised. It should be noted that the ISO document referred in this thesis is undergoing modifications and is routinely updated by ISO; therefore, the thesis work can be used to provide possible suggestions for improvements to refine the ISO V&V Standard protocol.

Popular Abstract

There are many evacuation models and simulators available in the market today. The performance of these models is generally different from each other. The new development of these evacuation models is continuously emerging to catch up on the newest development of buildings and the newest knowledge of human behaviour in fire. It produces an issue of how to assess the reliability and the usability of sub-models embedded in these models and the simulated results? Therefore, verification and validation (V&V) test procedure is introduced to evaluate these models.
Verification can be summarised as ‘to solve equations right’. Validation can also be summarised as ‘to solve right equations. Before the ISO test procedure, there are three test procedures: IMO (International Maritime Organization) test procedure, RiMEA test procedure and NIST test procedure. Since they are used in V&V for evacuation models designed for buildings, there are issues in the application. For instance, IMO test procedure is initially designed for maritime ships but not mainly designed for models for buildings so that tests could be unsuitable for these evacuation models. Validation is not involved in IMO and RiMEA test procedure due to the lack of experimental data. NIST test procedure has a validation section but rough. Therefore, a comprehensive and widely acceptable test procedure is demanded.
The ISO V&V standard was proposed to solve this issue. It has deliberately designed tests procedures for both verification and validation. In the verification section, most tests are on the foundation of the existing test procedure and modified from them. In the validation section, improvement is enormous since it has tests designed for component validation and a list of recommended validation. All verification tests have a list of recommended experimental data, which cannot be found in IMO and RiMEA test procedures. NIST test procedure has tests for validation, but they are not as comprehensively designed as them in the ISO test procedure.
Meanwhile, NIST has few available data for each validation test. Overall, Comparison with existing verification test procedures, it can be found that the ISO document established new comprehensive categories for verification tests. Meanwhile, new designed and modified tests improve the usability of the ISO verification procedure to modern buildings. Comparison with existing validation test procedures, ISO document establishes a new and comprehensive validation test procedure with sufficient experimental data for each designed validation test.
All verification and validation tests designed in the ISO test procedure have to be conducted except test 26, which has no available experimental data. FDS+Evac was used as the simulator in the thesis. Therefore, issues and challenges for evacuation models are summarized based on the simulation. For instance, available sub-models have to be expanded and refined to fit the newest development of buildings and knowledge. The available outputs should be expended to match these often-measured parameters in experiments, such as density. More experiments should be conducted to solve the lack of experimental data for validation. Testers who use these evacuation models should have good knowledge and experience in the field of evacuation simulation. A clear and accessible user guide for models should be provided to testers for a better understanding of the performance and limits of models.

• author: Wu, Yalong
• supervisor: Enrico Ronchi
• organization: Division of Fire Safety Engineering
• course: VBRM05 20191
• year: 2019
• type: H2 - Master's Degree (Two Years)
• subject: Technology and Engineering
• publication/series: LUTVDG/TVBB
• report number: 5593
• other publication id: LUTVDG/TVBB—5593—SE
• language: English
• id: 8981724
• date added to LUP: 2019-06-20 13:04:32
• date last changed: 2019-06-20 13:04:32