Development, Demonstration and Implementation of a Virtual Reality Simulation Capability for Coal Mining Operations

PM Stothard¹, JM Galvin², JCW Fowler^3
1Envirodaq Pty Ltd, Australia (formerly Senior Research Fellow, The University of New South Wales)

²Professor of Mining Engineering, The University of New South Wales, Sydney, Australia

³Senior Research Fellow, The University of New South Wales, Sydney, Australia

This paper describes the development, demonstration and implementation of a Virtual Reality simulation capability for the Australian coal mining industry. The aim is to improve Occupational Health and Safety management and performance through the provision of more effective education, training and assessment. Immersive interactive simulations are being developed by the School of Mining Engineering at The University of New South Wales and are being transferred to the industry through construction of a Virtual Reality Theatre at the Newcastle Mines Rescue Station in New South Wales, Australia.

Virtual Reality simulations enable trainees to experience a range of real situations that they may otherwise not encounter prior to a critical event. They also condition the trainee’s response in a safe and forgiving environment. Following a Feasibility Study, funded by the New South Wales Joint Coal Board Health and Safety Trust, into the suitability of Virtual Reality as a tool for training mine personnel, three issues have been selected for simulation. These are sprains and strain injuries, coal rib hazards and self escape. All have high Occupational Health and Safety implications that are very dependent on the actions of individuals themselves.

Introduction

Virtual Reality (VR) simulation refers to the generation of an immersive, interactive, computer generated, three-dimensional (3D) environment. As a result of its interactive nature, the VR program senses the user’s response to a situation or event. This interaction is reciprocated by the program and feedback to one or more of the user’s senses is invoked. This produces the illusion of presence within the simulated environment. Many computer technologies are available that support VR. They range from inexpensive desktop simulations to ‘million dollar’, 360 degree, immersive CAVE (Cave Automatic Virtual Environment) systems. All have their advantages and disadvantages.

This paper describes the development, deployment and implementation of a VR simulation capability by the School of Mining Engineering at The University of New South Wales to address the specific needs of the Australian coal mining industry. The aim is to improve Occupational Health and Safety management and performance through the provision of more effective education, training and assessment. The simulation capability is a hybrid system designed to provide simulation technology to operators, large and small. The decreasing cost of sophisticated computer hardware and software has made interactive VR simulation accessible to coal mining operations of all scales.

The identification of affordable VR technology that would be readily accessible to all coal mine operators in New South Wales (NSW), Australia, was one of the main objectives of a Feasibility Study commissioned by the Joint Coal Board (JCB) Health and Safety Trust in 2001 and undertaken by the School of Mining Engineering at The University of New South Wales (UNSW). The findings of the Feasibility Study led to further development funding being provided by the JCB Health and Safety Trust, Australian Coal Research Limited (under the Australian Coal Association Research Program) and The University of New South Wales.

Equipment employed in the coal industry has become much more sophisticated over recent years, with many machines using state of the art technology. Consequently, its operation has become much more complex. The way in which work is organised and performed has also changed significantly. As a result, managers, operators and maintenance personnel are required to assimilate many complex operating and safety procedures in order to safely perform their duties. In many instances it is very difficult to absorb such information as knowledge or, perhaps more importantly, as experience due to its complexity and shear volume. The extent and complexity of the legislative and regulatory environment also presents a dilemma for mine safety training. The JCB (now part of Coal Services Pty Limited) and the UNSW identified that what was required was a truly effective method of training, assessment and re-assessment that provides a trainee with knowledge, experience and understanding of the environment in which they are to work.

The main objective of the UNSW research is to provide a facility that will increase employees’ understanding of the environments and procedures that are associated with coal mining. An improved understanding and level of training and assessment and the facility to provide continual improvements in safety procedures is expected to lead to reduced accidents, a safer working environment and reduced operating costs. Simulation technology has the capacity to meet this objective and provide existing trainers and training techniques with a powerful tool that may ultimately save lives.

Applying VR to coal mining operations

The JCB and the UNSW School of Mining Engineering considered that a solution to changed risks and training and assessment procedures may lie in the timely and effective training, retraining and continuous improvement of the skills of operators and maintenance personnel through the utilisation of VR simulation technology. Consequently, a VR training simulator has been designed. It is tailored to the needs of the NSW coal mining industry and incorporates the capacity to integrate knowledge and skills to achieve safe human responses and enhance existing training methods.

Three high-risk items of plant, a Continuous Miner, a Dump Truck and a Roof Bolter, were selected for the initial Feasibility Study. These items were chosen because the manner of their operation and maintenance has high implications for safety and production and requires the multi-skilling of personnel and the execution of complex tasks.

An extensive and complex legislative and regulatory environment governs the operation and maintenance of such mining equipment. However, despite the existence of complex rules and regulations, there is a continuing recurrence of similar types of accident, as recorded in NSW coal mining industry statistics (Anon 2000, 2001 & 2003), suggesting that there may be deficiencies either inherent within the operation and maintenance procedures themselves or in their method of delivery.

Alternatively, some other extraneous influences may be present. For example, procedures maybe ineffective because they do not encapsulate learning experiences gained from accidents and incidents that have occurred at other sites in Australia and overseas. Sometimes they do not encapsulate information that is available or in a form which makes it easy to collate, distribute, use, or understand. It may be also be that the training and the safety procedures themselves are satisfactory but that they are ineffective because of the risk-taking attitudes of some employees.

Any considered response to improved training via VR simulation should encompass these issues. A VR simulation should replicate not only the work place environment into which a trainee or team will be placed. It should also be capable of presenting the trainee or team with a problem based learning exercise (McAlpine & Stothard 2003) where information relating to the task can be accessed from within the simulation and the trainee’s ability to identify and remedy risks can be quantified. In order to enhance the cognitive learning process, the simulation should also demonstrate to the trainee the consequences of poor decision-making or risk-taking behaviour.

It was envisaged that the development, deployment and implementation of training simulators would be delivered through a four-stage process. The first two stages have been completed. They were run in parallel and constituted the formal Feasibility Study (Stothard et al 2001) that identified appropriate technologies and systems. Applying these technologies in a format acceptable to the industry presents not only a technical challenge but also a challenge with respect to the industry’s perception of the technology. The technology should be seen not as a threat but as an opportunity to improve the safety of co-workers that should be embraced.

The third stage is the construction of the simulator hardware and software based around the findings of stages one and two. Stage three also includes the development of simulated mine environments based on mine plans and the development of systems to control the large amounts of data generated during model development.

The fourth stage is the transfer of the simulation technology to industry. Once the prototype simulation built at UNSW has been proven, a copy of the simulator will be installed at the Newcastle Mines Rescue Station and tested under mining industry conditions.

Technology options

VR is an interactive technology that has the capacity to greatly improve safety through improved training of operators in a safe environment. The Feasibility Study included a review of available VR technologies. It was concluded that VR simulators are effective training tools that allow trainees to make decisions and provide them with the opportunity to experience the consequences of those decisions. An appraisal of VR as an industrial training and marketing tool is presented by Squelch (2000). The conclusions drawn are that VR has the potential to provide relevant and improved training, a conclusion that concurs with the finding of Stothard et al (2001). The study found that VR-based training is an affordable technology and that inexpensive VR simulations that run on personal computers, such as that described by Denby et al (1998), are powerful enough to manage complex VR. Their performance is continually increasing and Williams et al (1998) describe a low cost truck simulator suitable for hazard spotting. In addition, the market for VR software is increasing and the price of its development is decreasing. This is a key factor when attempting to make VR simulation technology available at all mine sites.

The Feasibility Study considered the design and application of a number of simulators and technologies. The conclusions drawn were that the technologies and frameworks for a simulator that could be tailored to the NSW coal industry certainly exist. However, they are fragmented and there is no one combination of technologies that currently satisfies the requirements. After considering the options available, it was concluded that the preferred solution is a modular based system that provides the framework for safety and training simulation. The design should be of open architecture using flexible software to allow for site specific and machine specific changes. The software must allow for splicing and pasting of video images and video footage so that the simulator is context sensitive and realistic. The simulations must be represented by textures inputted from digital and/or CAD images of the mining machinery and environment that the trainee will experience in reality. Such a system has been designed and is being developed.

The virtual reality theatre

One of the main objectives of the development process is to demonstrate to the coal mining industry the concept of employing virtual reality simulation to improve OH&S management and performance. A basic underground simulation was constructed for this purpose. It included detailed 3D models of many items of mine equipment. The simulation was based upon an actual mine plan, allowing mine specific data to be incorporated into the simulation, with the models being built from drawings of real mine equipment. Data obtained from mine sites was used for real time interactive information that can be accessed by ‘touching’ the relevant piece of equipment from within the simulation. Where a complex situation or concept was present in the simulation, detailed video clips of the process were incorporated. The use of video clips of real situations enhances the realism of the simulation and allows the trainee to relate to the real equipment through viewing real mine personnel using real equipment.

The next stage in the development process was the development of a prototype simulation model that incorporated the systems and processes identified by the earlier studies. To this end, a Virtual Reality Theatre was built at the UNSW School of Mining Engineering. The theatre was constructed using off the shelf components and was based around high-end personal computers and peripheral equipment. Key components were a large screen for group interaction, a touch screen for trainee interaction, real machine controls and a joystick for manoeuvring around the simulated environment. The theatre, financed by UNSW Research Infrastructure Grants, included the development of the production system required for complex 3D interactive mine models.

A schematic arrangement of the simulator is shown in figure 1 below.

Fig. 1 Schematic arrangement of immersive, interactive simulator with real machine controls

The training and assessment component

Issues raised prior to developing the training and assessment component of the VR simulation included the following.

•   What are the objectives of placing a trainee in the simulation?

•   What is the simulation ‘mission’?

•   How will the trainee perform this mission and meet the objectives?

•   How will the trainee be assessed to ensure the objectives have been met?

•   Is the simulation to be used for training, visualisation, or to promote group discussion of a situation?

•   What are sufficient levels of detail within the simulation?

•   What information will be made available to the trainee?

The issues indicate that the implementation of the simulation must be carefully considered in order to provide the necessary benefits to the trainee. They fall into two general areas. The first is concerned with specifying the experience that a trainee is to gain from interacting with the simulation while the second involves defining how much detail is required in the development of the ‘virtual world’. While it is impractical to replicate every item in the real world, there must be sufficient detail for the simulation to appear authentic.

The development of simulations with the functionality that provides trainees with the appropriate experience required both expert input and the analysis of relevant incident and accident statistics and mine specific accident data. Expert input on each scenario (self escape, sprains and strains and coal rib stability) provided a foundation for the scenarios. The second factor that needed consideration was the necessary level of detail for trainees to relate to objects in the simulations. Many simulations have presented anecdotal evidence for the positive contribution of VR to the training of personnel. However, information on the level of detail required for mining personnel was scarce. Consequently, the required levels of realism were investigated as described in Buchanan (2003).

Technical details

The three main areas for simulation considered in phase three of the project currently under development are sprains and strains, coal rib stability and self escape. In order to produce accurate simulations in these three areas a model of an underground longwall panel was developed based upon an actual NSW coal mine. Essentially, the plan of the panel was digitised and converted into a 3D representation. The aim of digitising a whole longwall panel from face to portal was to provide a generic template within which localised simulations relating to sprains and strains, coal rib stability and self escape could be developed. A single frame of a simulation showing the longwall face end is given in figure 2.

Fig. 2 ‘Frame grab’ from a simulation showing the longwall face end

Once digitised and rendered in 3D, the simulated mine can form the basis of any longwall mine through the substitution of local longwall conditions. Different rock types, stress regimes, equipment and environmental conditions (gas, smoke, visibility etc) can be invoked once the template is developed. This has been achieved by treating each pillar as a separate object. This approach provides the necessary flexibility and level of interaction required of the VR simulations. Pillars containing mine specific information or parameter modification can be easily substituted for an existing pillar. The mine pillars can be modified to contain the necessary levels of realism for a specific simulation. Level of detail is something that has to be considered on a scenario-by-scenario basis and may be identified through clear identification of training goals prior to placing trainees in the simulation.

Conclusions

The project is progressing with the Newcastle Mines Rescue Station VR theatre due for commissioning in late 2004. It will incorporate the three simulations related to self escape from underground coal mines, rib stability in underground coal mines and slips and trips in both surface and underground mines. Indications to date are that virtual reality simulations may be effectively employed in providing more effective education, training and assessment with the specific aim of improving Occupational Health and Safety management and performance in the Australian coal mining industry.

Acknowledgements

The authors extend their sincere thanks to the following organizations for their support of the research: Australian Coal Research Limited (ACARP), the Joint Coal Board Health & Safety Trust and The University of New South Wales. The contributions of the many coal mining companies, equipment suppliers and other coal mining industry stakeholders who provided invaluable assistance and information are also gratefully acknowledged.

References

Anon 2000, Lost‑Time Injuries and Fatalities – New South Wales Coal Mines – 1999‑00, publ. Sydney, Australia: Joint Coal Board, ISSN 1324‑9959.

Anon 2001, Lost‑Time Injuries and Fatalities – New South Wales Coal Mines – 2000‑01, publ. Sydney, Australia: Joint Coal Board, ISSN 1324‑9959.

Anon 2003, Lost‑Time Injuries and Fatalities – New South Wales Coal Mines – 2001‑02, publ. Sydney, Australia: Coal Services, ISSN 1324‑9959.

Buchanan L 2003, Levels of Realism in Virtual Reality Mine Training Programs, Unpublished BE Thesis, UNSW School of Mining Engineering.

Denby B, Schofield D, McClarnon DJ, Williams MW & Walsha T 1998, ‘Hazard awareness training for mining situations using virtual reality’, in APCOM '98 – Application of Computers and Mathematics in the Mineral Industries (Proc. 27^th APCOM International Symposium, London, UK, 19‑23 Apr. 1998), publ. London, UK: Institution of Mining and Metallurgy.

McAlpine I & Stothard PM 2003, ‘Using multimedia technologies to support PBL for a course in 3D modeling for mining engineers’, Proc. ED‑MEDIA 2003 World Conference on Educational Multimedia, Hypermedia and Telecommunications, Honolulu, Hawaii, 24‑28 June 2003, pp 2449‑2455, publ. Norfolk, VA: Association for the Advancement of Computers in Education.

Squelch AP 2000, ‘VR as an industrial training and marketing tool’, CSSA Western Cape International Virtual Reality Workshop: Is it ‘Now or Never’?, Cape Town, South Africa, 23 May 2000.

Stothard PM, Otto D, Laurence DC, Galvin JM & Zenari L 2001, E‑minesafe Safety and Training Simulator – the Integration of Knowledge and Skills to Achieve Safe Human Responses, UMRC Research Report RR10/01, UNSW Mining Research Centre, Sydney, NSW, ISBN 0 7334 1844 9.

Williams M, Hollands R, Schofield D & Denby B 1998, ‘Virtual haulage trucks: improving the safety of surface mines’, in APCOM 98 (Technical Proceedings Third Regional APCOM Conference, Kalgoorlie, WA, 7‑9 Dec. 1998, pp 165‑170), publ. Carlton South, Vic: The Australasian Institute of Mining and Metallurgy, ISBN 1 875776 64 8.