In Canada alone, railway systems cover 60,000 km of main track-line, and transport 75 million people as well as $250 billion worth of goods each year. With such a high number of commuters relying on rail transit, it is evident that safety is a top priority in organizing public transportation systems. Train accidents cause damage to infrastructure as well as service disruption and cause casualties and damage the environments. More than 60% of rail accidents were reported due to derailments. The most significant causes of derailment accidents are related to defects of tracks/wayside elements and impacts of environmental factors. Careful and frequent inspection of railway infrastructure is, therefore, the most important way to identify such problems. Also, integrating the data acquired from the railway infrastructure to the risk/mitigation-measure assessment systems will be a great help to railway engineers for predicting/avoiding railway disasters and being prepared for such incidents/accidents. In this study, we present a cutting-edge technology for developing a safety-improvement framework using mobile mapping systems mounted with laser scanner and cameras. The developed system aims to automate the reconstruction of rail infrastructure and identification of derailment hotspots from collected images and laser point clouds. This vision system is implemented with a multi-scale binary template matching and the optimization of detection results is conducted through context-based energy minimization techniques. We also present our on-going research works for assessing derailment risks using existing derailment and wayside information database and simulating derailment disasters for enhancing the performance of existing railway network screening system.