Level crossings are amongst the most complex of road safety issues, due to the addition of rail infrastructure, trains and train operations. The differences in the operational characteristics of different warning devices together with varying crossing, traffic or/and train characteristics, cause different driver behaviour at crossings. This paper compares driver behaviour towards two novel warning devices (rumble strips and in-vehicle audio warning) with two conventional warning devices (flashing light and stop sign) at railway level crossings using microsimulation modelling. Two safety performance indicators directly related to collision risks, violation and time-to-collision, were adopted. Results indicated the active systems were more effective at reducing likely collisions compared to passive devices. With the combined application of driving simulation and traffic microsimulation modelling, traffic safety performance indicators for a level crossing can be estimated. From these, relative safety comparisons for the different traffic devices are derived, or even for absolute safety evaluation with proper calibration from field investigations.