Interactive Machine Learning for More Expressive Game Interactions

Videogame systems incorporate varied sensors to increase the range of player interactions and improve player experience. However, implementing robust recognisers for player actions with sensors presents significant challenges to developers. Further, sensor-based controls offer little player customisation compared to traditional input interfaces (gamepads, keyboards and joysticks). Past research on motion-driven music systems has successfully used interactive machine learning (IML) techniques to facilitate the development and customisation of sensor-based interfaces, both by developers and end users. However, existing standalone software tools for IML are not ideal for use in game development and distribution. In order to support more effective and flexible use of sensors by game developers and players, we developed an integrated IML solution for Unity3D in the form of a visual node system supporting classification, regression and time series analysis of sensor data.