How to (partially) evaluate automated decision systems

Depicting the social impact of automated decision systems requires multiple interdisciplinary entry-points. In this paper we focus on the actual data and algorithms that produce specific outputs for the purposes of decision-making. The aim of this report is to outline the range of prominent methods that are used for auditing algorithms in data-driven systems and to also consider some of their limitations.