Julian Schulte

A literature review from a personnel economics and psychology perspective

Paul Hemsen, Paderborn University
Julian Schulte, Bielefeld University
Katharina Schlicher, Bielefeld University

Crowdsourcing describes a type of participative online activity in which an organization proposes a task to a group of individuals via a flexible open call (Estellés-Arolas & González-Ladrón-de-Guevara, 2012).

This study focuses on a literature review of crowdsourcing platforms especially for paid work on so-called “crowdworking platforms”. The main contributions are the overview of empirical studies on the subject from a personnel economics and psychological perspective and the development of a comprehensive Input-Process-Output Model (IPO-Model) from the workers’ perspective. In the remainder of this abstract, we will discuss the focus, theoretical background and design of our systematic review.

Research on crowd work is heterogeneous in nature and driven by multiple disciplines. Not surprisingly, researchers have already conducted literature reviews (e.g. Chittilappilly, Chen, & Amer-Yahia, 2016; Ghezzi, Gabelloni, Martini, & Natalicchio, 2017; Kittur et al., 2013; Zhao & Zhu, 2014). However, these focus broadly of the crowdsourcing concept, thereby not differentiating whether it is a digital gainful work or an unpaid voluntary participation. Such reviews usually study what crowdsourcing is, how it is different from similar or related concepts, and how crowdsourcing works (conceptualization focus) (Zhao & Zhu, 2014). They also discuss how crowdsourcing is applied in different situations and for different purposes.

This review contributes to the previous literature by evaluating existing research systematically and describing empirical connections between constructs by grouping similar research into clusters. In contrast to past research which approaches crowdsourcing holistically, we focus specifically on crowd work, i.e. crowd sourcing in which contributors are paid and enter a particular employment relationship with the platform. We are particularly interested in outcomes of this relationship for the individual crowd worker.

Since crowdworking has important similarities to other types of work both typical (e.g. permanent and temporary employment) and atypical (e.g. teleworking, freelancing, self-employment), extant research in personnel economics and personnel psychology can be used to shed light on the factors that might influence crowdworking initiatives.

We reviewed 91 empirical articles, systematically codified these studies and developed an IPO-Model from a personnel economics and psychological perspective. IPO-Models are widely used in sciences for describing processes in system analysis and mechanisms of action in research.

Studies were identified by applying a number of search terms: crowd work*, crowdwork*, crowd sourc*, crowdsource*, platform economy, gig economy or crowd employment. The most important databases for both psychology and economics were searched, namely PsycINFO, EconLit and Business Source Complete. Due to a high number of hits, the search was narrowed down to empirical studies. Additionally, we applied a backward and forward search strategy on the references of key articles. This search resulted in 1173 primary studies overall. We selected relevant primary studies by applying three selection criteria. The studies had to (1) report research on the construct of crowdworking, (2) show an emphasis on personnel economic and psychological research questions, (3) and collect empirical data.

As a result, 91 studies remained and were systematically codified by publication data; information about sample, crowdworking platform, research design, methodology and findings. An iterative bottom-up approach then aggregated these codified variables into clusters based on similarity and content-related proximity. The clusters are divided into the three stages of the IPO-model, namely input, process and output.

The input variables were grouped into seven clusters: monetary incentives; nonmonetary incentives; task design; market-related variables; workers’ qualification/profile; workers’ traits/characteristics; individual working history on the platform. The input variables were modeled in primary studies to explain the variations of specific process- or output variables.

The output variables were grouped into six clusters: job satisfaction, worker commitment towards the platform, participation in crowd work, qualitative performance, quantitative performance and employability of the crowd worker.

Involved process variables which potentially moderate or mediate the relationship between input and output variables were grouped into six clusters: workers’ intrinsic and extrinsic motivation; workers’ affect; workers' perceived competence; invested effort for task completion; workers’ trust towards the platform and workers’ perceived fairness of the processes on crowdworking platforms.

Further analyses of the literature expand the IPO-Model by information about statistically significant and non-significant relations. Hence, our review shows how often a research question has been addressed and which statistical effects evolved in the studies.

Overall, our review provides a roadmap for future research on the topic of crowd work as digital gainful work. We identify and quantify the state of the art in current research of personal economics and psychology on the topic of crowd work. Our review has important implications on how to enhance factors that are critical to worker and platforms alike, such as attraction, motivation and commitment of self-employed workers on crowdworking platforms.

References

• Chittilappilly, A. I., Chen, L., & Amer-Yahia, S. 2016. A Survey of General-Purpose Crowdsourcing Techniques. IEEE Transactions on Knowledge and Data Engineering, 28(9): 2246–2266.
• Estellés-Arolas, E., & González-Ladrón-de-Guevara, F. 2012. Towards an integrated crowdsourcing definition. Journal of Information Science, 38(2): 189–200.
• Ghezzi, A., et al. 2017. Crowdsourcing: A review and suggestions for future research. International Journal of Management Reviews.
• Kittur, A., et al. 2013. The future of crowd work, Proceedings of the 2013 conference on Computer supported cooperative work: 1301–1318.
• Zhao, Y., & Zhu, Q. 2014. Evaluation on crowdsourcing research: Current status and future direction. Information Systems Frontiers, 16(3): 417–434.