Survey research is facing a multitude of challenges to its validity across the social sciences. Online surveys with non-probability, convenience samples are simultaneously seen as part of the problem and a promising solution. One such popular convenient data source is that of employer reviews, especially from online job boards such as Glassdoor and Indeed. Researchers in management (De Neve et al., 2023; Gornall et al., 2024; Ward, 2023), economics (Almeida et al., 2024; Sockin & Sojourner, 2023) and sociology (2023) have produced novel studies with impressive methodologies using this data. Methodological literature argues that researchers should not think of data quality of online surveys in terms of ‘good’ and ‘bad’ but in degrees, with a series of recommendations scattered across disciplines for assessing and managing data limitations. We present an evaluation of one source of employer review data: Indeed’s Work Wellbeing Score survey (IWWS). IWWS is an ongoing international survey of subjective work wellbeing, with over 20,000,000 responses and growing. In this study we evaluate the UK subsample collected by October 2023 (N = 1,463,503). While a prima facie valuable source of data, the data generation process raises concerns of selection bias and inattentive responses. We evaluate the extent of bias, variation in bias, response rates, internal consistency and employer cluster-level reliability. We then turn to considering what types of research questions a researcher may want to answer with the data, especially unit comparisons at different survey units and inter item relationships. Overall, we suggest that at the individual, employee level, the survey suffers from selection and binary bias in responses, but that at the employer-level IWWS offers a valuable resource to supplement existing random probability surveys of work and wellbeing. In our conclusions we offer practical methodological recommendations for others using Big, online convenience samples like employer reviews. Finally, we provide commentary on the strengths and limitations of the IWWS for ongoing and future research, as well as the value for businesses, jobseekers and policy-makers.
