In strategy research and practice, the growing attention towards openness has sparked numerous articles investigating transparency and inclusion within the strategy process. While previous research has primarily conceptualized and qualitatively examined individuals’ practices and the process of how openness unfolds, empirically verified knowledge is still limited. This paper aims to capture organizations’ level of openness in the strategy process, enabling investigations of the relationships between antecedents but also consequences. By using innovative secondary textual data sources and supervised machine learning (ML), more precisely text classification, we create a measure of openness in the strategy process, identifying organizational levels of transparency, opacity, inclusion, and exclusion to capture the aggregated development of openness in the strategy process over time among a large sample. Our findings within a S&P1500 employee reviews sample indicate that there is mostly a perceived decrease in closed practices rather than an increase in open practices. Furthermore, transparency increases are more consistent across industries than changes in inclusion. This paper contributes to literature on Open Strategy by (1) enabling quantitative validation of conceptualizations, and (2) delivering first descriptive results. (3) It provides an outline for future research in strategy-as-practice on using new methodological opportunities and (4) demonstrates the application of ML in Open Strategy to broaden the analytical toolkit available for positivistic investigations.