Organizational search is crucial for firm innovation. Recent research suggests Artificial Intelligence (AI) may revolutionize the organizational search process, enabling access to distant knowledge beyond conventional reach. However, search not only involves looking for solutions but also evaluating them – an aspect less understood in the context of AI-generated results. We propose a model that highlights a temporal shift in AI-based search: while the efficiency of identifying solutions may increase, evaluation times may lengthen, and outcome variance may rise. We attribute this to diminished routine recombination and knowledge accumulation within firms using AI. Our model identifies two key contingencies: startups, with their limited established routines and knowledge stocks, are particularly vulnerable to these challenges; conversely, firms with robust search routines are better positioned to mitigate evaluation difficulties in AI-based search. This study enhances our understanding of the tradeoffs associated with AI in organizational search and innovation, contributing to both the search and AI literatures. By examining the often-overlooked evaluation component of search, we provide insights into the complex dynamics of AI integration in organizational innovation processes.