Search is simultaneously one of the most underappreciated and overappreciated technologies of our time. It’s an assumed feature whenever a platform has more than five items to look through but an absolute treasure when you’re trying to find that awesome video.
On its own, search is also rather vague. What are you supposed to be searching for? Insights? Or perhaps for beauty? Even when it comes to search in software, it can be a chore to determine whether a tool is genuinely novel or just an abstraction on top of existing ones.
To keep us from any semantic loops, it would help to realize that “search” really refers to one of two scenarios - reduction and magnification. Picture yourself figuring out what to wear at a party or on a video call. One way to go about it would be by laying all your clothing options on your bed, effectively reducing the problem from being scattered all over your house to however big your mattress is. The other way to go about it would be taking each possible outfit and putting it on in front of the mirror so you magnify what each option looks like rather than just speculating.
When faced with a surplus of information (an uncertainty about where to look), “search” is about reducing the problem into something you can actually grok. An example of this is the insanely large number of pages on the internet and reducing this absurdity into a simple small text input via web search (ie Google).
When faced with a lack of information (an uncertainty about what you’re looking at), “search” is about magnifying the problem into something you can actually learn from. An example of this is the lack of digitization of real world startups/investments and magnifying information about businesses via Crunchbase.
However this modeling works regardless if we’re looking at search as a product or as a feature. Search-as-reduction can be found in Tik Tok’s explore view even though that’s not their core product and search-as-magnification can be found in YouTube video analytics even though that’s not why people go on YouTube.
While some people could use those apps just for those specific functionalities, they wouldn’t be able to prosper as their own products. In order to help an instance of search evolve into something powerful that would, the strength of the “lens” must be increased to the point that it can’t be replaced by better practices.
If we look at what Google provides for the web, that nightmare isn’t something that can be fixed with a different protocol, the problem is simply too large. When we look at domain-oriented search like Sourcegraph, the lack of queryability over a codebase isn’t something that can be resolved by writing more documentation.
However, when it comes to Tik Tok’s explore view, you could save URLs of funny videos as you come across them instead of looking them up in retrospect and you could let the feed curate your content instead of you browsing by categories. Then, with YouTube analytics, you could make actually good content instead of trying to A/B test your way to a larger audience.
Another note to be made about the sort of “search” these businesses provide is the Thiel-ian remark that they need have a monopolistic strength of search. If they are a search-as-reduction, they need to be able to compile an obscene amount of data that nobody else can (hence why Google is on top in their game). If they are a search-as-magnification, they need to be the only one who can provide the knowledge/information they do (hence why you go to the web archive to see how online dating looked in 2005).
So where does that leave us? We can look at the world around us and confidently see that there is a massive and growing amount of data of various formats so there will continue to be potential avenues, and the stream of 9+ figure exits won’t be going anywhere. However, we know a company needs to either capture a unique amount of data or provide a unique insight into some respective data. This means that something that’s just “privacy respecting” won’t mean much unless it gets something dramatically unique from your data and that a new digital solve-all won’t have much value unless it works with a whole new degree of data.
With all this, finding a well calibrated search model is simply a matter of searching for search