A programmer wants to take on Google Search, starting with two PCs that sit in his laundry room. Ryan Pearce, who worked in both enterprise software and game development roles, was curious about what it takes to build a search engine. According to Fast Company, Pearce built two search engines: Searcha Page and its incognito variant, Seek Ninja. These two engines both have a database with over 2 billion entries. Although it’s less than half a percent of Google’s 400-billion-strong search index, it’s still a considerable number, and he expects to double it in the next six months or so.
This self-hosted search engine sits in Pearce’s home — but instead of going with a low-powered Raspberry Pi or a mini-PC that most people use when self-hosting, he opted for a used 32-core AMD EPYC 7532, which was one of the best workstation CPUs when it launched in 2020. After all, if you want to create a massive search engine, you’d need something punchier than an entry-level CPU. Nevertheless, since that processor is already five years old at the time of writing, you can now easily get a used copy for less than $200. “I could have gotten another chip for the same price, which would have had twice as many threads, but it would have produced too much heat,” the programmer told Fast Company.
Pearce said that he’s building his search engine piece by piece, and that he’s already written around 150,000 lines of code. However, he says that he’s done so much more than that, and that he has iterated over 500,000 lines of code. Most of these changes were made to reduce the code’s reliance on large language models and make it work without relying on AI. This technique enabled him to create a complex system and then lock in modules that work by removing the variable of AI.
