4 ms·
Now you can search on Google for free: Solution with API
- karust 3y agoLately I've been struggling to get search engine results via API. If you ever had the same situation, this project might help you
- retox 3y agoPlease tell me you aren't making the same mistake of building any products around this. I can't stomach another startup "we're shutting down" Tell HN post. >API access for search engines results if available isn't free.
- karust 3y agoAhah, no, that's mostly for small researchers like me. Creating a startup around it would be difficult, with questionable financial capabilities.
- RamblingCTO 3y agoIt's a wrapper around a headless chrome apparently, fyi. This will see some growth is my guess. I'm also working on something like that with 2markdown.com. A pity that you need a full blown browser to retrieve web content. Maybe these weird semantic web folks have been right all along ;)
- krackers 3y agoWon't you get served captchas if you're querying frequently?
- tourmalinetaco 3y agoI doubt it. Its user agent should look like a normal Chromium browser, and under a normal residential IP it should look like normal usage. Under a proxy service or VPN, however, that may be different.
- taskforcegemini 3y agothe amount of usage per IP-address is limited though
- ttt3ts 3y agoYes.
- RamblingCTO 3y agoActually depends. I prefer rod to drive headless chrome and you can use rod-stealth for that. You have to mimic traffic to replicate humans. And if you don't spam a website it won't trigger any defence mechanisms. Check https://bot.sannysoft.com/ https://bot.sannysoft.com/ ;)
- jiggywiggy 3y agoSemantic web wont do you good. Google doesn't like scraping and won't make it easy for you to do so.
- CodeCompost 3y agoNow do Reddit next :-P
- malermeister 3y agoYou can just extract the API key from the official app and patch your third party app of choice to use it, at least on Android.
- flangola7 3y agoWhat if the app is closed source
- malermeister 3y agoDoesn't matter, you can patch binaries as well. Here's patches for Boost and Sync for example: https://revanced.app/patches?s=reddit https://revanced.app/patches?s=reddit
- dievskiy 3y agoAs far as I can see reverse image search is not supported?
- karust 3y agoNot yet, I have to do some research with this one
- maltelandwehr 3y agoAre they scraping the Google web interface or using the API from Google?
- awaisraad 3y agoscraping - just looked at the code.
- pedalpete 3y agoWhy is this valuable, who is it valuable to? If you're just looking for a way to scrape search results, what end use are you trying to serve with that? I'm not saying it isn't valuable, but as maybe the tagline needs maybe needs work. As far as I am aware, I am already searching google for "free". It is my understanding that search engines heavily use cookies in order to tailor results to the user.
- az226 3y agoUse an LLM to go through the first 50 links, explore the content, and give you a summary of your query along with the link to the best result.
- karust 3y agoThe search is indeed free. But if you try to automate the process, it becomes "paid" :) For instance, I use it as part of my other solution. To inspect the content of URLs I find during some queries.
- lgats 3y agoquite similar to SearX, but I don't think SearX uses headless chrome. https://searx.github.io/searx/ https://searx.github.io/searx/
- ttt3ts 3y agoYea, they don't use anything that heavy. I use https://github.com/searxng https://github.com/searxng as my primary search engine (fork of searx)
- bosch_mind 3y agoI also use searxng as my primary search engine protected by Authelia on a remote server. This way, the IP logged is my throwaway homelab using certs from Let’s encrypt. It’s pretty nice overall and takes not too much RAM (350mb?).
- deleted 3y ago[deleted]