php - How to fetch the amount of results Google returns when you query "site: example.com"?
Get the solution ↓↓↓I know this question is asked a lot, but all the answers are outdated because the Search API is disabled by Google.
Google used to have a Search API and with that API it was possible to search in Google using an API. Most answers on StackOverflow link to the deprecated documentation of that API, but because Google turned off that API, I can't solve the issue using the API.
I tried to fix the issue by crawling the google result page and fetching the count from the "about x results found in y seconds" text. But as far as I know, this is restricted because of Google Policy. https://support.google.com/webmasters/answer/66357?hl=en
Anybody has an idea how to fix this issue without breaking the Google Policy? I work with the Symfony Framework in PHP, but I won't mind it if I need to load in a javascript library or a command line tool?
Answer
Solution:
Now that the Search API is gone, it is no longer possible to lawfully scrape Google's general search results.
From the title to your question, it appears that you're looking to scrape a specific site. That can still be done lawfully using the Custom Search JSON/Atom API at https://developers.google.com/custom-search/json-api/v1/overview.
Share solution ↓
Additional Information:
Link To Answer People are also looking for solutions of the problem: an exception occurred in the driver: could not find driver
Didn't find the answer?
Our community is visited by hundreds of web development professionals every day. Ask your question and get a quick answer for free.
Similar questions
Find the answer in similar questions on our website.
Write quick answer
Do you know the answer to this question? Write a quick response to it. With your help, we will make our community stronger.