Question:I am trying to scrape this website: https://bartleby.com, I tried to write a code using Python requests and it works. But I am trying to convert it to PHP because I want the result to be printed on my website and my Cpanel does not read python, so I am forced to use CURL to do this but did not work the code below returns:
My PHP Code (Not Working):
Line 11 is
$response = file_get_contents($url, false, stream_context_create($arrContextOptions));
Full code (Not Working):
Answer:You did not set user agent.
It’s look like that website required user agent from real user such as Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:101.0) Gecko/20100101 Firefox/101.0.
Here is my code that just work.
It should be…
$reqHeaders = curl_getinfo($ch, CURLINFO_HEADER_OUT);to debug request headers.
Your current code did not sent
user-agentat all that’s why it doesn’t work.
If you have better answer, please add a comment about this, thank you!