I have a site that uses the new version of Google ReCaptcha (I am not a robot version), and am having trouble getting the challenge response on my shared server.
I cant use curl or file_get_contents due to restrictions on the server. Is there any other ways to get the response?
The code I was using locally, that does not work on the live site is:
CURL
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$response=$this->get_curl_response("https://www.google.com/recaptcha/api/siteverify?secret=SECRET&response=".$_POST['g-recaptcha-response']."&remoteip=".$_SERVER['REMOTE_ADDR']);
File Get Contents
$response=$this->get_curl_response("https://www.google.com/recaptcha/api/siteverify?secret=SECRET&response=".$_POST['g-recaptcha-response']."&remoteip=".$_SERVER['REMOTE_ADDR']);
This turned out to be a problem with the hosting company that could not be resolved, the challenge response from the captcha was being blocked by the hosting company's firewall, and therefore the capthcha always failed.
Because I am on a shared server they could not enable file_get_contents() as it would be a security risk for all the sites on that server.
I installed PHP Captcha (https://www.phpcaptcha.org/) as an alternative, whoch works fine as all the logic is done locally.
Related
I am recently working on a project on which I need to scrape some data from an external website. It is working on Localhost but stopped working on the live host. I explored on google as well on StackOverflow where people suggested that open PHP curl extension etc but everything already opened because I am doing a lot more scraping on that hosting which is working as cake.
Code is
$url = "https://pakesimdata.com/sim/search-result.php";
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_TIMEOUT, 40);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POSTFIELDS, "cnnum=3005210911");
echo $html = curl_exec($ch);
When I echo the result with URL https://pakesimdata.com/sim/search.php or https://pakesimdata.com/ I got the result but it is not working when I do the POST request to acquire the result on URL https://pakesimdata.com/sim/search-result.php. It shows me nothing. I also do the error handling but got no luck, gave no error which put me a great headache. I can not grab what's going on and on which section I need to work to get the results.
It is working on Localhost but stopped working on the live host.
This would suggest two possibilites
1.) Your host does not like outgoing http connections which you rules out already
2.) The remote host does not like your scraping
ad 2.)
Can it be they have blocked your IP address or some other mechanism is protecting the page from beeing used by your script as usually operators of services like this do not like to be scraped by bots.
LOCALHOST will probably appear as your own private IP address with probably a dynamic
always changing IP address - like an ordinary user.
Your real server will have a fixed IP adress and probably the external website analzes the traffic and blocks ip adresses for abuse if a lot of requests are coming from the same IP.
I have been using cURL code to access Beanstream for awhile now, and all of a sudden it stopped working as of yesterday. I have determined that this has to do with accessing the HTTPS URL for Beanstream processing, as if I test that same code and just go to the HTTP URL, it works (of course it returns an insecure connection code).
Prior to the below code I was setting VERIFYPEER / VERIFYHOST to FALSE, which was working up until yesterday.
$url = 'https://www.google.com';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_CAINFO, "/xxxx/public_html/_cert/GeoTrustGlobalCA_google.crt");
$getResponse = (curl_exec($ch));
curl_close($ch);
print_r($getResponse);
I have setup the above test code to try and access Google via HTTPS. I had read that a proper connection requires the VERIFYPEER / CAINFO options to be set, which I have done and grabbed the Google .crt and placed it on the web server as well.
I still get a 503 Service Unavailable error back though.
The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
Additionally, a 503 Service Unavailable error was encountered while
trying to use an ErrorDocument to handle the request.
I have a dedicated server hosted at OVH and I have the following PHP CURL script that calls the Facebook Graph API to post on a user wall:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://graph.facebook.com/v2.6/'.$user.'/feed');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'link='.$link.'&access_token=************');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
if(curl_errno($ch)){
error_log(curl_error($ch));
}
else {
error_log($response);
}
The expected behavior is that the script should log the facebook response. I noticed a LOT of times, the script doesn't return ANY response at all and not even a CURL error.
I have also a script that uses file_get_contents. I noticed also that every few days I suddenly get a huge amount of connection timed out calling the Facebook Graph API. It happens randomly and for no reason!
When I try to call the same URL from browser immediately after it's logged in the error log, it works successfully.
I'm really confused :( Do you think these are connectivity issues with OVH servers or the Graph API or what?
Thanks
wanted to comment, not post an answer but I don't have enough rep :(
For https using curl I usually add:
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
I installed localhost (xammp, wampserver) on a VDS. When I try to get a file using PHP Curl and file_get_contents, the download speed is very low. I can download a 100mb file in 10 minutes. If I try to download the same file with a browser, the duration is only 3 seconds. What can be the reason?
Thanks for your interest.
Downloading content at a specific URL is common practice on the internet, especially due to increased usage of web services and APIs offered by Amazon, Alexa, Digg, etc. PHP's cURL library, which often comes with default shared hosting configurations, allows web developers to complete this task.
u can try
/* gets the data from a URL */
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
The Usage
$returned_content = get_data('http://davidwalsh.name'); //something like this
Alternatively, you can use the file_get_contents function remotely, but many hosts don't allow this.
As recommended by my DocuSign account manager, I am using Fiddler2 to capture the necessary trace for API certification. I am unable to retrieve the trace from the DocuSign domain and have narrowed it down to the fact that these are cURL calls.
According to Fiddler2, http://fiddler2.com/documentation/Configure-Fiddler/Tasks/ConfigurePHPcURL, the advice is to add the following to code:
curl_setopt($ch, CURLOPT_PROXY, '127.0.0.1:8888');
where $ch = curl_init().
I've also tried
curl_setopt($curl, CURLOPT_PROXY, '127.0.0.1:8888');
Still no dice. I only get traffic from my application site. The following is all of my curl code:
$url = "https://demo.docusign.net/restapi/v2/login_information";
$curl = curl_init($url);
$ch = curl_init();
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, array("X-DocuSign-Authentication: $header"));
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($curl, CURLOPT_CAINFO, getcwd() ."/**the cert info");
curl_setopt($ch, CURLOPT_PROXY, '127.0.0.1:8888');//allows fiddler to see requests
$json_response = curl_exec($curl);
$status = curl_getinfo($curl, CURLINFO_HTTP_CODE);
It's definitely talking to the DocuSign domain as my application is working, I'm just trying to get the trace. Any help is appreciated.
fiddler is client side program, it cannot see server traffic to other servers only traffic between client and server.
Unless your server is running locally (on the same computer that you are running fiddler) using 127.0.0.1 this will not work as 127.0.0.1 is the loopback ip for the computer, in this case the server would be trying to use itself as a proxy (which would be ok if the server computer itself was the one running fiddler). You need to change the ip to the computer running fiddler and make sure the server can access that port.
I was facing the exact same scenario, and I used a protocol analyzer such as Wireshark or TCPDUMP to see HTTP traffic at network level.
Of course the server needs to be running locally. Bellow you can find a screenshot example of traffic capture where you can clearly see the HTTP GET going out.