I have bought a linux server from Godaddy . I tested to connect and get response from remote server , but it says CURL Failed . Also , it doesn't work on Node app too .
Here is the PHP Code :
<?php
$url = 'http://dev.api/pay';
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_POST, true );
curl_setopt( $ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_POSTFIELDS, "<xml>here</xml>" );
curl_setopt($ch, CURLOPT_PORT, '9085');
$result = curl_exec($ch);
$output = curl_getinfo($ch);
echo "<pre>";
var_dump($output);
echo "</pre>";
curl_close($ch);
var_dump($result);
. Actually this code is working on localhost , but it doesn't work when accessed from the server. If the outgoing connection is being blocked from the server's firewall i rented , how can i disable it or allow that port i intended to use only ?
Do you have root access?
if so, use iptables to remove outgoing connection block.
If you do not have root acces, your host just does not allow outbound connections and there is nothing you can do about it.
Related
I am tasked with building an API to receive inbound XML data. On my client, I have this code.
$url = "http://stackoverflow.com";
$xml = '<?xml version="1.0" encoding="UTF-8"?><Request PartnerID="asasdsadsa" Type="TrackSearch"> <TrackSearch> <Title>love</Title> <Tags> <MainGenre>Blues</MainGenre> </Tags> <Page Number="1" Size="20"/> </TrackSearch> </Request>';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt( $ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
curl_setopt( $ch, CURLOPT_POSTFIELDS, "xml=".$payload );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
$request = curl_exec($ch);
curl_close($ch);
On my remote server, I have this code
function TransmitRx()
{
$xml = trim(file_get_contents('php://input'));
file_put_contents("newStandard/".rand(100,500)."received.xml", $xml);
}
//Listen for inbound data
TransmitRx()
If I open up the server endpoint URL, there is an empty file saved. I don't know why. But when I run the client-side script. I get nothing. No errors. Nothing.
I have looked at several of the pages here and every one of them has a similar cURL statement to send data.
Why am I not receiving any post data at the API endpoint?
I have been unsuccessful at any information via the WWW.
UPDATE
Final Code that works:
function get_url($request_url, $payload)
{
$headers = [
"Access-Control-Allow-Origin: *",
"Content-type: text/xml",
"Content-length: " . strlen($payload),
"Connection: close",
];
$data = ['xml' => $payload];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $request_url);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt( $ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt( $ch, CURLOPT_POST, true );
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
$response = curl_exec($ch) or die(curl_error($ch));;
if (curl_errno($ch)) {
print curl_error($ch);
} else {
curl_close($ch);
}
return $response;
}
$request_url = 'https://stackoverflow.com/posts';
$response = get_url($request_url, $payload);
I wish I knew for sure what caused it to start working. I was reading this page last.
https://www.php.net/manual/en/curlfile.construct.php
Okay, so if adding the lines I suggested to disable certification / peer validation enabled the process to work, then that just means the remote server is using an SSL certificate that is not trusted by cURL.
Those lines are NOT the final fix. You never want to disable SSL validation in a real environment. I only suggested you temporarily disable them to see if that was truly the problem. If you leave them disabled, then you are leaving your code vulnerable to man-in-the-middle (MITM) attacks.
The correct fix is usually to point cURL at an updated CA bundle. The official cURL website graciously provides these here:
https://curl.haxx.se/docs/caextract.html
The process is that you download the CA bundle file and put it somewhere where your script can find it, and then add this curl option:
curl_setopt ($ch, CURLOPT_CAINFO, "full path to the CA bundle file.pem");
If the remote server's certificate came from any of the major CA vendors out there, this should be all you need to do.
If the remote server's certificate is self-signed or something then you might need to download the specific CA certificate and any supporting intermediate CA certificates and tell cURL to find them.
I have and xml url from a supplier which generates xml content dynamically with php like;
http://www.example.com/outputxml/index.php?xml_service_id=161
This url is valid for a static ip so I gave him my websites hosting ip. Is there a way to open that url in browser with data scraping? Because My internet connection has no static ip.
Thank you.
I have tried below code;
$url = 'http://www.example.com/outputxml/index.php?xml_service_id=161?';
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_POST, true );
curl_setopt( $ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_POSTFIELDS, $xml );
$result = curl_exec($ch);
curl_close($ch);
echo $result;
But it gave html format.
Save the content on your server with something like a wget and then serve it. Please notice that you are probably going to infringe the policy of the xml's author (I don't know the consequences or the policy itself, but you should be careful), so you might consider to at least add a .htacces authentication on your server's page, just to not make the xml public.
I am trying to send push notification some android devices using the registration ids which is stored in database table.
Here is my code
//message and title for push notification
$msg=$_POST['message'];
$title=$_POST['title'];
$result=$connect->prepare("SELECT `push_id` FROM `user_devices`");
$result->execute();
//get registration id's
$ids=$result->fetchAll(PDO::FETCH_COLUMN);
$key="XXXXX";
$url = 'https://android.googleapis.com/gcm/send';
$fields = array(
'registration_ids' =>$ids,
'data' => array( "message" =>$msg,"title"=>$title,"soundname"=>"beep.wav" ),
);
$headers = array(
'Authorization: key=' . $key,
'Content-Type: application/json'
);
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_POST, true );
curl_setopt( $ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_POSTFIELDS, json_encode( $fields ) );
$res = curl_exec($ch);
if($res === false)
echo 'Curl failed ' .curl_error();
}
I am getting curl Failed error message but didn't see anything from curl_error().
UPDATE
I am running this script from windows server which is hosted in azure cloud service.
Now i am run this from linux based server its works fine,but it didnt work in windows server.
You are probably getting error due to SSL (in your url https) issue. You have to configure your curl to handle secured request. You can find a hints from here: Configuring cURL for SSL
You are not getting the error msg because you need to print it like below:
echo 'Curl failed ' .curl_error($ch); // you are missing $ch here
Alternately you can run your curl with verbose enabled to see the things happening with your curl request!
curl_setopt( $ch, CURLOPT_VERBOSE, true );
It was the problem of My SSL Certificate.
Solved using one line code
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); is placed in after curl_init.
NOTE
This is a temporary solution.Actual way is generate an SSl certificate.
My server sents GCM request to the GCM api for the push notification.The push notification works fine for some couple of push notification, after that the curl function which is making the request is getting timed out with the following error
"Curl failed: Failed to connect to android.googleapis.com port 443: Connection timed out"
without any response.But the response code is 200Ok.
But again after sometime if I try it will sent the notification with a proper success response and if I try sending for sometime, to different devices it again hangs and shows the above message.
I have opened port 5228, 5229, and 5230 on my firewall.
Why does this happen.I can't figure why it is happening like this.
This is the code Im using
$headers = array(
'Authorization: key=' . $this->serverApiKey,
'Content-Type: application/json'
);
// Open connection
$ch = curl_init();
// Set the url, number of POST vars, POST data
curl_setopt( $ch, CURLOPT_URL, $this->url );
curl_setopt($ch, CURLOPT_PORT, 443);
curl_setopt( $ch, CURLOPT_POST, true );
curl_setopt( $ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_POSTFIELDS, json_encode( $fields ) );
// Avoids problem with https certificate
curl_setopt( $ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false);
// Execute post
$result = curl_exec($ch);
Please help me to find a solution for this.
This seems to be more of a firewall/proxyserver issue.. Make sure you have port 443 open in your server. Make use of the ping and the curl command to verify this.
See this Stack Overflow issue to get more insight.
I am trying to get some information from a UK retailer's website, and on many of the scrapes I have done it's quite simple. However, there are a number where I just cannot get around issues where most of the time they are caused by cookies. This was a a great SO question, but it's not helped.
I have the following PHP function...
function file_get_contents_curl_many_redir2( $url, $timeout = 15 ) {
$cookie = tempnam ("/tmp", "CURLCOOKIE");
$verbose = fopen('php://temp', 'rw+');
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0" );
curl_setopt( $ch, CURLOPT_COOKIEJAR, $cookie );
curl_setopt( $ch, CURLOPT_COOKIEFILE, $cookie ); //
curl_setopt( $ch, CURLOPT_COOKIESESSION, true );
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false); //
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_ENCODING, "" );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_AUTOREFERER, true );
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, $timeout );
curl_setopt( $ch, CURLOPT_TIMEOUT, $timeout );
curl_setopt( $ch, CURLOPT_MAXREDIRS, 100 );
curl_setopt( $ch, CURLOPT_VERBOSE, true);
curl_setopt( $ch, CURLOPT_STDERR, $verbose);
$data = curl_exec($ch);
rewind($verbose);
$verboseLog = stream_get_contents($verbose);
echo "Verbose information:\n<pre>", htmlspecialchars($verboseLog), "</pre>\n";
$curlVersion = curl_version();
extract(curl_getinfo($ch));
$metrics = <<<EOD
URL....: $url
Code...: $http_code ($redirect_count redirect(s) in $redirect_time secs)
Content: $content_type Size: $download_content_length (Own: $size_download) Filetime: $filetime
Time...: $total_time Start # $starttransfer_time (DNS: $namelookup_time Connect: $connect_time Request: $pretransfer_time)
Speed..: Down: $speed_download (avg.) Up: $speed_upload (avg.)
Curl...: v{$curlVersion['version']}
EOD;
var_dump($metrics);
if(curl_errno($ch)){
echo 'Curl error: ' . curl_error($ch) . "on url:" .$url;
var_dump(curl_getinfo($ch, CURLINFO_HTTP_CODE));
}
curl_close($ch);
return $data;
}
On the majority of websites this works (I in fact have simpler functions as most don't redirect lots of times.)
But with www.homebase.co.uk when I use this url http://www.homebase.co.uk/SearchDisplay?pageSize=43&searchSource=Q&resultCatEntryType=2&pageView=&catalogId=10011&showResultsPage=true&beginIndex=0&langId=110&categoryId=&storeId=10201&sType=SimpleSearch&searchTerm=$sku where $sku is the 6 number SKU that Homebase uses (which is also the same as the last 6 numbers of any product page) I get no information.
When I run the URL through Redurect Detective it seems to be an unsupported browser issue, I presume because it's not a browser accessing the webite, and it knows this.
The Main Question:
How do I fix it so I can then get the correct source code to this page? (I am currently just getting blank text, not the product page(s) I want)
Related questions
There's a handful of other websites that don't "let me in" either, is this just cookie related, or is the coding of their website "smart enough" to know whether it's a browser or not? Can I fool it into thinking (this may be answered via my primary question).
Related extra information that may help answer the above.
* When I use CURLOPT_MAXREDIRS to 100, it seems to only let me go up to 40. Could this be the issue?
* using $sku = 323526; you should arrive at the final URL after some redirects to http://www.homebase.co.uk/en/homebaseuk/sovereign-petrol-self-propelled-rotary-mower---1493cc---40cm-323526. I do not need to know where the CURL ends up, I just want to be able to pinch the title, image and other info from the product page from knowing the SKU!