I want to grab this number (28/28) with PHP from a tracking software (https://aircrasher.livefpv.com/live/scoring/) for drone races. But I can't. I think it's a websocket, but I don't know how to retrieve information from it. I tried curl, I tried the "ultimate-web-scraper" from GitHub (https://github.com/cubiclesoft/ultimate-web-scraper). I tried a code example from another post from here (PHP simple web socket client) but I'm not able to get this number. Is it imposible to get this number (with php)?
This is what I tried so far with the help of "ultimate-web-scraper" (https://kleesit.de/beta/livetime/webscraper/test.php):
<?php
require_once "support/web_browser.php";
require_once "support/tag_filter.php";
// Retrieve the standard HTML parsing array for later use.
$htmloptions = TagFilter::GetHTMLOptions();
// Retrieve a URL (emulating Firefox by default).
$url = "https://aircrasher.livefpv.com/live/scoring/";
$web = new WebBrowser();
$result = $web->Process($url);
// Check for connectivity and response errors.
if (!$result["success"])
{
echo "Error retrieving URL. " . $result["error"] . "\n";
exit();
}
if ($result["response"]["code"] != 200)
{
echo "Error retrieving URL. Server returned: " . $result["response"]["code"] . " " . $result["response"]["meaning"] . "\n";
exit();
}
// Get the final URL after redirects.
$baseurl = $result["url"];
// Use TagFilter to parse the content.
$html = TagFilter::Explode($result["body"], $htmloptions);
// Find all anchor tags inside a div with a specific class.
// A useful CSS selector cheat sheet: https://gist.github.com/magicznyleszek/809a69dd05e1d5f12d01
echo "All the URLs:\n";
$result2 = $html->Find("div.someclass a[href]");
if (!$result2["success"])
{
echo "Error parsing/finding URLs. " . $result2["error"] . "\n";
exit();
}
foreach ($result2["ids"] as $id)
{
// Faster direct access.
echo "\t" . $html->nodes[$id]["attrs"]["href"] . "\n";
echo "\t" . HTTP::ConvertRelativeToAbsoluteURL($baseurl, $html->nodes[$id]["attrs"]["href"]) . "\n";
}
// Find all table rows that have 'th' tags.
// The 'tr' tag IDs are returned.
$result2 = $html->Filter($html->Find("tr"), "th");
if (!$result2["success"])
{
echo "Error parsing/finding table rows. " . $result2["error"] . "\n";
exit();
}
foreach ($result2["ids"] as $id)
{
echo "\t" . $html->GetOuterHTML($id) . "\n\n";
}
?>
This is what I tried with CURL and the help of a stackoverflow post (https://kleesit.de/beta/livetime/test2.php):
<?php
// random 7 chars digit/alphabet for "t" param
$random = substr(md5(mt_rand()), 0, 7);
$socket_server='https://aircrasher.livefpv.com/live/scoring/';
// create curl resource
$ch = curl_init();
//N°1 GET request
curl_setopt_array(
$ch,
[
CURLOPT_URL => $socket_server,
CURLOPT_RETURNTRANSFER => TRUE,
// since it is TRUE by default we should disable it
// on our localhost, but from real https server it will work with TRUE
CURLOPT_SSL_VERIFYPEER => FALSE
]);
// $output contains the output string
$output = curl_exec($ch);
$output=substr($output, 1);
echo $socket_server;
// server response in my case was looking like that:
// '{"sid":"4liJK2jWEwmTykwjAAAR","upgrades":["websocket"],"pingInterval":25000,"pingTimeout":20000,"maxPayload":1000000}'
var_dump($output);
$decod=json_decode($output);
// setting ping Interval accordingly with server response
$pingInterval = $decod->pingInterval;
//N°2 POST request
$socket_server_with_sid = $socket_server.'&sid='.$decod->sid;
curl_setopt_array(
$ch,
[
CURLOPT_URL => $socket_server_with_sid,
CURLOPT_POST => TRUE,
CURLOPT_TIMEOUT_MS => $pingInterval,
// 4 => Engine.IO "message" packet type
// 0 => Socket.IO "CONNECT" packet type
CURLOPT_POSTFIELDS => '40'
]);
$output = curl_exec($ch);
// ok
var_dump($output);
// Pervious 2 requests are called "hand shake" now we can send a message
// N°3 socket.emit
if ($output==='ok') {
curl_setopt_array(
$ch,
[
CURLOPT_URL => $socket_server_with_sid,
CURLOPT_TIMEOUT_MS => $pingInterval,
CURLOPT_POST => TRUE,
// 4 => Engine.IO "message" packet type
// 2 => Engine.IO "EVENT" packet type
CURLOPT_POSTFIELDS => '42["chat message","0devhost message 10"]'
]);
$output = curl_exec($ch);
// ok
echo $output.'<br/>';
echo $socket_server_with_sid;
}
// close curl resource to free up system resources
curl_close($ch);
?>
Related
I have never used Curl but I am trying to complete a self api project at my job just to gain some experience.
And I am stuck on the first step... I want to authenticate with an api.
So I am running this code and I expect to see a Success 200 response with my access token, etc but I get nothing.
No error, no feedback, the page just opens up blank
I have tired to use CURLINFO_HEADER_OUT from this page What Steps do you Take to Troubleshoot Problems with PHP cURL? but still I got a blank page
Anyway thank you to anyone in advantage for some tips
<?php
const TOKEN_ENDPOINT = 'xxxxxx';
const GRANT_TYPE = 'xxxxx';
const CLIENTID = 'xxxxxxxxxxx';
const CLIENTSECRET = 'xxxxxxxxxxxxxx';
const USERNAME = 'xxxxxxxxxxxxxxxxx';
const PASSWORD = 'xxxxxxxxxxxxxxx';
$clientCredentials = base64_encode(CLIENTID . ':' . CLIENTSECRET);
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => TOKEN_ENDPOINT,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 0,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'POST',
CURLOPT_POSTFIELDS =>'grant_type=' . GRANT_TYPE . '&username=' . USERNAME . '&password=' . PASSWORD ,
CURLOPT_HTTPHEADER => array(
'Content-Type: application/x-www-form-urlencoded',
'Accept: application/json',
'Authorization: Basic ' . $clientCredentials
),
));
$response = curl_exec($curl);
curl_close($curl);
echo $response ;
?>
To check curl error, the best way is to use curl_error function
$response = curl_exec($curl);
if (curl_errno($curl)) {
$error_msg = curl_error($curl);
}
curl_close($curl);
echo $response ;
See the description of libcurl error codes here
See the description of PHP curl_errno() function here
See the description of PHP curl_error() function here
When all else fails, i do this (code snippet from my ApiHelper curl wrapper). Use your own logger or evidence printing mechanism. Most every time the answer to the puzzle is in the printed stuff :
// we simply stream curlopt debug info to a temporary
// file, so we can log it out later (when helper is set
// to verbose)
$st = microtime(true);
$verbiage = null;
if ($this->verbose) {
// write out the curl debug stuff
curl_setopt($ch , CURLINFO_HEADER_OUT , false);
curl_setopt($ch , CURLOPT_VERBOSE , true);
$verbiage = fopen('php://temp' , 'w+');
curl_setopt($ch , CURLOPT_STDERR , $verbiage);
}
$resp = curl_exec($ch);
$end = microtime(true); // get as float
$delta = 1000.0 * ($end - $st); // treat as float
$this->roundTripInMs = sprintf("%.2f" , $delta);
$this->getInstanceLogger()->debug("WS call : round trip took " . sprintf("%.2f" , $delta) . " ms.");
if ($this->verbose) {
// rewind and log the verbose output
rewind($verbiage);
$verboseLog = stream_get_contents($verbiage);
$this->getInstanceLogger()->info("Verbose cURL : \n$verboseLog");
fclose($verbiage);
}
$this->curlinfo = curl_getinfo($ch);
$this->verbose && $this->getInstanceLogger()->info("cURL info : \n" . json_encode($this->curlinfo , PEHR_PRETTY_JSON));
$this->httpStatus = curl_getinfo($ch , CURLINFO_RESPONSE_CODE);
if (!$resp) {
if ($this->verbose) $this->getInstanceLogger()->error("CURL request to [$url] failed\n" . json_encode($this->curlinfo , PEHR_PRETTY_JSON));
return 0;
}
curl_close($ch);
if ($resp && $this->verbose) $this->getInstanceLogger()->debug("Received json \n" . $resp);
I tried to open an external url. It works fine in my local server. But when i moved to live server it showing time out error.
When i replaced the url by a url in the same domain ,it works fine.
allow_url_fopen is ON in the server.
<?php
if ($fp = fopen('https://www.google.com/', 'r')) {
$content = '';
// keep reading until there's nothing left
while ($line = fread($fp, 1024)) {
$content .= $line;
}
echo $content;
echo 'do something with the content here';
// ...
} else {
echo 'an error occured when trying to open the specified url';
}
?>
Updated
$curl_handle=curl_init();
curl_setopt($curl_handle,CURLOPT_URL,'https://www.google.co.in/');
curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,2);
curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);
if (empty($buffer)){
print "Nothing returned from url..<p>";
}
else{
print $buffer;
}
I tried cURL too. It returns "Nothing returned from url..". But it works fine in my local and demo server.
You should use curl to get data from third party URL.
please check the below link and example :
What is cURL in PHP?
Example :
$ch = curl_init();
$curlConfig = array(
CURLOPT_URL => "http://www.example.com/yourscript.php",
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_POSTFIELDS => array(
'field1' => 'some date',
'field2' => 'some other data',
)
);
curl_setopt_array($ch, $curlConfig);
$result = curl_exec($ch);
curl_close($ch);
Currently I have a system that sends multiple requests to a REST API. It is structured something like this:
foreach ($data as $d)
{
$ch = curl_init( $url );
curl_setopt( $ch, CURLOPT_POST, 1);
curl_setopt( $ch, CURLOPT_HTTPHEADER, (array of data here));
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec( $ch );
$retry = 0;
while((curl_errno($ch) == 7 || curl_errno($ch) == 52) && $retry < 3)
{
$response = curl_exec($ch);
$retry++;
}
curl_close($ch);
(decode XML Response and loop)
}
(I can't expose the whole code so I have filled in the operations that are happening in brackes)
However after a few hundred requests the FastCGI script stalls. The REST API will still respond during this period if I query it in another fashion, but this batch client will not send anymore requests. After a few minutes, it will start responding again. I'm not sure why this is stalling, I can see via htop that there is no CPU activity on the threads at either end whilst this is happening.
Is there any reason why the cURL/PHP script would stall here?
if you allowed to use external PHP libraries;
I'd like to suggest this method:
https://github.com/php-curl-class/php-curl-class
// Requests in parallel with callback functions.
$multi_curl = new MultiCurl();
$multi_curl->success(function($instance) {
echo 'call to "' . $instance->url . '" was successful.' . "\n";
echo 'response: ' . $instance->response . "\n";
});
$multi_curl->error(function($instance) {
echo 'call to "' . $instance->url . '" was unsuccessful.' . "\n";
echo 'error code: ' . $instance->error_code . "\n";
echo 'error message: ' . $instance->error_message . "\n";
});
$multi_curl->complete(function($instance) {
echo 'call completed' . "\n";
});
$multi_curl->addGet('https://www.google.com/search', array(
'q' => 'hello world',
));
$multi_curl->addGet('https://duckduckgo.com/', array(
'q' => 'hello world',
));
$multi_curl->addGet('https://www.bing.com/search', array(
'q' => 'hello world',
));
$multi_curl->start();
When i'm trying to invoke the YQL via cURL i'm getting the following error.
HTTP Version Not Supported
Description: The web server "engine1.yql.vip.bf1.yahoo.com" is using an unsupported version of the HTTP protocol.
Following is the code used
// URL
$URL = "https://query.yahooapis.com/v1/public/yql?q=select * from html where url=\"http://www.infibeam.com/Books/search?q=9788179917558\" and xpath=\"//span[#class='infiPrice amount price']/text()\"&format=json";
// set url
curl_setopt($ch, CURLOPT_URL, $URL);
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// $output contains the output string
$output = curl_exec($ch);
// close curl resource to free up system resources
curl_close($ch);
echo $output;
?>
Invoking the same URL from thr browser works fine
https://query.yahooapis.com/v1/public/yql?q=select * from html where
url="http://www.infibeam.com/Books/search?q=9788179917558" and
xpath="//span[#class='infiPrice amount price']/text()"&format=json
Can someone please point me what is wrong in the code?
The problem is probably caused because the url you feed to cURL is not valid. You need to prepare / encode the individual values of the query strings for use in a url.
You can do that using urlencode():
$q = urlencode("select * from html where url=\"http://www.infibeam.com/Books/search?q=9788179917558\" and xpath=\"//span[#class='infiPrice amount price']/text()\"");
$URL = "https://query.yahooapis.com/v1/public/yql?q={$q}&format=json";
In this case I have only encoded the value of q as the format does not contain characters that you cannot use in a url, but normally you'd do that for any value you don't know or control.
Okay I gottacha .. The problem was with the https. Used the following snippet for debug
if (false === ($data = curl_exec($ch))) {
die("Eek! Curl error! " . curl_error($ch));
}
Added below code to accept SSL certificates by default.
$options = array(CURLOPT_URL => $URL,
CURLOPT_HEADER => "Content-Type:text/xml",
CURLOPT_SSL_VERIFYPEER => 0,
CURLOPT_RETURNTRANSFER => TRUE
);
Complete code is here
<?php
// create curl resource
$ch = curl_init();
// URL
$q = urlencode("select * from html where url=\"http://www.infibeam.com/Books/search?q=9788179917558\" and xpath=\"//span[#class='infiPrice amount price']/text()\"");
$URL = "https://query.yahooapis.com/v1/public/yql?q={$q}&format=json";
echo "URL is ".$URL;
$ch = curl_init();
//Define curl options in an array
$options = array(CURLOPT_URL => $URL,
CURLOPT_HEADER => "Content-Type:text/xml",
CURLOPT_SSL_VERIFYPEER => 0,
CURLOPT_RETURNTRANSFER => TRUE
);
//Set options against curl object
curl_setopt_array($ch, $options);
//Assign execution of curl object to a variable
$data = curl_exec($ch);
echo($data);
//Pass results to the SimpleXMLElement function
//$xml = new SimpleXMLElement($data);
echo($data);
if (false === ($data = curl_exec($ch))) {
die("Eek! Curl error! " . curl_error($ch));
}
if (200 !== (int)curl_getinfo($ch, CURLINFO_HTTP_CODE)) {
die("Oh dear, no 200 OK?!");
}
//Close curl object
curl_close($ch);
?>
My question is I want to get acess my fb friends using curl and decode into json then i want to show only those friends whose name starting with letter a such as aman,adam etc pls help me..Following is my code.
<?php
// create a new cURL resource
$json_url="https://graph.facebook.com/100001513782830/friends?access_token=AAACEdEose0cBAPdK62FSjs4RvA21efqc8ZBKyzAesT5r4VSpu0XScAYDtKrCxk4PmcRBVzE2SLiGvs2d5FeXvZAD72ZCShwge3vk4DQqRAb8vLlm1W3";
$ch = curl_init( $json_url );
// Configuring curl options
/* $options = array(
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => array('Content-type: application/json')
);
// Setting curl options
curl_setopt_array( $ch );
*/// Getting results
$result = curl_exec($ch); // Getting jSON result string
$obj = json_decode($result, true);
foreach($obj[data] as $p)
{
echo '
Name: '.$p[name][first].'
Age: '.$p[age].'
';
}
You will offcourse try not to hardcode "a" but for this purpose :
foreach($obj[data] as $p){
if(strtolower(substr(trim($p[name][first]),0,1)) == 'a'){
echo 'Name: '.$p[name][first].'Age: '.$p[age];
}
}
Btw, it is not a good idea to post security tokens (in URL) to public places.
Since the name is string, you can simply iterate over that array and filter by name:
$letter = 'A';
foreach($obj['data'] as $p) {
if ($p['name'][0] == $letter) {
// do something with $p
}
}
But there is a little problem with UTF-8 -- this solution (and that one with substr too) will not work on multibyte characters. So you need to use mb_substr instead of plain substr function:
foreach($obj['data'] as $p) {
if(mb_strtolower(mb_substr($p['name'], 0, 1))) == 'Á'){
echo "Name: ", $p['name'], "\n",
"Age: ", $p['age'], "\n";
}
}