How to handle JSON data from multiple responses in PHP? - php

I am using curl_multi to send 2 post requests, the data is coming back as one $response correctly, I need to know the correct way to now handle this JSON data in PHP.
When I echo between <pre> tags my JSON is displayed correctly, however now I'm not sure how to take the data I want from it with PHP.
This is usually an easy task with a single API but I'm not sure why I'm having so much trouble with this here whilst using curl_multi!
json being returned inside my pre tags;
{
"Vehicles": [
{
"ExternalVehicleId": "9uq0jz1c",
"StockId": "1234",
"Errors": []
}
]
}
{
"Finance": [
{
"ExternalVehicleId": "9uq0jz1d",
"StockId": "4321",
"Errors": []
}
]
}
This is how the json comes back, but there are way more nested arrays. All appear to be in correct syntax wise.
And here's my php which I'm pretty sure is all fine;
$mh = curl_multi_init();
foreach ($urls as $key => $url) {
$chs[$key] = curl_init($url);
curl_setopt($chs[$key], CURLOPT_RETURNTRANSFER, true);
curl_setopt($chs[$key], CURLOPT_HEADER, false);
curl_setopt($chs[$key], CURLOPT_CONNECTTIMEOUT, 200);
curl_setopt($chs[$key], CURLOPT_TIMEOUT_MS, 25000);
curl_setopt($chs[$key], CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($chs[$key], CURLOPT_POST, true);
curl_setopt($chs[$key], CURLOPT_POSTFIELDS, json_encode($request_contents[$key]));
curl_setopt($chs[$key], CURLOPT_HTTPHEADER, $headers);
curl_multi_add_handle($mh, $chs[$key]);
}
$running = null;
do {
curl_multi_exec($mh, $running);
} while ($running);
foreach (array_keys($chs) as $key) {
$error = curl_error($chs[$key]);
$last_effective_URL = curl_getinfo($chs[$key], CURLINFO_EFFECTIVE_URL);
$time = curl_getinfo($chs[$key], CURLINFO_TOTAL_TIME);
$response = curl_multi_getcontent($chs[$key]); // get results
if (!empty($error)) {
echo "The request $key return a error: $error" . "\n";
} else {
echo "The request to '$last_effective_URL' returned '$response' in $time seconds." . "\n";
echo "<pre>";
echo $response;
echo "</pre>";
}
curl_multi_remove_handle($mh, $chs[$key]);
}
curl_multi_close($mh);

Join the responses into a single JSON array, which you can then json_decode():
$responses = [];
foreach (array_keys($chs) as $key) {
$responses[] = curl_multi_getcontent($chs[$key]);
}
$json = json_decode( '[' . join(',', $responses) . ']' );
var_dump($json);

You could unpack the response object with a foreach.
Something like
foreach($response as $value->$index){
echo “value ” . $value . “ index: ” . $index;
}
This is an example and should give you an idea how to debug your code.
I’m typing from a phone so I’m sorry

You may need to put a comma after closing the Vehicles brackets
{
"Vehicles": [
{
"ExternalVehicleId": "9uq0jz1c",
"StockId": "1234",
"Errors": []
}
]
}
,
{
"Finance": [
{
"ExternalVehicleId": "9uq0jz1d",
"StockId": "4321",
"Errors": []
}
]
}

Related

Why does curl_error not work with curl_multi in php8?

I am trying to get response from an array of webpages parallelly, with the help of curl_multi. Following is what I have tried:
$urls = ["wrongUrl1", "http://Wrongurl2.com"]
$mh = curl_multi_init();
foreach ($urls as $key => $url) {
$chs[$key] = curl_init($url);
curl_setopt($chs[$key], CURLOPT_RETURNTRANSFER, true);
curl_setopt($chs[$key], CURLOPT_FAILONERROR, true);
curl_multi_add_handle($mh, $chs[$key]);
}
//running the requests
$running = null;
do {
curl_multi_exec($mh, $running);
if ($running) {
// Wait a short time for more activity
curl_multi_select($mh);
}
} while ($running);
//getting the responses
foreach(array_keys($chs) as $key){
$error = curl_error($chs[$key]);
$header = curl_getinfo($chs[$key], CURLINFO_HTTP_CODE);
$time = curl_getinfo($chs[$key], CURLINFO_TOTAL_TIME);
$response = curl_multi_getcontent($chs[$key]); // get results
if (!empty($error)) {
echo "The request $key return a error: $error" . "\n";
}
else {
echo "The request to $urls[$key] : $error returned in $time seconds as $header" . "<br>";
}
curl_multi_remove_handle($mh, $chs[$key]);
}
// close current handler
curl_multi_close($mh);
$error always remains an empty string, why?.
Secondly I am able to get individual times for all urls. I wonder if the total time taken by curl_multi_exec to fetch all urls is same as the largest value $time?

Ignore "404" status pages with cURL

I´m using cURL to get informations out of an API and write them in a MySQL table.
The url to the API looks like that:
https://eu.api.blizzard.com/data/wow/item-class/4/item-subclass/1?namespace=static-eu&locale=de_DE&access_token=US6XqgbtQ6rh3EVIqPsejuF62RwP8ljzWn
You can change the number from the url part 1?namespace to another value, for example to 4?namespace to get other informations from the API.
I´m using php "range" to generate the numbers for the url.
Problem:
Some numbers in the url leads to a 404 response, since there are no informations in the API. Example URL:
https://eu.api.blizzard.com/data/wow/item-class/4/item-subclass/18?namespace=static-eu&locale=de_DE&access_token=US6XqgbtQ6rh3EVIqPsejuF62RwP8ljzWn
These "404" pages should get ignored and nothing should be written in MySQL. How is this possible with cURL?
Complete code:
$ids = [];
foreach(range(0, 20) as $number) {
$ids[] = $number;
}
$userAgent = 'Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0';
$mh = curl_multi_init();
$channels = [];
foreach ($ids as $id) {
$fetchURL = 'https://eu.api.blizzard.com/data/wow/item-class/4/item-subclass/' . $id . '?namespace=static-eu&locale=de_DE&access_token=US6XqgbtQ6rh3EVIqPsejuF62RwP8ljzWn';
$channels[$id] = curl_init($fetchURL);
curl_setopt($channels[$id], CURLOPT_RETURNTRANSFER, 1);
curl_setopt($channels[$id], CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($channels[$id], CURLOPT_SSL_VERIFYPEER, 0);
curl_multi_add_handle($mh, $channels[$id]);
}
// execute all queries simultaneously, and continue when all are complete
$running = null;
do {
curl_multi_exec($mh, $running);
curl_multi_select($mh);
} while ($running > 0);
//close the handles
foreach ($ids as $id) {
curl_multi_remove_handle($mh, $channels[$id]);
}
curl_multi_close($mh);
$response = [];
foreach($ids as $id){
$res = curl_multi_getcontent($channels[$id]);
$response[$id] = ($res === false) ? null : json_decode($res, true);
}
echo ("<pre>");
foreach ($response as $item) {
$sqle= "REPLACE INTO `itemsubclasses`
(`class_id`, `subclass`, `name`)
VALUES
('{$item['class_id']}', '{$item['subclass_id']}', '{$item['display_name']}')";
if ($conn->query($sqle) === TRUE) {
echo "Geklappt";
} else {
echo "Problem";
}
}

Access multiple URL at once in Curl PHP

I am working on API that return single currency record in one request. One request take 0.5-1 sec to response, and 15 requests take 7-15 seconds.
As i know server can manage 100s of request per seconds.
I want to hit 15 request on server at once so server will give response in 1-2 seconds not in 15 Seconds. Return all data in one single array to save my loading time.
Check my Code
I am using Loop, loop wait until previous curl request not complete. How can i say to loop, keep continue and dont wait for response.
$time_Start = microtime(true);
$ids = array(1,2,11,15,20,21); // 6 ids in demo, 15+ ids in real
$response = array();
foreach ($ids as $key => $id) {
$response[$id] = get_data($id);
}
echo "Time: ". (microtime(true)-$time_Start)."sec";
// output 5 seconds on 6 request
function get_data($id){
$fcs_api_key = "API_KEY";
$ch=curl_init();
curl_setopt($ch,CURLOPT_URL,"https://fcsapi.com/api/forex/indicators?id=".$id."&period=1d&access_key=".$fcs_api_key);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$buffer = curl_exec($ch);
curl_close($ch);
return $buffer;
}
You can use PHP multi curl https://www.php.net/manual/en/function.curl-multi-init.php
Below I write a code that open Parallel request.
$time_Start = microtime(true);
$ids = array(1,2,3,4,5,6); // You forex currency ids.
$response = php_curl_multi($ids);
echo "Time: ". (microtime(true)-$time_Start)."sec";
// Time: 0.7 sec
Function
function php_curl_multi($ids){
$parameters = "/api/forex/indicators?period=1d&access_key=API_KEY&id="; // ID will set dynamic
$url = "https://fcsapi.com".$parameters;
$ch_index = array(); // store all curl init
$response = array();
// create both cURL resources
foreach ($ids as $key => $id) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url.$id);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$ch_index[] = $ch;
}
//create the multiple cURL handle
$mh = curl_multi_init();
//add the handles
foreach ($ch_index as $key => $ch) {
curl_multi_add_handle($mh,$ch);
}
//execute the multi handle
do {
$status = curl_multi_exec($mh, $active);
if ($active) {
curl_multi_select($mh);
}
} while ($active && $status == CURLM_OK);
//close the handles
foreach ($ch_index as $key => $ch) {
curl_multi_remove_handle($mh, $ch);
}
curl_multi_close($mh);
// get all response
foreach ($ch_index as $key => $ch) {
$response[] = curl_multi_getcontent($ch);
}
return $response;
}

Php Curl to Json

I can get data from a website with CURL.
I want to convert this data to json.
my code:
<?php
function Curlconnect($start,$end,$website) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $website);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$website = curl_exec($ch);
preg_match_all('#'.$start.'(.*?)'.$end.'#si',$website,$ver);
return $ver[1];
curl_close($ch);
}
function nt($start,$bit,$data,$a) {
preg_match_all('#'.$start.'(.*?)'.$bit.'#si',$data,$ver);
return $ver[1];
}
$url = 'http://www.url.com';
$getdata = Curlconnect('<h4','h4>',$url);
for ($a=0; $a<count($getdata); $a++) {
$printdata = nt('>','</',$getdata[$a],$a);
echo $printdata[0]. '<br />';
}
?>
Output:
1
27
32
66
94
I want to convert this data to json like that:
{
"data":{
"numbers":
[
"1",
"27",
"32",
"66",
"94",
]
}
}
How Can I do that?
Thank you very much.
Please try this
$url = 'http://www.url.com';
$getdata = Curlconnect('<h4','h4>',$url);
$jsonData = ["data"];
$jsonData["numbers"] = [];
for ($a=0; $a<count($getdata); $a++) {
$printdata = nt('>','</',$getdata[$a],$a);
$jsonData["numbers"][] = $printdata[0];
}
echo json_encode($jsonData);

Parsing Inconsistent JSON data: PHP

The application Pipedrive gives me inconsistent json data. For example, in some array elements, it gives me "formatted_value":"$3,500","weighted_value":2100,"formatted_weighted_value":"$2,100","rotten_time":null, while in others, it gives me, "formatted_value":"$2,950","rotten_time":null,"weighted_value":2950,"formatted_weighted_value":"$2,950". I would like the json data to be in the order of formatted_value,weighted_value,formatted_weighted_value,rotten_time in every array element, but that's not the case sadly.
Does anyone know of a way to check that the right data is written the right column based on column name and key name?
Below is my code to parse the json data:
function parseFunction($startPos) {
$url = 'urlToCallJsonData';
$ch = curl_init($url); //initialize connection with a URL
if(is_callable('curl_init'))
{
echo "Enabled";
}
else
{
echo "Not enabled";
}
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt ($ch, CURLOPT_CAINFO, dirname(__FILE__)."/cacert.pem");
$json_response = curl_exec($ch);
$info = curl_getinfo($ch);
$status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if ( $status != 200 )
{
die("Error: call to URL $url failed with status $status, response $json_response, curl_error " . curl_error($ch) . ", curl_errno " . curl_errno($ch));
}
curl_close($ch);
$response = json_decode($json_response, true);
$count = Count($response['data']);
for ($x=0; $x<$count; $x++)
{
$currentRecord = $response['data'][$x];
}
//open writing to file
if($startPos == 0)
{
$fp = fopen('cacheDeals.csv', 'w');
}
else
{
$fp = fopen('cacheDeals.csv', 'a');
}
$test_array = $response['data'][0];//test_array = first row of data
// writes the headers to the csv file.
if($startPos == 0)
{
$keys = array_keys($test_array);
fputcsv($fp, $keys);
}
$array_records = $response['data'];//all of incoming data
//write data to csv file
foreach ($array_records as $fields)
{
fputcsv($fp, $fields);
}
//check to see if more data should be written
$more = $response[additional_data][pagination][more_items_in_collection];
$nextStart = $response[additional_data][pagination][next_start];
if($more =="true")
{
downloadPipedriveDealsData($nextStart);
}
}//end of function
parseFunction(0);
Sorry but I cant point this out in comments alone, some of this could be cleaner such as
//open writing to file
if($startPos == 0)
{
$fp = fopen('cacheDeals.csv', 'w');
}
else
{
$fp = fopen('cacheDeals.csv', 'a');
}
$test_array = $response['data'][0];//test_array = first row of data
// writes the headers to the csv file.
if($startPos == 0)
{
$keys = array_keys($test_array);
fputcsv($fp, $keys);
}
Could be simply this
// writes the headers to the csv file.
if($startPos == 0){
$fp = fopen('cacheDeals.csv', 'w');
fputcsv($fp, array_keys($response['data'][0]));
}else{
$fp = fopen('cacheDeals.csv', 'a');
}
I don't see a purpose to this whole block at all
$count = Count($response['data']);
for ($x=0; $x<$count; $x++)
{
$currentRecord = $response['data'][$x];
}
This syntax is invalid
$more = $response[additional_data][pagination][more_items_in_collection];
$nextStart = $response[additional_data][pagination][next_start];
And will issue a Notice ( undefined constant assuming '' ) etc. because there are no quotes around the string keys in the arrays. In the unlikly event that one of those keys is a constant, that's a whole other can of worms. Because you will never get your data out then. They should be done this way with ' or " 's around them. see also What does the PHP error message "Notice: Use of undefined constant" mean?
$more = $response['additional_data']['pagination']['more_items_in_collection'];
$nextStart = $response['additional_data']['pagination']['next_start'];
Just saying.

Categories