I have a script that fetches data from another API. The script is taking too much time while retrieving data from API around 1 hour and 30 minutes. The purpose of this script is to copy the reviews and send them to another account. The script first gathers the feedbacks and then based on order id obtained from feedback, iterates that feedback array and then performs curl request to some other API to obtain some data based on that order id. But that curl requests in loop taking too much time to return data.
The feedback are 3000 around.
How can i handle script to return data sooner?
$feedbacks = getFeedbacks();
if(count($feedbacks)>0){
foreach($feedbacks as $feedback){
$getData($feedback->order_id);
}
}
function getData($orderId)
{
$orderId = $orderId;
$api = "api address here";
$curl = curl_init($api);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$curl_response = curl_exec($curl);
$json = json_decode($curl_response);
}
The problem here is when u call curl in loop the second call need the first to complete then the second can start and that take long time
To fix this u can use (pclose popen) the script will pass them in background and in second script u can manage your return data:
First script :
<?php
$feedbacks = getFeedbacks();
if(count($feedbacks)>0){
foreach($feedbacks as $feedback){
$getData($feedback->order_id);
}
}
function getData($orderId)
{
$orderId = $orderId;
pclose(popen('sudo /usr/bin/php /home/secondscript.php -o '.$orderId." >/dev/null &", 'r'));
}
Second script :
<?php
$command_line_args= getopt("o:");
$orderId=$command_line_args["o"];
$api = "api address here";
$curl = curl_init($api);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$curl_response = curl_exec($curl);
$json = json_decode($curl_response);
{here u can use the json return ..}
Related
I am using a php script, where I want to send a sms through the api with cURL, but if my phone no is multiple (it is in an array) and I used the foreach to send the sms for every contact no, but when I use cURL request in foreach then it is taking only first contact no, and after that it is not able to send the sms to another no which is available into my contact array.
Here is the code to understand.
function sendSMS($data){
$phones = $data['recipient_contact'];
$explode_contact = explode(",", $phones);
//read all the contact and send the mail
foreach($explode_contact as $k=>$v){
$contact_number = '91'.$v;
$sms_body = $data['sms_body'];
$curl = curl_init();
$data = array(
'aid'=>'XXXX',
'pin'=>'XXXX',
'mnumber'=>$contact_number,
'message'=>$sms_body,
'singnature'=>'XXXXX');
$url = sprintf("%s%s","http://mysmslink/HttpLink",http_build_query($data));
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($curl);
sleep(5);
curl_close($curl);
}
return "SMS Send Successfully";
}
in you php.ini set execution timeout in second as your need
max_execution_time = 60
if you want to define it in php file. Add this line
ini_set('max_execution_time', 60);
I want to process a script every minute using a cron on my server but I need to pass a variable in the URL or some other way. I have researched this and I've seen solutions using arguments in the cron but I don't think that works with what I'm doing.
Here is what I am trying to do:
script.php (runs every minute)
<?php
$marker = $_GET['marker'];
$accountObj = new etAccounts($consumer);
$request_params = new TransactionHistoryRequest();
$request_params->__set('count', 50); //how many will be shown
if($marker_get != ''){
$request_params->__set('marker', $marker_get); //starting point ex. 14293200140265
}
$json = $accountObj->GetTransactionHistory($account, $request_obj, $request_params );
echo $json; //shows most recent 50 transactions starting from marker value
//process json data here...
//included in json is a marker variable that will be used to return the next 50 json results
//after data is processed reload the page with marker in URL
header('Location: script.php?marker=14293200140265');
?>
I understand that cron is CLI on the server side and that it can't process redirections or header locations but how is this possible. I saw someone mention using CURL, how might this work? Example?
Simple example to send post variables to an url:
$fields = array(
'id' => $id,
'mail' => $mail,
);
$url = "yourdomain.com";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
Based on the comments, you don't need to redirect or use query variables.
You could use a loop that runs while your marker variable is not empty:
$your_marker_variable = '';
do {
$accountObj = new etAccounts($consumer);
$request_params = new TransactionHistoryRequest();
$request_params->__set('count', 50); //how many will be shown
if($marker_get != '') {
$request_params->__set('marker', $marker_get); //starting point ex. 14293200140265
}
$json = $accountObj->GetTransactionHistory($account, $request_obj, $request_params );
echo $json; //shows most recent 50 transactions starting from marker value
//process json data here...
//included in json is a marker variable that will be used to return the next 50 json results
// set the new value of $your_marker_variable
$your_marker_variable = ...
} while (!empty($your_marker_variable));
Note that the undefined and unused variables in the script you posted, make it hard to see what variable is used for what, so you would need to adapt this a bit.
I want to convert given postcode to latitude and longitude to integrate in my cart project.
But when I try to grab latitude and longitude with google api they are showing some error like,
"We're sorry... ... but your computer or network may be sending
automated queries. To protect our users, we can't process your request
right now."
What is wrong with my code? My code is shown below.
function getLatLong($code){
$mapsApiKey = 'AIzaSyC1Ky_5LFNl2zq_Ot2Qgf1VJJTgybluYKo';
$query = "http://maps.google.co.uk/maps/geo?q=".urlencode($code)."&output=json&key=".$mapsApiKey;
//---------
// create a new cURL resource
$ch = curl_init();
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, $query);
curl_setopt($ch, CURLOPT_HEADER, 0);
// grab URL and pass it to the browser
$data = curl_exec($ch);
// close cURL resource, and free up system resources
curl_close($ch);
//-----------
//$data = file_get_contents($query);
// if data returned
if($data){
// convert into readable format
$data = json_decode($data);
$long = $data->Placemark[0]->Point->coordinates[0];
$lat = $data->Placemark[0]->Point->coordinates[1];
return array('Latitude'=>$lat,'Longitude'=>$long);
}else{
return false;
}
}
print_r(getLatLong('SW1W 9TQ'));
Use useragent
curl_setopt($ch, CURLOPT_USERAGENT,'Mozilla/5.0');
Also check whether you missed or not to send any HTTP Request header. Also check whether you are using required parameters(GET or POST) with your request.
By the way, If you are using too many requests then you have nothing to do with this error. Just stop sending requests, or limit your requests so that it doesn't upset the server.
I've done a simple twitter feed for a webpage. I save a cached file with the JSON response of the API, and then read it using jQuery.
It works well, the problem is that it randomly reaches the REST API limit of 150 requests, while I'm doing just 6 per hour (1 each 10 minutes), and as far as I remember, I don't have any other feed in my hosting (MediaTemple gs) that could've been doing many requests per hour.
I know that I can auth with an account and get 350 requests limit, I haven't tested yet, but I don't think this will solve the problem at all.
Here is the cron I execute each 10 minutes:
<?php
//Set local timezone
putenv("TZ=America/Caracas");
//Function to get contents using cURL
function url_get_contents ($Url) {
if (!function_exists('curl_init')){
die('CURL is not installed!');
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $Url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
//The path of the file that contains cached tweets
$cache = '/home/xxxxx/domains/example.com/html/demos/webname/twitter-json.txt';
//Call the api and get JSON response
$data = url_get_contents("https://api.twitter.com/1/statuses/user_timeline.json?include_entities=true&include_rts=true&screen_name=webname&count=100");
$parsed = json_decode($data, true);
//This is a workaround I made to exit the script if there's an error(it works)
if ($parsed['error']) exit;
//I change the twitter date format to d/m/y H:i
foreach ($parsed as $key => $value) {
$parsed[$key]['created_at'] = date('d/m/y H:i',strtotime($value['created_at']));
}
//I encode to JSON
$data = json_encode($parsed);
//Save the file
$cachefile = fopen($cache, 'wb');
fwrite($cachefile,$data);
fclose($cachefile);
?>
I have an array containing the contents of a MySQL table. I need to put each of these contents into curl_multi_handles so that I can execute them all simultaneously
Here is the code for the array, in case it helps:
$SQL = mysql_query("SELECT url FROM urls") or die(mysql_error());
while($resultSet = mysql_fetch_array($SQL)){
$urls[]=$resultSet
}
So I need to put be able to send data to each url at the same time. I don't need to get any data back, and in fact I'll be having them time out after two seconds. It only needs to send the data and then close.
My code prior to this, was executing them one at a time. here is that code:
$SQL = mysql_query("SELECT url FROM shells") or die(mysql_error()); while($resultSet = mysql_fetch_array($SQL)){
$ch = curl_init($resultSet['url'] . $fullcurl); //load the urls and send GET data
curl_setopt($ch, CURLOPT_TIMEOUT, 2); //Only load it for two seconds (Long enough to send the data)
curl_exec($ch);
curl_close($ch);
So my question is: How can I load the contents of the array into curl_multi_handle, execute it, and then remove each handle and close the curl_multi_handle?
You still call curl_init and curl_setopt. Then you load it into a multi_handle, and keep calling execute until it's done. This is based on the documentation at curl_multi_init. Since you're timing out in two seconds, and not processing responses, I think you can just sleep for two seconds at a time. curl_multi_select might be better if you actually need to process the responses.
$SQL = mysql_query("SELECT url FROM shells") ;
$mh = curl_multi_init();
$handles = array();
while($resultSet = mysql_fetch_array($SQL)){
//load the urls and send GET data
$ch = curl_init($resultSet['url'] . $fullcurl);
//Only load it for two seconds (Long enough to send the data)
curl_setopt($ch, CURLOPT_TIMEOUT, 2);
curl_multi_add_handle($mh, $ch);
$handles[] = $ch;
}
// Create a status variable so we know when exec is done.
$running = null;
//execute the handles
do {
// Call exec. This call is non-blocking, meaning it works in the background.
curl_multi_exec($mh,$running);
// Sleep while it's executing. You could do other work here, if you have any.
sleep(2);
// Keep going until it's done.
} while ($running > 0);
// For loop to remove (close) the regular handles.
foreach($handles as $ch)
{
// Remove the current array handle.
curl_multi_remove_handle($mh, $ch);
}
// Close the multi handle
curl_multi_close($mh);
If i were you, i would write class mysql and a class curl.
Its very good at all.
First i would create a method witch would return all urls from a passed mysql result.
Something like
public function getUrls($mysql_fetch_array)
{
foreach($mysql_fetch_array as $result)
{
$urls[] = $result["url"];
}
}
then you could write a method like curlSend($url,$param)
//remember you have to edit i dont know your full code so its just
// a way you could do it
public function curlSend($url,$param="")
{
$ch = curl_init($resultSet['url'] . $fullcurl); //load the urls and send GET data
curl_setopt($ch, CURLOPT_TIMEOUT, 2); //Only load it for two seconds (Long enough to send the data)
curl_exec($ch);
curl_close($ch);
}
public function send()
{
$urls = getUrls($this->mysql->result($sql));
foreach($urls as $url)
{
$this->curlSend($url);
}
}
Now this is how you could do it.