Slow response rate from Curl / Ajax - php

General Info
Hi guys,
I was deciding to post this or not due to the question but i said hell, why not!
What am i doing?
I have an array of music tracks which is pushed into a ajax request which i then loop through in php and pass each track into a curl request to then go off and pull down coverart for them specific tracks! The coverart of course is in .jpg or .png format of a link, so rihanna.jpg is returned. I then push them into another array which is my end result which is then pushed back to my ajax result (data) and i go from there.
My issue?
When i push in 60 tracks, the response is around 30 seconds which is a long ass wait! I am trying to retrieve information as quick as possible so i can update the end-users GUI without leaving them with a loading bar for periods of time while the data loads.
Now i looked into this and my theory behind why its so slow is either down to the api call is lacking in the response department or the curl is hogging and slowing it all down! I heard of curl_multi and all those but have never used nor do i no were to start! I am not even sure if that will solve this. Can someone please input on this to try ease my mind? Is this speed all i will ever get or is it possible to speed this up? Before i was creating ajax calls within a loop and i thought it was because of that so i did a change to push all the tracks into one array which is then passed into one ajax but it did not speed anything up.. Same response rate to be honest!
My PHP code:
function curlGet($url, $proxy = null)
{
$ch = curl_init($url);
#curl_setopt($curl, CURLOPT_HTTPHEADER, array("Connection: close"));
#curl_setopt($ch, CURLOPT_NOSIGNAL, 1);
#curl_setopt($curl, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);
if (isset($proxy)) {
if ($proxy['enable'] === '1') {
$proxy['user'] === '' || #curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxy['user'].':'.$proxy['pass']);
#curl_setopt($ch, CURLOPT_PROXY, $proxy['host']);
}
}
#curl_setopt($ch, CURLOPT_CONNECTTIMEOUT_MS, 400);
#curl_setopt($ch, CURLOPT_TIMEOUT, 10);
#curl_setopt($ch, CURLOPT_HEADER, 0);
#curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
curl_close($ch);
return $response;
}
$trackarray = $_POST['trackarray'];
$returnarray = [];
foreach($trackarray as $i => $track) {
$strippedarray = explode(" - ", $track, 2);
$strippedartist = $strippedarray[0];
$strippedtrack = $strippedarray[1];
$ArtistTrack = "http://ws.audioscrobbler.com/2.0/?method=track.getInfo&api_key=KEYHERE&artist=".$strippedartist."&track=".$strippedtrack."&format=json";
$ArtistL = "http://ws.audioscrobbler.com/2.0/?method=artist.getInfo&api_key=KEYHERE&artist=".$strippedartist."&format=json";
$AToutput = json_decode(curlGet($ArtistTrack, $proxy), true);
$Aoutput = json_decode(curlGet($ArtistL, $proxy), true);
if($AToutput == "" || $AToutput['track']['album']['image']['3']['#text'] == "") //Artist-Track failed, lets try just arist!
{
if($Aoutput == "" || $Aoutput['artist']['image']['3']['#text'] == "") //If Artist Failed, serve default-coverart.png
{
array_push($returnarray, "../location/cover-default.png");
}
else //Artist success, serve artist.png
{
array_push($returnarray, $Aoutput['artist']['image']['3']['#text']);
}
}
else //Artist-Track Success, Serve artist-track.png
{
array_push($returnarray, $AToutput['track']['album']['image']['3']['#text']);
}
}
echo json_encode($returnarray);
My Javascript:
$('#spinner-db').removeClass('hide');
var windowwidth = $(window).width();
var blockwidth = 350;
var rowcount = parseInt(windowwidth / blockwidth);
var buffersize = parseInt(rowcount * 4);
var endindex = parseInt(buffersize * 2);
var loadimagearray = localcontent.slice(0,endindex);
var currentdatabase = "#database-entries";
$.ajax({
url : '/location/script.php?cmd=commandhere',
cache: false,
type: 'POST',
data: {trackarray: loadimagearray}
}).done(function(data)
{
$.each( loadimagearray, function( i, l ) //Loop through each item in array
{
if($(currentdatabase+' li[track-info="' + l + '"] img').attr('src') == "../assets/img/cover-default.png")
{
$(currentdatabase+' li[track-info="' + l + '"] img').attr('src',JSON.parse(data)[i]);
if(i == (loadimagearray.length - 1)) //Hide Loader
{
$('#spinner-db').addClass('hide');
}
}
});
});

Related

file_get_contents get data after 5 second from loading the web-page

i want to check if source web-page has certain word or not in PHP , but the web-page load after 5 seconds
tried classic way but didn't worked because it load the page immediately
<?php
$urlmain = "http://example.com";
$url = file_get_contents("$urlmain");
if (strpos($url, 'buy') !== false) {
$pid= 'Available';
}elseif (strpos($url, 'sold') !== false) {
$pid= 'Sold';
}else{
$pid= 'can't get data';
}
echo $pid;
?>
in previous code i want file_get_contents to get data after 5 second from loading the web-page
$url = file_get_contents("$url");
Any Idea?
If you need to load page before requesting data, you wouldn't be able to do it in 1 request.
Your best bet would be to load page normally (without any file_get_contents), wait for 5 seconds, send request via JS to a PHP script that would actually do the file_get_contents. Note that your code should end with die();, otherwise your second request will get whole page on top of your results.
Try following:
<?php
if ($_SERVER['REQUEST_METHOD'] == 'POST') {
// This is your code to get data
$url = file_get_contents("$url");
if (strpos($url, 'buy') !== false) {
$pid = 'Available';
} elseif (strpos($url, 'sold') !== false) {
$pid = 'Sold';
} else {
$pid = 'can\'t get data';
}
echo $pid;
die();
}
?>
<div id="output"></div>
<script>
// On page load we start counting for 5 seconds, after which we execute function
window.onload = setTimeout(function (){
// We prepare AJAX request to the same PHP script, but via POST method
var http = new XMLHttpRequest();
var url = '';
http.open('POST', url, true);
http.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');
// When we get back successful results, we will output them to HTML node with ID = output
http.onreadystatechange = function() {
if(http.readyState === 4 && http.status === 200) {
document.getElementById('output').innerHTML = http.responseText;
}
}
http.send();
}, 5000);
</script>
You should use PHP cURL extension:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
$response = curl_exec($ch);
curl_close($ch);

Get alert response from external site

I'm trying ot figure out away to get a response from this website:
https://www.primarycare.nhs.uk/publicfn/catchment.aspx?oc=P91012&h=600&w=800&if=0
oc=P91012 - this is the practice identifier.
We then have a postcode field where a user checks if they are in the catchment area
The site issues a javascript alert to inform the user if they are / or not in the catchment area.
Is it possible to send a PHP Curl request that sends a postcode to this website and checks whether or not the postcode is in the catchment area and send a reponse back to my site?
<?php
$POSTurl = 'https://www.primarycare.nhs.uk/publicfn/geocode.ashx';
$fields = array(
'postcode' => urlencode('M41 0UX'),
);
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string, '&');
$ch = curl_init();
curl_setopt($ch,CURLOPT_URL, $POSTurl );
curl_setopt($ch,CURLOPT_POST, count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS, $fields_string);
$result = curl_exec($ch);
$url = 'https://www.primarycare.nhs.uk/publicfn/catchment.aspx?oc=P91012&h=600&w=800&if=0';
$ch = curl_init();
$result = curl_exec($ch);
curl_close($ch);
?>
So, here it is:
the problem, why you were not unable to pick proper response is because all valid coordinates was in source code of NHS. I stripped out unwanted code, and checked coordinates by postcode with existing coordinates from the list. Keep in mind, that if anything will be changed in the list in original source, you will need to update that also.
Also, this code, technically is not 100% correct and probably you will spent more time to make it more correct for your needs. As i said in the comment, this will work as little hack.
Just copy/paste into your php file, and refresh the page.
Code:
<?php
$postcode = 'M41 0UX';
$postcode = urlencode($postcode);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://www.primarycare.nhs.uk/publicfn/geocode.ashx");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "PostCode=".$postcode);
curl_setopt($ch, CURLOPT_POST, 1);
$result = curl_exec($ch);
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
else {
echo $result;
echo '<input id="checker" type="hidden" value="'.$result.'" />';
?>
<script src="http://code.jquery.com/jquery-latest.min.js" type="text/javascript"></script>
<script src="https://www.primarycare.nhs.uk/Private/js/gpos.js" type="text/javascript"></script>
<script type="text/javascript" src="https://maps-api-ssl.google.com/maps/api/js?v=3.7&sensor=false"></script>
<script type="text/javascript">
var point;
var poly1=[{x:53.45801,y: -2.33768},{x:53.46046,y: -2.32996},{x:53.45882,y: -2.32447},{x:53.45637,y: -2.3188},{x:53.45351,y: -2.31709},{x:53.44932,y: -2.31468},{x:53.44523,y: -2.31365},{x:53.44216,y: -2.31279},{x:53.43961,y: -2.31417},{x:53.43858,y: -2.31657},{x:53.4394,y: -2.32155},{x:53.44001,y: -2.32893},{x:53.44145,y: -2.33408},{x:53.44329,y: -2.33751},{x:53.44053,y: -2.33648},{x:53.4395,y: -2.34095},{x:53.43838,y: -2.34352},{x:53.44083,y: -2.34627},{x:53.44022,y: -2.35107},{x:53.43899,y: -2.35193},{x:53.43787,y: -2.35107},{x:53.43797,y: -2.35399},{x:53.43715,y: -2.35622},{x:53.43562,y: -2.3576},{x:53.43592,y: -2.35949},{x:53.43449,y: -2.36103},{x:53.43275,y: -2.36017},{x:53.43153,y: -2.36137},{x:53.43275,y: -2.36343},{x:53.43183,y: -2.36515},{x:53.43347,y: -2.36738},{x:53.4349,y: -2.36704},{x:53.43664,y: -2.36738},{x:53.43736,y: -2.3679},{x:53.43705,y: -2.37167},{x:53.43725,y: -2.37459},{x:53.43592,y: -2.37511},{x:53.43664,y: -2.377},{x:53.43623,y: -2.37974},{x:53.43787,y: -2.38077},{x:53.43858,y: -2.37888},{x:53.4395,y: -2.3782},{x:53.43981,y: -2.38197},{x:53.4392,y: -2.38403},{x:53.43797,y: -2.38352},{x:53.43848,y: -2.38609},{x:53.4395,y: -2.38592},{x:53.44073,y: -2.3842},{x:53.44063,y: -2.38747},{x:53.44001,y: -2.3909},{x:53.43776,y: -2.39193},{x:53.43603,y: -2.39141},{x:53.4348,y: -2.38884},{x:53.43255,y: -2.39038},{x:53.43388,y: -2.39553},{x:53.43459,y: -2.39914},{x:53.43684,y: -2.40051},{x:53.43766,y: -2.40223},{x:53.4394,y: -2.4012},{x:53.44012,y: -2.40429},{x:53.4394,y: -2.40824},{x:53.43868,y: -2.41219},{x:53.44533,y: -2.40807},{x:53.4485,y: -2.40669},{x:53.4529,y: -2.40309},{x:53.45545,y: -2.39965},{x:53.45995,y: -2.39313},{x:53.46363,y: -2.38438},{x:53.46608,y: -2.37614},{x:53.46894,y: -2.3703},{x:53.46588,y: -2.36532},{x:53.46455,y: -2.35931},{x:53.46281,y: -2.35296},{x:53.46158,y: -2.3461},{x:53.45995,y: -2.34198},{x:53.4577,y: -2.33974}];
function isPointInPoly(poly, pt){
for(var c = false, i = -1, l = poly.length, j = l - 1; ++i < l; j = i)
((poly[i].y <= pt.y && pt.y < poly[j].y) || (poly[j].y <= pt.y && pt.y < poly[i].y))
&& (pt.x < (poly[j].x - poly[i].x) * (pt.y - poly[i].y) / (poly[j].y - poly[i].y) + poly[i].x)
&& (c = !c);
return c;
};
function ispostcodein(postcode) {
var ok = 1;
if (ok == 1) {
var responseText = $('#checker').val();
point = responseText.split(';');
}
var incatch = 0;
if (isPointInPoly(poly1, {x: point[0],y: point[1]})) { incatch = 1 };
if (incatch==0) {
alert("We're sorry, but your postcode is outside the catchment area for PRIMROSE AVENUE SURGERY.");
}
else {
alert("Your postcode is in the catchment area for PRIMROSE AVENUE SURGERY. You are able to register with this practice.");
}
}
ispostcodein('<?= $result; ?>');
</script>
<?php
}
curl_close($ch);
?>

Google Script Error on Post

I am posting data to google sheets using php, everytime I post the data it get added on the sheet correctly, but i get the error message as
We8d#39;re sorry, a server error occurred. Please wait a bit and try
again.
My php curl post code is:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $gsurl);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
$res = json_decode($output, 1);
My GS App Script is :
var SCRIPT_PROP = PropertiesService.getScriptProperties(); // new property service
function doGet(e){
return handleResponse(e);
}
function doPost(e){
return handleResponse(e);
}
function handleResponse(e) {
var lock = LockService.getPublicLock();
lock.waitLock(30000);
try {
var doc = SpreadsheetApp.openById(SCRIPT_PROP.getProperty("key"));
var sheet = doc.getSheetByName(SHEET_NAME);
var headRow = e.parameter.header_row || 1;
var headers = sheet.getRange(1, 1, 1, sheet.getLastColumn()).getValues()[0];
var nextRow = sheet.getLastRow()+1;
var row = [];
for (i in headers){
if (headers[i] == "Timestamp"){
row.push(new Date());
} else {
row.push(e.parameter[headers[i]]);
}
}
sheet.getRange(nextRow, 1, 1, row.length).setValues([row]);
// return json success results
return ContentService
.createTextOutput(JSON.stringify({"result":"success", "row": nextRow}))
.setMimeType(ContentService.MimeType.JSON);
} catch(e){
return ContentService
.createTextOutput(JSON.stringify({"result":"error", "error": e}))
.setMimeType(ContentService.MimeType.JSON);
} finally {
lock.releaseLock();
}
}
function setup() {
var doc = SpreadsheetApp.getActiveSpreadsheet();
SCRIPT_PROP.setProperty("key", doc.getId());
}
Please help me.

determine and sort distance for several locations

I have about 15 locations in a mysql table with lat and long information.
Using PHP and google maps API Am able to calculate distance between 2 locations.
function GetDrivingDistance($lat1, $lat2, $long1, $long2)
{
$url = "https://maps.googleapis.com/maps/api/distancematrix/json?origins=".$lat1.",".$long1."&destinations=".$lat2.",".$long2."&mode=driving&language=en-US";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXYPORT, 3128);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
$response = curl_exec($ch);
curl_close($ch);
$response_a = json_decode($response, true);
$dist = $response_a['rows'][0]['elements'][0]['distance']['text'];
$time = $response_a['rows'][0]['elements'][0]['duration']['text'];
return array('distance' => $dist, 'time' => $time);
}
I want to to select one as fixed e.g. row 1 given lat and long
$query="SELECT lat, long from table WHERE location=1"
$locationStart = $conn->query($query); =
I want to calculate the distance to all other locations in the tables (other rows) and return the the outcome sorted by distance
tried to calculate each one alone and end up with very long code and takes too long to fetch that via api, also still not able to sort them this way!
any hint?
Disclaimer: This is not a working solution, nor have I tested it, it is just a quick example I've done off the top of my head to provide a sort of code sample to go with my comment.
My brains still not fully warmed up, but I believe the bottom should at least act as a sort of guide to help put across the idea I was making in my comment, i'll try to answer any questions you have when I'm free. Hope it helps.
<?php
define('MAXIMUM_REQUEST_STORE', 5); // Store 5 requests in each multi_curl_handle
function getCurlInstance($url) {
$handle = curl_init();
curl_setopt($handle, CURLOPT_URL, $url);
curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);
return $handle;
}
$data = []; // Build up an array of Endpoints you want to hit. I'll let you do that.
// Initialise Variables
$totalRequests = count($data);
$parallelCurlRequests = [];
$handlerID = 0;
// Set up our first handler
$parallelCurlRequests[$handlerID] = curl_multi_init();
// Loop through each of our curl handles
for ($i = 0; $i < $totalRequests; ++$i) {
// We want to create a new handler/store every 5 requests. -- Goes off the constant MAXIMUM_REQUEST_STORE
if ($i % MAXIMUM_REQUEST_STORE == 1 && $i > MAXIMUM_REQUEST_STORE) {
++$handlerID;
}
// Create a Curl Handle for the current endpoint
// ... and store the it in an array for later use.
$curl[$i] = getCurlInstance($data[$i]);
// Add the Curl Handle to the Multi-Curl-Handle
curl_multi_add_handle($parallelCurlRequests[$handlerID], $curl[$i]);
}
// Run each Curl-Multi-Handler in turn
foreach ($parallelCurlRequests as $request) {
$running = null;
do {
curl_multi_exec($request, $running);
} while ($running);
}
$distanceArray = [];
// You can now pull out the data from the request.
foreach ($curl as $response) {
$content = curl_multi_getcontent($response);
if (!empty($content)) {
// Build up some form of array.
$response = json_decode($content);
$location = $content->someObject[0]->someRow->location;
$distance = $content->someObject[0]->someRow->distance;
$distanceArray[$location] = $distance;
}
}
natsort($distanceArray);

Instagram async feed

Following this post How to get a user's Instagram feed , i use it to display the last image
function fetchData($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
$result = fetchData("https://api.instagram.com/v1/users/123456789/media/recent/?access_token=123456789.123asdsd.asdadasdas23423423&count=1");
$result = json_decode($result);
foreach ($result->data as $post) {
if(empty($post->caption->text)) {
// Do Nothing
}
else {
// Display img
}
}
How can be loaded asynchronous? Sometimes is takes even 2-3s to load and delays the entire page to be displayed. Tks for you time and help!
EDIT
tks to #steve, i solved it by query instagram api once per hour and save the response to instagram.json
get-social-media.php
function get_instagram($user_id=instagram_user_id,$count=1){
$instaurl = `https://api.instagram.com/v1/users/`.$user_id.`/media/recent/?access_token=instagram_access_token&count=`.$count;
$instacache = `instagram.json`;
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_URL,$instaurl);
$instadata=curl_exec($ch);
curl_close($ch);
if(file_exists($instacache) && filemtime($instacache) > time() - 60*30){
//echo "ok instagram";
} else {
$jsonInstaData = json_decode($instadata,true);
file_put_contents($instacache,json_encode($jsonInstaData));
}
}
echo get_instagram();
and that ajax for frontend social-media-block.phtml (magento & bootstrap)
jQuery(document).ready(function($) {
$("#instagram-img").html("");
$.ajax({
type: "GET",
async: true,
contentType: "application/json; charset=utf-8",
url:"resources/socialmedia-cache/instagram.json",
dataType: "json",
cache: true,
beforeSend: function () {
$("#loading").show();
},
success: function (data) {
console.log(data);
$("#loading").hide();
if (data == "") {
$("#InstaContainer").hide();
} else {
$("#InstaContainer").show();
for (var i = 0; i < data["data"].length; i++) {
var dataForJson = JSON.stringify(data.data[i]);
var date = new Date(parseInt(data.data[i].caption.created_time) * 1000);
$("#instagram-img").append("<a target=`_blank` href=`" + data.data[i].link + "` title=`" + data.data[i].caption.text + "`><img src=`" + data.data[i].images.low_resolution.url + "` class=`img-responsive socialmedia-img`></img></a>");
$("#instagram-img").append("<p align=`left`><script>" + "jQuery(document).ready(function() { jQuery(`a.timeago`).timeago();});" + "</" + "script><a class=`timeago` style=`color:#484848;` title=`" +(date.getMonth()+1)+"/"+date.getDate()+"/"+date.getFullYear()+", "+date.getHours()+":"+date.getMinutes()+ "`>" +(date.getMonth()+1)+"/"+date.getDate()+"/"+date.getFullYear()+", "+date.getHours()+":"+date.getMinutes()+ "</a></p>");
}
}
}
});
});
this also works for facebook
for pinterest, i use http://ajax.googleapis.com/ajax/services/feed/load?v=1.0&num=1&q=https://www.pinterest.com/MyPinterest/feed.rss , a quick solution to convert the rss to json. since i need images larger than 236px, next parsed is 736px. also, the img src needs to be extracted from content
var string = data.responseData.feed.entries[i].content;
var filtered = string.replace('/236x/', '/736x/');
var source = filtered.match(/src\s*=\s*"(.+?)"/);
probably not the best code, but at least is a working solution.

Categories