Cache data server side for ajax request - php

I'm using TwitterAPIExchange and Instagram-PHP-API to request data from both APIs on my server, concatenate them into an array and build this into JSON which I can consume on the client with a simple GET request in jQuery.
This is all working fine, however it is quite slow. When I make a request to static JSON data as a test, it is far quicker. It seems the php script on my server to get this data is running each time (so starting with an empty array, using the wrappers the libraries provide to make requests etc.). What I would like to do is to cache/store this data, and only make another request once or twice a day.
My PHP knowledge is not the best so I'm sure there is a relatively simple way of doing this.
Here is the php script on my server:
<?php
header('Content-type:application/json;charset=utf-8');
require_once('TwitterAPIExchange.php');
require_once('Instagram.php');
unset($allFeeds);
$allFeeds = array();
$settings = array(
'oauth_access_token' => "XXXX' => "XXXX",
'consumer_key' => "XXXX",
'consumer_secret' => "XXXX"
);
$url = 'https://api.twitter.com/1.1/statuses/user_timeline.json';
$getfield = '?screen_name=XXX';
$requestMethod = 'GET';
$twitter = new TwitterAPIExchange($settings);
$twitter_response = $twitter->setGetfield($getfield)->buildOauth($url,$requestMethod)->performRequest();
$jsonTweets = json_decode($twitter_response);
foreach ($jsonTweets as $tweet) {
$tweetObj = new stdClass();
$tweetObj->type = 'Twitter';
$tweetObj->created_at = strtotime($tweet->created_at);
$tweetObj->text = $tweet->text;
$allFeeds[] = $tweetObj;
}
use MetzWeb\Instagram\Instagram;
$instagram = new Instagram(array(
'apiKey' => 'XXXX',
'apiSecret' => 'XXXX',
'apiCallback' => 'XXXX'
));
$token = 'XXX';
$code = $token;
$instagram->setAccessToken($code);
$id_one = 'XXX';
$id_two = 'XXX';
$insta_response_one = $instagram->getUserMedia($id_one, 20);
$insta_response_two = $instagram->getUserMedia($id_two, 10);
foreach ($insta_response_one->data as $insta_one) {
$instaObjOne = new stdClass();
$instaObjOne->type = 'Instagram One';
$instaObjOne->created_at = (int)$insta_one->created_time;
if (isset($insta_one->caption->text)) {
$instaObjOne->text = $insta_one->caption->text;
}
$instaObjOne->img = $insta_one->images->standard_resolution->url;
$allFeeds[] = $instaObjOne;
}
foreach ($insta_response_two->data as $insta_two) {
$instaObjTwo = new stdClass();
$instaObjTwo->type = 'Instagram Two';
$instaObjTwo->created_at = (int)$insta_two->created_time;
if (isset($insta_two->caption->text)) {
$instaObjTwo->text = $insta_two->caption->text;
}
$instaObjTwo->img = $insta_two->images->standard_resolution->url;
$allFeeds[] = $instaObjTwo;
}
function dateSort($a, $b) {
return $b->created_at - $a->created_at;
}
usort($allFeeds, "dateSort");
$data = json_encode($allFeeds);
// cache $data here?
echo $data;
?>
I bringing it into my frontend like so:
$.ajax({
type: 'GET',
url: 'path/to/script.php'
dataType: 'json',
}).done(function(data) {
// do stuff with data here
});
}
Hope that makes sense, thanks

This is the simplest way to add caching to your code.
It's using file_put_contents('cache.txt',$data) to cache the request data and then at the top of the file it checks the current time and compares with the last time the file was modified. If it was modified less than 24 hours ago, it just outputs the content of the cache file using echo file_get_contents('cache.txt); and stops the script.
<?php
header('Content-type:application/json;charset=utf-8');
// Check if file was modified less than 24 hours ago
if ((time() - filemtime('cache.txt')) < 24 * 60 * 60) {
// Output contents of cache file and stop script
echo file_get_contents('cache.txt');
exit();
}
require_once('TwitterAPIExchange.php');
require_once('Instagram.php');
unset($allFeeds);
$allFeeds = array();
$settings = array(
'oauth_access_token' => "XXXX" => "XXXX",
'consumer_key' => "XXXX",
'consumer_secret' => "XXXX"
);
$url = 'https://api.twitter.com/1.1/statuses/user_timeline.json';
$getfield = '?screen_name=XXX';
$requestMethod = 'GET';
$twitter = new TwitterAPIExchange($settings);
$twitter_response = $twitter->setGetfield($getfield)->buildOauth($url,$requestMethod)->performRequest();
$jsonTweets = json_decode($twitter_response);
foreach ($jsonTweets as $tweet) {
$tweetObj = new stdClass();
$tweetObj->type = 'Twitter';
$tweetObj->created_at = strtotime($tweet->created_at);
$tweetObj->text = $tweet->text;
$allFeeds[] = $tweetObj;
}
use MetzWeb\Instagram\Instagram;
$instagram = new Instagram(array(
'apiKey' => 'XXXX',
'apiSecret' => 'XXXX',
'apiCallback' => 'XXXX'
));
$token = 'XXX';
$code = $token;
$instagram->setAccessToken($code);
$id_one = 'XXX';
$id_two = 'XXX';
$insta_response_one = $instagram->getUserMedia($id_one, 20);
$insta_response_two = $instagram->getUserMedia($id_two, 10);
foreach ($insta_response_one->data as $insta_one) {
$instaObjOne = new stdClass();
$instaObjOne->type = 'Instagram One';
$instaObjOne->created_at = (int)$insta_one->created_time;
if (isset($insta_one->caption->text)) {
$instaObjOne->text = $insta_one->caption->text;
}
$instaObjOne->img = $insta_one->images->standard_resolution->url;
$allFeeds[] = $instaObjOne;
}
foreach ($insta_response_two->data as $insta_two) {
$instaObjTwo = new stdClass();
$instaObjTwo->type = 'Instagram Two';
$instaObjTwo->created_at = (int)$insta_two->created_time;
if (isset($insta_two->caption->text)) {
$instaObjTwo->text = $insta_two->caption->text;
}
$instaObjTwo->img = $insta_two->images->standard_resolution->url;
$allFeeds[] = $instaObjTwo;
}
function dateSort($a, $b) {
return $b->created_at - $a->created_at;
}
usort($allFeeds, "dateSort");
$data = json_encode($allFeeds);
// Cache $data here
file_put_contents('cache.txt', $data);
echo $data;
?>

Related

how to use dreamscape api in nodejs for check domain name

I have already work this in php. Here is my working code:
$request = array(
'DomainNames' => $domain_names
);
$response = dreamScapeAPI('DomainCheck', $request);
$available = false;
$alt_domains = array(); // Alternative to user's expected domain names
if (!is_soap_fault($response)) {
// Successfully checked the availability of the domains
if (isset($response->APIResponse->AvailabilityList)) {
$availabilityList = $response->APIResponse->AvailabilityList;
foreach ($availabilityList as $list) {
if ($list->Available){
if ($domain == $list->Item) {
$available = true; // user prefered domain found
}
else {
$alt_domains[] = $list->Item;
}
}
}
}
else {
$error = $response->APIResponse->Errors;
foreach ($error as $e) {
$api_error = $e->Message;
//echo $e->Item . ' - ' . $e->Message . '<br />';
}
}
}
function dreamScapeAPI($method, $data = null) {
$reseller_api_soap_client = "";
$soap_location = 'http://soap.secureapi.com.au/API-2.1';
$wsdl_location = 'http://soap.secureapi.com.au/wsdl/API-2.1.wsdl';
$authenticate = array();
$authenticate['AuthenticateRequest'] = array();
$authenticate['AuthenticateRequest']['ResellerID'] = '**';
$authenticate['AuthenticateRequest']['APIKey'] = '**';
//convert $authenticate to a soap variable
$authenticate['AuthenticateRequest'] = new SoapVar($authenticate['AuthenticateRequest'], SOAP_ENC_OBJECT);
$authenticate = new SoapVar($authenticate, SOAP_ENC_OBJECT);
$header = new SoapHeader($soap_location, 'Authenticate', $authenticate, false);
$reseller_api_soap_client = new SoapClient($wsdl_location, array('soap_version' => SOAP_1_2, 'cache_wsdl' => WSDL_CACHE_NONE));
$reseller_api_soap_client->__setSoapHeaders(array($header));
$prepared_data = $data != null ? array($data) : array();
try {
$response = $reseller_api_soap_client->__soapCall($method, $prepared_data);
} catch (SoapFault $response) { }
return $response;
}
I tried with this : https://github.com/dan-power/node-dreamscape
But domain search can not work properly. Mainly, I want it on nodejs using nodejs dreamscape api but this method is not available on nodejs.
Here is my working demo: click here

504 Gateway time out php

i'm running script(svc/rest services) that request a server which response me a XML string.. this request is going through an intermediate server.
My aim is to fetch the details of guests o.e. rooms which will be response..
but after certain number of request i am getting 504 gateway out and only about 40 rooms are able to retreived
1: file...source hots
$ROOMS = ["301","302","304","305","306","307","308","309","310",
"311","312","314","315","316","317","318","319","320",
"401","402","403","404","405","406","407","408","409",
"410","411","412","414","415","416","802","802","802",
"802","802","802","802","802","802","802","802","802",
"802","802","802","802","802","802","802","802","802",
"802","802","802","802","802","802","802","802"
];
// all rooms more than 75
for ($x = 0; $x < count($ROOMS); $x++){
$end_point = "https://www.innkeyapp.com/SerRest.svc/GetOccupiedRoomData";
$sharedkey = "2diSq8bjZ4N3ZR4XB5KXXXXXXXXXXXXXXXXXXXXXXX";
$url = 'https://eastern-services.com/innkeyPMS_room_ststus.php';
$room_no = $ROOMS[$x];
$postdata = http_build_query([
'end_point' => $end_point,
'room_no' => $room_no,
'sharedkey' => $sharedkey
]);
$opts = [
'http' => [
'method' => 'POST',
'content' => $postdata
]
];
$context = stream_context_create($opts);
$result = file_get_contents($url, false, $context);
$arrayData = json_decode($result,true);
if (empty($arrayData)) {
// Redirect('failure.php?msg=Error in service');
echo "Error in Service";
exit();
}
print_r($arrayData);
echo $room_no."Added";
}
2: file.... innkeyPMS_room_ststus.php i.e. intermediate host.
$room_no = $post_var['room_no'];
$end_point = $post_var['end_point'];
$sharedkey = $post_var['sharedkey'];
$registerno = '';
$prprtxt = '';
$validunit = '';
// echo $post_var;
function curl_get_file_contents($URL)
{
return file_get_contents($URL);
}
$xmlString = curl_get_file_contents(
$end_point.'sharedkey='.$sharedkey.'=&roomno='.$room_no
);
$xml = simplexml_load_string($xmlString);
header("Content-Type: application/json");
echo json_encode($xml);
why this connection is getting lost or why its showing 504 gateway time out?

502 Bad Gateway Foreach loop

hope for some help, basically i have a script that gets the latest posts from facebook users, and basically i check if there is any new post that is not available on my database, in case that this post is new, than i save it in my database along with the post id (this way i check if exist on DB).
But i have a issue with it, in my case i need to check a number of users, and this users keeps growing, in my case i have 400 users. If i go more thatn 100 users i get the 500 error of course, it is many requests.
So does someone have a ideia of how could i handle it?
My code: FarcebookParcer.php
public function facebook($id, $num) {
//Set your App ID and App Secret.
$appID = 'xxxxxxxxxx';
$appSecret = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx';
//Create an access token using the APP ID and APP Secret.
$accessToken = $appID . '|' . $appSecret;
//Tie it all together to construct the URL
$url = "https://graph.facebook.com/$id/posts?fields=attachments,created_time&limit=$num&access_token=$accessToken";
if (Helper::get_http_response_code($url) != 200) {
return false;
}
//Make the API call
$opts = array(
'http' => array(
'method' => 'GET',
'timeout' => 120
)
);
$context = stream_context_create($opts);
$result = file_get_contents($url, false, $context);
//Decode the JSON result.
$dt = json_decode($result, true);
$posts = $dt;
return $posts;
}
CronController.php
public function socialfacebook() {
$facebook = SocialSnap::all();
$socialparser = new FacebookParser();
$appID = 'xxxxxxxxxxx';
$appSecret = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx';
$accessToken = $appID . '|' . $appSecret;
set_time_limit(0);
foreach ($facebook as $key => $value) {
if ($value->facebook != NULL) {
$facebook_user = $socialparser->facebook_user(substr($value->facebook, 1));
$facebook_posts = $socialparser->facebook(substr($value->facebook, 1), 1);
//Check if valid url
if ($facebook_posts == false || $facebook_user == false) {
continue;
}
if (isset($facebook_posts['data'][0]['attachments']['data'][0]['target']['url']) && isset($facebook_posts['data'][0]['attachments']['data'][0]['description'])) {
SnapChat::where('facebook', $value->facebook)->update(['facebook_photo' => 'https://graph.facebook.com/'. substr($value->facebook, 1) . '/picture/?type=normal']);
$post_current = SocialSnap::where('id_social', $facebook_posts['data'][0]['id'])->first();
//return $post_current;
if ($post_current == NULL) {
$post = new SocialSnap;
$post->id_social = $facebook_posts['data'][0]['id'];
$post->id_snapchats = $value->id;
$post->date_social = isset($facebook_posts['data'][0]['created_time']) ? date("Y-m-d H:m:s", strtotime($facebook_posts['data'][0]['created_time'])) : "";
$post->type = 'facebook';
$post->url = $facebook_posts['data'][0]['attachments']['data'][0]['target']['url'];
$post->message = $facebook_posts['data'][0]['attachments']['data'][0]['description'];
if (isset($facebook_posts['data'][0]['attachments']['data'][0]['media']['image']['src'])) {
$post->image = $facebook_posts['data'][0]['attachments']['data'][0]['media']['image']['src'];
}
$post->save();
}
}
}
}
echo 'DONE';
}

BigQuery example for current Google API in PHP

All the question about examples are like 2 years old, and I can't find ANY working example for current API client version (https://github.com/google/google-api-php-client). Every single one is missing something or throwing exception....
Could anybody provide working example? There is no documentation AT ALL, anywhere.
This is the most working one:
<?php
require_once 'inc/google-api/autoload.php'; // or wherever autoload.php is located
$client = new Google_Client();
$client->setApplicationName("whatever");
$client->setDeveloperKey("some key");
$service = new Google_Service_Bigquery($client);
$postBody = "[
'datasetReference' => [
'datasetId' => $datasetId,
'projectId' => $projectId,
],
'friendlyName' => $name,
'description' => $description,
'access' => [
['role' => 'READER', 'specialGroup' => 'projectReaders'],
['role' => 'WRITER', 'specialGroup' => 'projectWriters'],
['role' => 'OWNER', 'specialGroup' => 'projectOwners'],
],
]";
$dataset = $service->datasets->insert($projectId, new Google_Dataset($postBody));
$postBody = "[
'tableReference' => [
'projectId' => 'test_project_id',
'datasetId' => 'test_data_set',
'tableId' => 'test_data_table'
]
]";
$table = $service->tables->insert($projectId, $datasetId, new Google_Table($postBody));
?>
But I am getting Fatal errors about Google_Dataset and Google_Table not defined...
Here is a code that
properly creates a Google_Client
runs a job async
displays the running job ID and status
You need to have:
service account created (something like ...#developer.gserviceaccount.com)
your key file (.p12)
service_token_file_location (writable path to store the JSON from the handshake, it will be valid for 1h)
code sample:
function getGoogleClient($data = null) {
global $service_token_file_location, $key_file_location, $service_account_name;
$client = new Google_Client();
$client->setApplicationName("Client_Library_Examples");
$old_service_token = null;
$service_token = #file_get_contents($service_token_file_location);
$client->setAccessToken($service_token);
$key = file_get_contents($key_file_location);
$cred = new Google_Auth_AssertionCredentials(
$service_account_name, array(
'https://www.googleapis.com/auth/bigquery',
'https://www.googleapis.com/auth/devstorage.full_control'
), $key
);
$client->setAssertionCredentials($cred);
if ($client->getAuth()->isAccessTokenExpired()) {
$client->getAuth()->refreshTokenWithAssertion($cred);
$service_token = $client->getAccessToken();
}
return $client;
}
$client = getGoogleClient();
$bq = new Google_Service_Bigquery($client);
/**
* #see https://developers.google.com/bigquery/docs/reference/v2/jobs#resource
*/
$job = new Google_Service_Bigquery_Job();
$config = new Google_Service_Bigquery_JobConfiguration();
$config->setDryRun(false);
$queryConfig = new Google_Service_Bigquery_JobConfigurationQuery();
$config->setQuery($queryConfig);
$job->setConfiguration($config);
$destinationTable = new Google_Service_Bigquery_TableReference();
$destinationTable->setDatasetId(DATASET_ID);
$destinationTable->setProjectId(PROJECT_ID);
$destinationTable->setTableId('table1');
$queryConfig->setDestinationTable($destinationTable);
$sql = "select * from publicdata:samples.github_timeline limit 10";
$queryConfig->setQuery($sql);
try {
// print_r($job);
// exit;
$job = $bq->jobs->insert(PROJECT_ID, $job);
$status = new Google_Service_Bigquery_JobStatus();
$status = $job->getStatus();
// print_r($status);
if ($status->count() != 0) {
$err_res = $status->getErrorResult();
die($err_res->getMessage());
}
} catch (Google_Service_Exception $e) {
echo $e->getMessage();
exit;
}
//print_r($job);
$jr = $job->getJobReference();
//var_dump($jr);
$jobId = $jr['jobId'];
if ($status)
$state = $status['state'];
echo 'JOBID:' . $jobId . " ";
echo 'STATUS:' . $state;
You can grab the results with:
$res = $bq->jobs->getQueryResults(PROJECT_ID, $_GET['jobId'], array('timeoutMs' => 1000));
if (!$res->jobComplete) {
echo "Job not yet complete";
exit;
}
echo "<p>Total rows: " . $res->totalRows . "</p>\r\n";
//see the results made it as an object ok
//print_r($res);
$rows = $res->getRows();
$r = new Google_Service_Bigquery_TableRow();
$a = array();
foreach ($rows as $r) {
$r = $r->getF();
$temp = array();
foreach ($r as $v) {
$temp[] = $v->v;
}
$a[] = $temp;
}
print_r($a);
You can see here the classes that you can use for your other BigQuery calls. When you read the file, please know that file is being generated from other sources, hence it looks strange for PHP, and you need to learn reading it in order to be able to use the methods from it.
https://github.com/google/google-api-php-client/blob/master/src/Google/Service/Bigquery.php
like:
Google_Service_Bigquery_TableRow

solr data can not be sent before solrcommit

What my problem is that I can not send array to solr machine in order to update. I am using codeigniter as a framework and here is my code:
$solrData = array();
$solrData['id'] = $this->data['profil_data']['id'];
$solrData['site'] = $this->data['profil_data']['site'];
$solrData['url_Domain'] = $this->data['profil_data']['url_Domain'];
$solrData['url_Page'] = $this->data['profil_data']['url_Page'];
$solrData['url_Profil'] = $this->data['profil_data']['url_Profil'];
$solrData['scr_Kobi_Rank'] = $this->data['profil_data']['scr_Kobi_Rank'];
$solrData['scr_A'] = $this->data['profil_data']['scr_A'];
$solrData['scr_B'] = $this->data['profil_data']['scr_B'];
$solrData['scr_C'] = $this->data['profil_data']['scr_C'];
$solrData['scr_D'] = $this->data['profil_data']['scr_D'];
$solrData['loc_City'] = $this->input->post('plakano');
$solrData['loc_Lat_Lon'] = $this->input->post('loc_Lat_Lon');
$solrData['com_Category'] = explode(',', $this->input->post('category'));
$urunData = $this->input->post('urun_list');
foreach($urunData as $row)
{
$ontoData = $this->m_onto->getOntoDataByOntoDataId($row);
$solrData['com_Products'][] = $ontoData['baslik'];
}
$hizmetData = $this->input->post('hizmet_list');
foreach($hizmetData as $row)
{
$ontoData = $this->m_onto->getOntoDataByOntoDataId($row);
$solrData['com_Services'][] = $ontoData['baslik'];
}
$solrData['com_Type'] = $this->input->post('sirketturu');
$solrData['com_Description'] = $this->input->post('description');
$solrData['com_Title_Selected'] = $this->input->post('title');
$solrData['com_Title_Long'] = $this->data['profil_data']['com_Title_Long'];
$solrData['crm_Tel'] = $this->input->post('tel');
$solrData['crm_Fax'] = $this->input->post('fax');
$solrData['crm_Email'] = $this->input->post('email');
$this->solr->updateSolrProfilData($solrData);
And solr process:
public function updateSolrProfilData($arrData)
{
if(count($arrData) == 0)
return FALSE;
$solrClientOptions = $this->solrClientOptionsYazProfil;
$solrClientOptionsCommit = $this->solrClientOptionsYazProfilCommit;
$solrClient = new SolrClient($solrClientOptions);
$solrDoc = new SolrInputDocument();
foreach($arrData as $firmaField => $firmaValue)
{
if(! is_array($firmaValue))
{
$solrDoc->addField($firmaField, $firmaValue);
}
else
{
foreach($firmaValue as $firmaField2 => $firmaValue2)
{
if($firmaValue2 != '')
{
$solrDoc->addField($firmaField, $firmaValue2);
}
}
}
}
try {
$this->_solrCommit($solrClientOptionsCommit);
} catch (Exception $e) {
echo $e->getMessage();
}
}
Solr Commit function:
private function _solrCommit($solrArr)
{
$urlCommit = 'http://' . $solrArr['hostname'] . ":" . $solrArr['port'] . '/' . $solrArr['path'] . "/update?stream.body=%3Ccommit/%3E&wt=json";
$output = file_get_contents($urlCommit);
$outputArr = json_decode($output, TRUE);
if ($outputArr['responseHeader']['status'] === 0)
return TRUE;
else
return FALSE;
}
And that is the options:
private $solrClientOptionsYazProfilCommit = array(
'hostname' => SOLR_HOST_YAZ,
'login' => '',
'password' => '',
'port' => SOLR_PORT,
'path' => 'solr/collection1'
);
Altough try-catch returns no error, the data can not be updated. Moreover, code sends solr commit succesfully. I checked the url but it is in correct form. What is wrong in here?
Dont use PHP/Pecl solr libs. If you can access solr via a URL then you should just use PHP and CURL:
static function doCurl($url, $username = null, $password = null) {
if (!function_exists('curl_init')) {
// throw error
}
$ch = curl_init();
$opts = array(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_POST => 1,
CURLOPT_TIMEOUT => 120,
CURLOPT_FAILONERROR => 1,
CURLOPT_HTTPAUTH => CURLAUTH_ANY
);
if ($password != null && $username != null) {
$opts[CURLOPT_USERPWD] = "$username:$password";
}
curl_setopt_array($ch, $opts);
$response = curl_exec($ch);
curl_close($ch);
return $response;
}
usage is:
doCurl("http://hostNameHere:8983/solr/select/?q=solr&start=0&rows=10&indent=on", "user", "pass");
Your issue is that you are never issuing a command to Solr to add the document that you have built to your index. You are only issuing the commit command, which is executing successfully.
Since you are using PHP, I would recommend using the PHP SolrClient. This will save you from having to manually write all of the functions (add, delete, commit, etc.) yourself. In this case, you would need to call the addDocument function.

Categories