Let's say I have this code in PHP:
public function getFromAzure($searchParam, $jobCategory, $top, $skip, $request){
$listingManager = $this->get('rd.model_manager.job_listing');
$url = $listingManager->getAzureSearchParam($request, 'azure_search_idx');
$apiKey = $listingManager->getAzureSearchParam($request, 'azure_key');
$searchParam = preg_replace('/\s+/', '+', $searchParam);
$postdata = json_encode(
array(
'search' => $searchParam,
'filter' => $category,
'orderby'=> 'publishedDate desc',
'facets' => array('locationName','employmentType', 'workSchedule','jobFunction','positionLevel','industry'),
'top' => $top,
'skip' => $skip,
'count' => true
)
);
$opts = array(
'http'=>array(
'method'=>"POST",
'header'=>"Content-type: application/json\r\n" .
"api-key: ". $apiKey . "\r\n" .
"Accept: application/json",
'content'=>$postdata
)
);
$context = stream_context_create($opts);
// Open the file using the HTTP headers set above
$file = file_get_contents($url, false, $context);
$file = json_decode($file,true);
return $file;
}
Is there a way I can iterate the $skip inside this function(PHP)?
Currently I am iterating skip through Ajax call from javascript. 1 ajax call per (15 record) skip.
I need to iterate the skip because I need to load about 2,000 records(and counting)on page load, which per query is limited to 1,000. and assuming if I have 10,000 records or more. then I would have to run ajax every 1000 (for example) which could have performance issue.
btw fyi only- I need this on facets. where I have to iterate throught all records and get their value of my facet categories.
Thank you in advance! Cheers!
Related
I have a php script to update my subscribers from my wordpress userlist to Mailchimp using batch-subscribe. (https://apidocs.mailchimp.com/api/2.0/lists/batch-subscribe.php)
Everything works fine when I submit about 400 records. All records are added, and I get a return from the API with the number of added records, etc.
If I submit about 600 or more (I have about 730 subscribers), all records are added to Mailchimp, but the API returns FALSE. I double checked it with === false, and it is false. I get no errors -- it just returns false (but all records are added to Mailchimp).
Mailchimp says "Maximum batch sizes vary based on the amount of data in each record, though you should cap them at 5k - 10k records, depending on your experience." (https://apidocs.mailchimp.com/api/2.0/lists/batch-subscribe.php).
I'm nowhere close to that, and every record is being added to the mailchimp list just fine. I just don't get the return from the API.
I've increased my timeout value to 5 minutes. I also switched to using different records, suspecting I might have had a record with something that was causing it to mess up, but it had the same behavior with different records.
I'm using the DrewM library to interface with Mailchimp API version 2.0. I double checked to make sure DrewM is using post for the request, and it does. (https://github.com/drewm/mailchimp-api/)
Any ideas what is causing this?
Here is the code:
function mailchimpdailyupdate () {
set_time_limit(300);
$api = get_mc_api();
$mcListId = get_mc_mailing_list();
$MailChimp = new \Drewm\MailChimp($api);
...
foreach ( $blogusers as $user ) {
$userinfo = get_userdata( $user->ID );
$location = ...//code to get location
$merge_vars = array(
'FNAME'=> $userinfo->first_name,
'LNAME'=> $userinfo->last_name,
'MMERGE3'=> $userinfo->user_login, //username
'MMERGE6'=> $location //location
);
$batch[] = array(
'email' => array('email' => $user->user_email),
'merge_vars' => $merge_vars
);
} //end foreach
//mailchimp call
$retval = $MailChimp->call('lists/batch-subscribe', array(
'id' => $mcListId, // your mailchimp list id here
'batch' => $batch,
'update_existing' => true
)
);
if ($retval === false) {
echo "Mailchimp API returned false";
}
echo 'Added: ' . $retval['add_count'] . "<br/>";
echo 'Updated: ' . $retval['update_count'] . "<br/>";
echo 'Errors: ' . $retval['error_count'] . "<br/>";
}
With help from Mailchimp support, I was able to locate and solve the problem.
The issue was actually in the DrewM wrapper.
The content-length section of the header was apparently not working correctly on long calls. I removed it, and everything began working fine.
Original section of DrewM code (not working):
$result = file_get_contents($url, null, stream_context_create(array(
'http' => array(
'protocol_version' => 1.1,
'user_agent' => 'PHP-MCAPI/2.0',
'method' => 'POST',
'header' => "Content-type: application/json\r\n".
"Connection: close\r\n" .
"Content-length: " . strlen($json_data) . "\r\n",
'content' => $json_data,
),
)));
Updated section of code (working):
$result = file_get_contents($url, null, stream_context_create(array(
'http' => array(
'protocol_version' => 1.1,
'user_agent' => 'PHP-MCAPI/2.0',
'method' => 'POST',
'header' => "Content-type: application/json\r\n".
"Connection: close\r\n",
'content' => $json_data,
),
)));
I have used the code below with the Bing image API to search images. But the code gets only 50 results. How can I get more results?
$acctKey = 'Account key here';
$rootUri = 'https://api.datamarket.azure.com/Bing/Search';
$query = 'Kitchen';
$serviceOp = 'Image';
$market ='en-us';
$query = urlencode("'$query'");
$market = urlencode("'$market'");
$requestUri = "$rootUri/$serviceOp?\$format=json&Query=$query&Market=$market";
$auth = base64_encode("$acctKey:$acctKey");
$data = array(
'http' => array(
'request_fulluri' => true,
'ignore_errors' => true,
'header' => "Authorization: Basic $auth"
)
);
$context = stream_context_create($data);
$response = file_get_contents($requestUri, 0, $context);
$response=json_decode($response);
echo "<pre>";
print_r($response);
echo "</pre>";
You should append your request with $top=50&$skip=0 to get the first 50 results and with $top=50&$skip=51 to get the next 50 results, it's better to create API key to get the second request as Bing permits 1 transaction per second and you may get this message error if you exceeded the limits: "503 The number
of requests per minute for the subscription has reached the maximum threshold that is allowed."
I am using the Wikimedia api and php. I need to get first main image from article and all text in article. I have got code for it, but it takes only short info and very little picture. I tried to change many parameters, but it is not working.
Code is here:
function get_wiki_url($title) {
$context = stream_context_create(array(
'http' => array(
'method'=>"POST",
'content' => $reqdata = http_build_query(array(
'action' => 'opensearch',
'search' => $title,
'prop' => 'info',
'format' => 'xml',
'inprop' => 'url'
)),
'header' => implode("\r\n", array(
"Content-Length: " . strlen($reqdata),
"User-Agent: MyCuteBot/0.1",
"Connection: Close",
""
))
)));
if (false === $response = file_get_contents("http://ru.wikipedia.org/w/api.php", false, $context)) {
return false;
}
//парсим строку
$xml = simplexml_load_string($response);
return $xml->Section->Item;
}
var_dump ($pages_data = get_wiki_url("article header"));
Your query appears to be running a search for $title (parameter action=opensearch); if you want the article and main image (I presume you want HTML, not wikitext), you need to use action=parse -- see the Mediawiki parse documentation.
Example URL for getting the Hyperloop page:
http://ru.wikipedia.org/w/api.php?action=parse&format=xml&page=Hyperloop
The documentation has details on all the options available.
I am currently posting images URLs on different facebook pages but I am using many calls to do the same thing.
Instead of this, I am trying to work on a batch request to save on execution time but I have an issue.
Doing many calls, I use this:
$urlLink = '/' . $pageId . '/photos';
$args = array(
'url' => $this->image_url,
'message' => $this->message,
'published' => false,
'scheduled_publish_time' => strtotime($this->programmed_dt),
);
$res = $this->fb->api($urlLink, 'POST', $args);
It works fine.
With the batch request I tried with that:
$urlLink = '/' . $facebookPage['id'] . '/photos';
$args['access_token'] = $facebookPage['access_token'];
$queries[] = array('method' => 'POST',
'relative_url' => $urlLink,
'body' => $args,
'url' => $this->image_url
);
$res = $this->fb->api('?batch=' . json_encode($queries), 'POST');
The response I have is:
{"error":{"message":"(#324) Requires upload file","type":"OAuthException","code":324}}
I tried to change the name field with all possibilities to send the image link, without success...
Any ideas for batch requests with image urls?
According to https://developers.facebook.com/docs/graph-api/making-multiple-requests/#multiple_methods I assume that your code which forms the request doesn't have the correct syntax.
From my point of view it should be the following:
$args['access_token'] = $facebookPage['access_token'];
$queries[] = array('method' => 'POST',
'relative_url' => $urlLink,
'body' => 'url=' . $this->image_url
);
$res = $this->fb->api('/?batch=' . json_encode($queries) . '&access_token=' . $args['access_token'], 'GET');
I can see everything ok in your code except this-
You should set upload support to true before posting image.
$this->fb->setFileUploadSupport(true);
for more detail you can check this answer:
CHECK HERE
I guess PHP's get_headers does not allow for a context, so I have to change the default stream context to only get the HEAD of a request. This causes some issues with other requests on the page. I can't seem to figure out how to reset the default stream context. I'm trying something like:
$default = stream_context_get_default(); //Get default stream context so we can reset it
stream_context_set_default( //Only fetch the HEAD
array(
'http' => array(
'method' => 'HEAD'
)
)
);
$headers = get_headers($url, 1); //Url can be whatever you want it to be
//var_dump($headers);
var_dump($default);
stream_context_set_default($default); //This doesn't work as it expects an array and not a resource pointer
Does anyone know a fix for this?
I know it has been suggested to use Curl, but I would rather not for this one. Thanks!
I ended up using the stream_get_meta_data() function to get the HTTP headers.
This is how I implemented it:
function get_headers_with_stream_context($url, $context, $assoc = 0) {
$fp = fopen($url, 'r', null, $context);
$metaData = stream_get_meta_data($fp);
fclose($fp);
$headerLines = $metaData['wrapper_data'];
if(!$assoc) return $headerLines;
$headers = array();
foreach($headerLines as $line) {
if(strpos($line, 'HTTP') === 0) {
$headers[0] = $line;
continue;
}
list($key, $value) = explode(': ', $line);
$headers[$key] = $value;
}
return $headers;
}
Called like this,
$context = stream_context_create(array('http' => array('method' => 'HEAD')));
$headers = get_headers_with_stream_context($url, $context, 1);
it gives you what you're after while leaving the standard stream_context unmodified.
Please note that this function will fail if passed anything other than an http url.
There seems to be a feature request for an additional argument for get_headers(), but the bug tracker is down as I'm writing this, so I can't check for other solutions there.
I had a similar issue but I just used the file_get_contents function with the custom stream context instead.
Here's how I implemented it:
$options = array(
'http' => array(
'method' => 'HEAD',
'follow_location' => 0
)
);
$context = stream_context_create($options);
#file_get_contents($url, NULL, $context);
var_dump($http_response_header);
Using this context only the headers will be fetched by file_get_contents and will populate the $http_response_header PHP variable.
Instead of the accepted answer, I did the following, which will work in PHP 5.3 and above, though I haven't fully tested it. (There's also a stream_context_get_params($context) but I think this is enough.)
$stream_context_defaults = stream_context_get_options(stream_context_get_default());
stream_context_set_default(
array(
'http' => array(
'method' => 'HEAD'
)
)
);
for ($i = 1; $i <= 10; $i++) {
$headers = get_headers('http://www.example.org');
}
stream_context_set_default($stream_context_defaults); // reset to defaults
As of PHP 7.1.0, get_headers() now accepts a third parameter for context.
$context = stream_context_create(
array(
'http' => array('method' => 'HEAD')
)
);
$headers = get_headers($url, true, $context);