How to display all data from Invision Power board Rest API - php

I am looking to display all data but my result is only the first 25 for page 1:
I want to extract all the download file name for example
what would be the best way to get all the data
documentation :https://invisioncommunity.com/developers/rest-api
<?php
$communityUrl = 'https://www.example.com/ips4/';
$apiKey = 'c7a349a1629f02cd2855a58d77646f6d';
$endpoint = '/downloads/files';
$endpoint = '/core/members';
$curl = curl_init( $communityUrl . 'api' . $endpoint );
curl_setopt_array( $curl, array(
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_HTTPAUTH => CURLAUTH_BASIC,
CURLOPT_USERPWD => "{$apiKey}:"
) );
$response = curl_exec( $curl );
?>
My php code
<?php
$result_files = json_decode($response , true);
if (is_array($result_files)) {
foreach ($result_files as $value) {
if (is_array($value)) {
foreach ($value as $info) {
echo $info['title'];
}
}
}
?>
The code above display only 25 files. I have 150.
How to do that ?
Thank you

Have you looked the rest download document, there are page that defaults to 25 files, you have to send call to fetch 150 files per page. Use the perPages atripute to declare amount per page.
page int Page number
perPage int Number of results per page - defaults to 25
After looking rest commands here id put perPage = 150 to collect 150 files in 1 call. Try this:
$endpoint = '/downloads/files/{perPage=150}';
or
$endpoint = '/downloads/files?perPage=150'
One or another way will work to get 150 results.
Here is dynamic version if you will later first get dynamically how many files there is and then fetch that result files amount:
$endpoint = '/downloads/files?perPage='.$amount

Related

Schedule php wordpress script to run in background

I hope I am not expecting too much. Any help would be extremely useful for me, because I am stuck for days now.
I created a relatively simple wordpress plugin in php.
What is the plugin supposed to do ?
-The plugin is supposed to communicate with external api and import product data in json file.
Import is started by pressing "start import" button that is created by pluin in the wordpress menu - import products.
Example request:
curl -X GET
-H 'X-Api-Key: [api-key]'
https://example.com/products/[PRODUCT ID]
[PRODUCT ID] is supposed to range from 1 to 10000
Plugin receives json file with product feed for every single request - every single [PRODUCT ID]
Plugin creates a woocommerce product and attaches imported information.
Does the plugin work ?
Yes and no, It imports first 100 products (in about 1min) correctly and then it just stops, sometimes resulting in error related to the request taking too much time to finish.
I know that the plugin doesn't work because the import script is executed in the browser and gets timed out.
I also know that I should do this process in the background, split it into batches and queue execution. The thing is I tried many ways to do so but failed miserably.
I have composer and action scheduler installed.
Unfortunately everytime I try to split this into batches, use action scheduler It just doesn't work or imports first product and stops.
I know that I'm dealing with large amount of products, I don't have error handling, checking if imported product exists etc, but I really have to run this import once
so there is no need to make this plugin very refined. This has to run, import products and I can get rid of it.
I do have wp debug on, so my attepmts on using action scheduler didn't create any errors or fatal errors, but it just didn't work properly.
I have 1500MB Ram available, this is shared hosting server, I/O 1MB, 60 available processes, 88000/600000 Inodes used, 5/200GB disc space used.
I increased php maxExecutionTime to 3000, memorylimit 1536M, maxInputTime 3000, but that didn't change anything.
I'm attaching my working code below. This is the version without my poor attemts on using action scheduler, splitting it into batches and running it in the backgroud.
I feel like this is going to be easier to read.
This code below runs in the web browser and works, but gets timed out.
I will be extremely grateful for any help with running it in the background.
Is it possible to just run this script from SSH linux terminal so it doesn't get timed out ?
`
<?php
/**
* Plugin Name: Product Importer
* Description: Imports products from an external API
* Version: 1.0
* Author: me
* Author URI: http://www.example.com
*/
// Include the Autoscheduler library
require_once '/home/user/vendor/woocommerce/action-scheduler/action-scheduler.php';
add_action('admin_menu', 'add_import_button');
function add_import_button() {
add_menu_page('Import Products', 'Import Products', 'manage_options', 'import_products', 'import_products_page');
}
function import_products_page() {
echo '<h1>Import Products</h1>';
echo '<form method="post">';
echo '<input type="submit" name="start_import" value="Start Import">';
echo '</form>';
if (isset($_POST['start_import'])) {
import_function();
}
}
function import_function() {
$product_id = 1;
while($product_id < 10000){
$product_id++;
$api_key = 'my-api-key';
$headers = array(
'X-Api-Key: ' . $api_key,
);
$url = 'https://example.com/products/';
$product_url = $url . $product_id;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $product_url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);
$product_data = json_decode($response, true);
// post array etc
// Set other product data as required.
}
}
}
`
One of the ways to solve this issue is using recursion where the code can run in the background. Take a look at the example below
require_once '/home/user/vendor/woocommerce/action-scheduler/action-scheduler.php';
add_action('admin_menu', 'add_import_button');
function add_import_button() {
add_menu_page('Import Products', 'Import Products', 'manage_options', 'import_products', 'import_products_page');
}
function import_products_page() {
echo '<h1>Import Products</h1>';
echo '<form method="post">';
echo '<input type="hidden" name="product_id" value="1">'; // optional
echo '<input type="submit" name="start_import" value="Start Import">';
echo '</form>';
if (isset($_POST['start_import'])) {
import_function();
}
}
// AJAX function
add_action( 'wp_ajax_nopriv_import_function', 'import_function' );
add_action( 'wp_ajax_import_function', 'import_function' );
function import_function() {
$product_id = ( ! empty( $_POST['product_id'] ) ) ? $_POST['product_id'] : 1;
$url = 'https://example.com/products/';
$args = array(
'headers' => array(
'Content-Type' => 'application/json',
'X-Api-Key' => 'apikey12345'
)
);
// this call the function and return the body
$results = wp_remote_retrieve_body(wp_remote_get($url . $product_id, $args));
// convert to array
$results = json_decode( $results );
// Stop the code execution on this conditions
if( ! is_array( $results ) || empty( $results ) ){
return false;
}
// Do your product creation here...
$product_id++; // increase $product_id
wp_remote_post( admin_url('admin-ajax.php?action=import_function'), [
'blocking' => false, // needed for the script to continue running on the background
'sslverify' => false, // needed if working on localhost.
'body' => [
'product_id' => $product_id
]
] );
}
OK, I managed to get it to work. I assume that it's dumb way to do this, but since it works it's fine for me. The only thing that is problematic now is the fact that the code for image import that I used previously (when code ran in browser) now makes the plugin stuck. Without it it works great and fast. Here is my current code:
<?php
/**
* Plugin Name: Product Importer
* Description: Imports products from an external API
* Version: 3.0
* Author: me
* Author URI: http://www.example.com
*/
register_activation_hook( __FILE__, 'schedule_import' ); //plugin starts when activated, fine for me
function schedule_import() {
wp_schedule_single_event( time(), 'import_products' );
}
add_action( 'import_products', 'import_function' );
function import_function() {
$batch_size = 50; //when split into batches it worked faster than one products after another, so ok
$batch_delay = 60; //give it some time to finish batch, to avoid problems
$last_imported_product_id = get_option( 'last_imported_product_id', 1 ); //need it for incrementation and rescheduling
$api_key = 'apikey123456789';
$headers = array(
'X-Api-Key: ' . $api_key,
);
$url = 'https://example.com/products/';
for ( $i = 0; $i < $batch_size; $i++ ) {
$product_id = $last_imported_product_id + $i;
$product_url = $url . $product_id;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $product_url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);
$product_data = json_decode($response, true);
$post = array(
'post_title' => $product_data['name'],
'post_content' => $product_data['description'],
'post_status' => 'publish',
'post_type' => 'product',
'meta_input' => array(
'_virtual' => 'yes',
'_regular_price' => $product_data['price'],
),
);
$post_id = wp_insert_post($post);
update_post_meta($post_id, '_sku', $product_data['Id']);
wp_set_object_terms($post_id, 'external', 'product_type');
$external_url = 'https://example.external.url';
update_post_meta( $post_id, '_product_url', $external_url );
update_option( 'last_imported_product_id', $product_id + 1 ); //incrementation of product_id for rescheduling
wp_schedule_single_event( time() + $batch_delay, 'import_products' ); //rescheduling
}
}
This above works well. However when I add my old code for image imports it imports nothing or 1 product (without image lol) and becomes stuck. After a minute it imports the same product over and over again. Old code for image import:
// Get the product images
$images = $product_data['images']['screenshots'];
$image_ids = [];
foreach ($images as $image) {
// Download the image
$image_url = $image['url'];
$response = wp_remote_get($image_url);
if (is_array($response)) {
$image_data = $response['body'];
$filename = basename($image_url);
$file_array = array(
'name' => $filename,
'tmp_name' => download_url($image_url),
);
// Insert the image into the media library
$attach_id = media_handle_sideload($file_array, $post_id);
if (!is_wp_error($attach_id)) {
array_push($image_ids, $attach_id);
}
}
// Set the product image gallery
update_post_meta($post_id, '_product_image_gallery', implode(',', $image_ids));
// Set the product featured image
update_post_meta($post_id, '_thumbnail_id', $image_ids[0]);
And no, I don't put it after rescheduling, but after $post array.
Honestly no idea why it would crash, tried smaller batches, longer wait between batches etc. I assume the code for images import is just very poorly written. Any solutions for this problem are going to be very appreciated ! :)
PS: I am totally fine with importing the first image available (there is around 6) and setting it up as thumbnail for product. I can give up on the gallery since It's going to slow down the process anyway.
EDIT: here is the fragment of json file received from api. (for better context of importing images):
"images":{
"screenshots":[
{
"url":"https://example.jpg",
"thumbnail":"https://example.jpg"
},
{
"url":"https://example.jpg",
"thumbnail":"https://example.jpg"
},
{
"url":"https://example.jpg",
"thumbnail":"https://example.jpg"
etc....

Scrape business listing content from google search

I am developing a simple PHP app which takes
business name,
business address
and business phone from user and then checks if that business is listed in Google or not and
Also compares the business name, address and phone returned by Google against the search terms.
The result I want to display whether the information found in Google is accurate or whether something is different or missing. Something similar as this site does
What I have tried:
I have tried to scrape page with phpQuery library but it does not include that part(which is circled in image below).
$buss_name = $_GET['business_name'];
$link = "https://www.google.com/search?q=" . urlencode($buss_name) . "&rct=j";
$resp_html = file_get_contents($link, false);
$resp_html = phpQuery::newDocumentHTML($resp_html);
echo $resp_html;
echo pq("div.kno-ecr-pt.kno-fb-ctx._hdf",$resp_html)->text();
Reason is that it is loaded via some sort of AJAX call.
I also tried this web service by google
http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=don%20jayne%20&%20assoc
But this also do not include that part I require.
Long story short >>>
Please tell me is there any API or whatever is available which checks for a business listed on Google or not?
probably you wont need this anymore but I will reply your post just for future reference.
One of the ways to check if a business is on google or not, is to check the google places api (documentation). You can preform a search and then check the results for the business you are looking for. Something like this:
$params = array(
'query' => 'mindseo',
'key' => "XXXXXXXXXXXXXXXXXXX");
$service_url = 'https://maps.googleapis.com/maps/api/place/textsearch/json';
//do the request
$placesSearch = request( $service_url, $params );
//print the result
print_r($placesSearch);
//loop the results
if ( count( $placesSearch['results'] ) >= 1 ) {
$params = array(
'placeid' => $placesSearch['results'][0]['place_id'],
'key' => "AIzaSyAc73-uGCLLIuN3Bb2idOwRbLBzoaTmPHI");
$service_url = 'https://maps.googleapis.com/maps/api/place/details/json';
$placeData = request( $service_url, $params );
//echo the place data
echo '//Place ID#'.$params['placeid'].' DATA -----------------------';
print_r($placeData);
}
//function to make the request
function request( $googleApiUrl, $params) {
$dataOut = array();
$url = $googleApiUrl . '?' . http_build_query($params);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$dataOut = json_decode(curl_exec($ch), true);
curl_close($ch);
return $dataOut;
}

PHP Instagram API - Getting all images for a tag. How to create a pagination with min and max_tag_id

I'm working around the Instagram PHP API to get all images/posts for a specific tag in Instagram.
Here is my code:
<?PHP
function callInstagram($url)
{
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_SSL_VERIFYHOST => 2
));
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
$tag = "bulgaria";
$client_id = "1e0f576fbdb44e299924a93cace24507";
$Next_URL = $_GET["nexturl"];
if($Next_URL == ""){
$url = 'https://api.instagram.com/v1/tags/'.$tag.'/media/recent?client_id='.$client_id.'&count=24';
} else {
$url = $Next_URL;
}
$inst_stream = callInstagram($url);
$results = json_decode($inst_stream, true);
$maxid = $results['pagination']['next_max_id'];
$nexturl = $results['pagination']['next_url'];
//Now parse through the $results array to display your results...
foreach($results['data'] as $item){
$image_link = $item['images']['thumbnail']['url'];
$Profile_name = $item['user']['username'];
echo '<div style="display:block;float:left;">'.$Profile_name.' <br> <img src="'.$image_link.'" /></div>';
}
echo "<div style='display:block;width:100%;clear:both;'>MaxID: $maxid <br>NextURL: <a href='?nexturl=$nexturl'>Next images</a> <br>N: $Next_URL</div>";
With this code it seems I'm able to get some images but I can not display the next ones when I click on Next images.
I am trying to create some kind of pagination with the min_tag_id and max_tag_id given here: https://instagram.com/developer/endpoints/
But I seems these two min_tag_id and max_tag_id are not working for me, but why?
As of Nov 2017, Instagram has just changed their policy again and does not allow people to apply for permissions for public content anymore.
The change may haven been made because Instagram plans to allow users to follow hashtags on IG itself. They don't want any (new) third party having a similar feature that they could make money from.

Get the Instagram image page link using PHP API

I'm trying to fetch images by using hashtags, and want to get the direct link of the actuall Instagram image page not the image link.
like this link : http://instagram.com/p/oRANLioVee/
But i wasn't able to do that, here is my code:
function callInstagram($url)
{
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_SSL_VERIFYHOST => 2
));
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
// Set number of photos to show
// Set height and width for photos
$count = '9999';
$tag = 'mytag';
$url = 'https://api.instagram.com/v1/tags/'.$tag.'/media/recent?client_id=765390678573452431546477809798&count='.$count;
$inst_stream = callInstagram($url);
$results = json_decode($inst_stream, true);
foreach($results['data'] as $item){
$image_link = $item['images']['thumbnail']['url'];
$fullsize = $item['images']['standard_resolution']['url'];
echo '<a target="_blank" href="'.$fullsize.'"><img src="'.$image_link.'" /></a>';
}
Any help would be appreciated.
And although, i set the count to 9999 but i just got 33 images.
Any thoughts?
count doesn't seem to be a valid parameter for the /tags endpoint. The documentation is here:
http://instagram.com/developer/endpoints/tags/
The data is paginated when it is returned so you need to use MIN_ID and MAX_ID to page through results.

Getting JSON response with PHP

I'm trying to connect an API that uses 0AUTH2 via PHP. The original plan was to use client-side JS, but that isn't possible with 0AUTH2.
I'm simply trying get a share count from the API's endpoint which is here:
https://api.bufferapp.com/1/links/shares.json?url=[your-url-here]&access_token=[your-access-key-here]
I do have a proper access_token that I am using to access the json file, that is working fine.
This is the code I have currently written, but I'm not even sure I'm on the right track.
// 0AUTH2 ACCESS TOKEN FOR AUTHENTICATION
$key = '[my-access-key-here]';
// JSON URL TO BE REQUESTED
$json_url = 'https://api.bufferapp.com/1/links/shares.json?url=http://bufferapp.com&access_token=' . $key;
// GET THE SHARE COUNT FROM THE REQUEST
$json_string = '[shares]';
// INITIALIZE CURL
$ch = curl_init( $json_url );
// CONFIG CURL OPTIONS
$options = array(
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => array('Content-type: application/json') ,
CURLOPT_POSTFIELDS => $json_string
);
// SETTING CURL AOPTIONS
curl_setopt_array( $ch, $options );
// GET THE RESULTS
$result = curl_exec($ch); // Getting jSON result string
Like I said, I don't know if this is the best method - so I'm open to any suggestions.
I'm just trying to retrieve the share count with this PHP script, then with JS, spit out the share count where I need it on the page.
My apologies for wasting anyone's time. I have since been able to work this out. All the code is essentially the same - to test to see if you're getting the correct response, just print it to the page. Again, sorry to have wasted anyones time.
<?php
// 0AUTH2 ACCESS TOKEN FOR AUTHENTICATION
$key = '[your_access_key_here]';
// URL TO RETRIEVE SHARE COUNT FROM
$url = '[your_url_here]';
// JSON URL TO BE REQUESTED - API ENDPOINT
$json_url = 'https://api.bufferapp.com/1/links/shares.json?url=' . $url . ' &access_token=' . $key;
// GET THE SHARE COUNT FROM THE REQUEST
$json_string = '[shares]';
// INITIALIZE CURL
$ch = curl_init( $json_url );
// CONFIG CURL OPTIONS
$options = array(
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => array('Content-type: application/json') ,
CURLOPT_POSTFIELDS => $json_string
);
// SETTING CURL AOPTIONS
curl_setopt_array( $ch, $options );
// GET THE RESULTS
$result = curl_exec($ch); // Getting jSON result string
print $result;
?>

Categories