PHP code for moving the cursor using the twitter API - php

So I already have a script that collects the first 4999 followers ids of a twitter user using the API in xml format. I semi understand how the cursor process works but I am confused how to implement it to loop until it gathers all the followers. The user I am attempting to gather will take about 8 calls. Any ideas on how to implement the cursor loop?
<?php
$xmldata = 'http://api.twitter.com/1/followers/ids/microsoft.xml';
$open = fopen($xmldata, 'r');
$content = stream_get_contents($open);
fclose($open);
$xml = simplexml_load_file($xmldata);
$cursor = $xml->next_cursor;
$file = fopen ('output1.csv', 'w+');
fwrite($file, "User id\n\r");
while($cursor =! 0)
{
foreach ($xml->ids->id as $id)
{
fwrite($file, $id . ", ");
fwrite($file, "\n");
}
$xmldata = 'http://api.twitter.com/1/followers/ids.xml?cursor='. $cursor
.'&screeb_name=microsoft';
?>

Let me take an example of Microsoft's followers (346K followers) as of now.
https://api.twitter.com/1/followers/ids.json?cursor=-1&screen_name=microsoft
It fetches only 5000 user IDs, that the twitter API limit. So, you need to take the next_cursor string from the json output
next_cursor_str":"1418048755615786027"
So, your next call would be
https://api.twitter.com/1/followers/ids.json?cursor=1418048755615786027&screen_name=microsoft
Keep doing this until the next_cursor is ZERO.
As you keep doing again and again, just keep storing all the ids..

Related

Export php array to json with a specific structure (google charts)

I'm trying to use the google charts api to present power (watt) over time. I have a sqlite database in which i store my data. A php script then gathers the data and outputs it into a json file. For the charts to work i need to keep a specific structure. Whenever i run the phpscript the json file gets overwritten with the new data. I need help with the php script to always output the data according to googles parameters.
I'm aiming to end up with an area chart that plots power on the y axis and the datestamps on the x axis.
I've read theese documents in googles documentation but i can't figure out how to output the data the way they do.
https://developers.google.com/chart/interactive/docs/php_example
https://developers.google.com/chart/interactive/docs/reference#dataparam
while ($res = $stmt->fetch(PDO::FETCH_ASSOC)){
$voltage[] = $res['voltage'];
$current[] = $res['current'];
$datestamp[] = $res['datestamp'];
}
$unixtimestamp = array();
foreach ($datestamp as $nebulosa){
$unixtimestamp[] = strtotime($nebulosa);
}
$power = array();
foreach ($voltage as $key => $door){
$power[] = $door * $current[$key];
}
//echo "<pre>", print_r($power, true), "</pre>";
$fp = fopen('data.json', 'w');
fwrite($fp, json_encode($power));
fwrite($fp, json_encode($datestamp));
fclose($fp);
The json file has this format after running the php script.
[3.468,5]["2016-10-14 14:56:22","2016-10-14 14:56:23"]

Getting all from json api array using php

I wanna improve on how to fetch data from an API. In this case I want to fetch every app-id from the Steam API, and list them one per line in a .txt file. Do I need an infinite (or a very high-number) loop (with ++ after every iteration) to fetch everyone? I mean, counting up from id 0 with for example a foreach-loop? I'm thinking it will take ages and sounds very much like bad practice.
How do I get every appid {"appid:" n} from the response of http://api.steampowered.com/ISteamApps/GetAppList/v0001?
<?php
//API-URL
$url = "http://api.steampowered.com/ISteamApps/GetAppList/v0001";
//Fetch content and decode
$game_json = json_decode(curl_get_contents($url), true);
//Define file
$file = 'steam.txt';
//This is where I'm lost. One massive array {"app": []} with lots of {"appid": n}.
//I know how to get one specific targeted line, but how do I get them all?
$line = $game_json['applist']['apps']['app']['appid'][every single line, one at a time]
//Write to file, one id per line.
//Like:
//5
//7
//8
//and so on
file_put_contents($file, $line, FILE_APPEND);
?>
Any pointing just in the right direction will be MUCH appreciated. Thanks!
You don't need to worry about counters with foreach loops, they are designed to go through and work with each item in the object.
$file = "steam.txt";
$game_list = "";
$url = "http://api.steampowered.com/ISteamApps/GetAppList/v0001";
$game_json = file_get_contents($url);
$games = json_decode($game_json);
foreach($games->applist->apps->app as $game) {
// now $game is a single entry, e.g. {"appid":5,"name":"Dedicated server"}
$game_list .= "$game->appid\n";
}
file_put_contents($file, $game_list);
Now you have a text file with 28000 numbers in it. Congratulations?

Music playlist via PHP. json_encode() array limit?

Im using PHP to create a playlist. Two random songs are chosen from a directory, and their name and location are stored in an array and then written to a file via json_encode().
$arraySongs[] = array('name' => $songName , 'mp3' => $webUrl);
This works great. I can make a very long playlist, two songs at a time. Id also like to remove songs, so I have an AJAX powered delete button, that posts the id of the track to be deleted, PHP then loads the whole tracklist...
$decoded = json_decode(file_get_contents($tracklist),true);
and removes the given song from the array, then re encodes and re writes the json text file. This all works great.
The problem comes whenever I try to delete anything with a playlist of more than 10 items.
Typically, my song.json file goes [{name:song1,mp3:song url},{name:song2,mp3:song2 url}]
However, when I have a list of more than 10 items, the re encoded playlist looks like this:
[{ ... },{name:song9,mp3:song9 url}],[10,{"name":song10,mp3:song10 url}]
Why is my re-encoded array get that strange [10,{"name"... [11,{"name"... [12,{"name"...
but everything below 10 is always fine?
Thanks for reading this! Any suggestions would be greatly appreciated, this is driving me nuts!
Here is the code im Using:
<?php
$json = "song.php";
$decoded = json_decode(file_get_contents($json),true);
$playlist = array();
$names = array();
// Now Get i From Javascript
$i=$_POST['id'];
//Select i's Array
$collect_array=$decoded[$i];
while (list ($key, $val) = each ($collect_array)) {
//Remove i's Values
//echo "<br />$key -> $val <br>";
unset($decoded[$i]);
}
//Take all the remaining arrays
$collect_array=$decoded;
while (list ($key, $val) = each ($collect_array)) {
$arraySongs[] = array($key , $val);
}
// Our New Array ready for json.
$jsonData = json_encode($arraySongs);
// open song.php and scribble it down
$tracklist = $json;
$fh = fopen($tracklist, 'w') or die("can't open filename: $tracklist");
fwrite($fh, $jsonData);
fclose($fh);
?>
try removing elements with unset
debug your code (not posted in the thread, so do it yourself) by adding a line where you var_dump or print_r the whole thing before json_encode
or it's a bug in json_encode which would not be nice...
Encode the track ID on 2 or even 3 digits using the php function sprintf with parameter %02d.
This worked fine for me.

Write very large array to file in PHP

I've got a client with a Magento shop. They are creating a txt file to upload to googlebase, which contains all of their products, but due to the quantity of products (20k), the script bombs out once it's taken up about 1gb. It's being run via cron.
Is there a way to either zip or segment the array, or write it to the file as it's created, rather than create the array and then write it?
<?php
define('SAVE_FEED_LOCATION','/home/public_html/export/googlebase/google_base_feed_cron.txt');
set_time_limit(0);
require_once '/home/public_html/app/Mage.php';
Mage::app('default');
try{
$handle = fopen(SAVE_FEED_LOCATION, 'w');
$heading = array('id','title','description','link','image_link','price','product_type','condition','c:product_code');
$feed_line=implode("\t", $heading)."\r\n";
fwrite($handle, $feed_line);
$products = Mage::getModel('catalog/product')->getCollection();
$products->addAttributeToFilter('status', 1);//enabled
$products->addAttributeToFilter('visibility', 4);//catalog, search
$products->addAttributeToFilter('type_id', 'simple');//simple only (until fix is made)
$products->addAttributeToSelect('*');
$prodIds=$products->getAllIds();
foreach($prodIds as $productId) {
$product = Mage::getModel('catalog/product');
$product->load($productId);
$product_data = array();
$product_data['sku']=$product->getSku();
$product_data['title']=$product->getName();
$product_data['description']=$product->getShortDescription();
$product_data['link']=$product->getProductUrl(). '?source=googleps';
$product_data['image_link']=Mage::getBaseUrl(Mage_Core_Model_Store::URL_TYPE_MEDIA).'catalog/product'.$product->getImage();
// Get price of item
if($product->getSpecialPrice())
$product_data['price']=$product->getSpecialPrice();
else
$product_data['price']=$product->getPrice();
$product_data['product_type']='';
$product_data['condition']='new';
$product_data['c:product_code']=$product_data['sku'];
foreach($product->getCategoryIds() as $_categoryId){
$category = Mage::getModel('catalog/category')->load($_categoryId);
$product_data['product_type'].=$category->getName().', ';
}
$product_data['product_type']=rtrim($product_data['product_type'],', ');
//sanitize data
foreach($product_data as $k=>$val){
$bad=array('"',"\r\n","\n","\r","\t");
$good=array(""," "," "," ","");
$product_data[$k] = '"'.str_replace($bad,$good,$val).'"';
}
$feed_line = implode("\t", $product_data)."\r\n";
fwrite($handle, $feed_line);
fflush($handle);
}
//---------------------- WRITE THE FEED
fclose($handle);
}
catch(Exception $e){
die($e->getMessage());
}
?>
I have two fast answers here:
1) Try to increase php's allowed maximum memory size (for the command line since it is a cron script)
2) The way the senior developers solve similar issues, where I currently work is something like the following:
Create a date field attribute with a name like googlebase_uploaded, and run the cron script with something like const MAX_PRODUCTS_TO_WRITE.
Then append to the file and flag each product that got appended.
What I am trying to say is slice the execution time into slower chunks that won't break the script.
Unfortunately that's where I'm missing java and c#

Generating Graph with 2 Y Values from Text File

I have remade my original post as it was terribly formatted. Basically I would like some advice / tips on how to generate a line graph with 2 Y Axis (temperature and humidity) to display some information from my text file. It is contained in a textfile called temperaturedata.txt I have included a link to one of my posts from the JpGrapher forum only because it is able to display the code clearly.
I understand that since it is JpGraph problem I shouldn't post here however the community here is a lot more supportive and active. Many thanks for all your help guys in advance!
my code
I don't see any reason why you shouldn't post here about jpgraph. And I don't see why you shouldn't post your sample code and data here, either.
The code you've posted on the other site is broken. Check line #42.
Furthermore, you're passing JpGraph a single row (specifically, the last row) via $keyval. $data is where all your data is stored, though in a wrong format. A very quick fix was:
$keyval = array();
$keyval['time'] = array();
$keyval['count'] = array();
$keyval['temperature'] = array();
$keyval['humidity'] = array();
if ($file) {
while (!feof($file)) {
$line = trim(fgets($file));
if (strlen($line)) {
$fields = explode(":", $line);
$keyval['time'][] = $fields[0];
$keyval['count'][] = $fields[1];
$keyval['temperature'][] = $fields[2];
$keyval['humidity'][] = $fields[3];
}
}
fclose($file);
}
which transposed $data and renamed it $keyval. (Where it used to hold time data in $data[x]['time'], it now holds it in $keyval['time'][x].) And we're passing $keyval['temperature'], which is a simple array of temperature values.

Categories