I wanna improve on how to fetch data from an API. In this case I want to fetch every app-id from the Steam API, and list them one per line in a .txt file. Do I need an infinite (or a very high-number) loop (with ++ after every iteration) to fetch everyone? I mean, counting up from id 0 with for example a foreach-loop? I'm thinking it will take ages and sounds very much like bad practice.
How do I get every appid {"appid:" n} from the response of http://api.steampowered.com/ISteamApps/GetAppList/v0001?
<?php
//API-URL
$url = "http://api.steampowered.com/ISteamApps/GetAppList/v0001";
//Fetch content and decode
$game_json = json_decode(curl_get_contents($url), true);
//Define file
$file = 'steam.txt';
//This is where I'm lost. One massive array {"app": []} with lots of {"appid": n}.
//I know how to get one specific targeted line, but how do I get them all?
$line = $game_json['applist']['apps']['app']['appid'][every single line, one at a time]
//Write to file, one id per line.
//Like:
//5
//7
//8
//and so on
file_put_contents($file, $line, FILE_APPEND);
?>
Any pointing just in the right direction will be MUCH appreciated. Thanks!
You don't need to worry about counters with foreach loops, they are designed to go through and work with each item in the object.
$file = "steam.txt";
$game_list = "";
$url = "http://api.steampowered.com/ISteamApps/GetAppList/v0001";
$game_json = file_get_contents($url);
$games = json_decode($game_json);
foreach($games->applist->apps->app as $game) {
// now $game is a single entry, e.g. {"appid":5,"name":"Dedicated server"}
$game_list .= "$game->appid\n";
}
file_put_contents($file, $game_list);
Now you have a text file with 28000 numbers in it. Congratulations?
Related
I'm very new to PHP, making errors and learning as I go. Please be gentle! :)
I want to access some data from Blizzard.com's API. For this particular data set, it's not a block of data in JSON, rather each object has it's own URL to access. I estimate that there are approx 150000 objects, however I don't know the start or end points of the number range. So I'm having to assume 1 and work past the highest number I know (269065)
To get the data, I need to access each object's data via a JSON file, which I read, get the contents of & drop in to a text file (this could be written as an insert in to a SQL db too, as I'm able to do this if it's the text file that's the issue). But to be honest, I would love to get to the bottom of why this is happening as much as anything!
I wasn't going to try and run ~250000 iterations in a for loop, I thought I'd try something I considered small, 2000.
The for loop starts with $a as 1, uses $a as part of the URL, loads & decodes the JSON, checks to see if the first field (ID) in the object is set, if it is, it writes a few fields to data.txt & if the first field (ID) isn't set it just writes $a to data.txt (so I know it's a null for other purposes not outlined here).
Simple! Or so I thought, after approx after 183 iterations, the data written to the text file goes awry as seen by the quote below. It is out of sequence and starts at 1 again, then back to 184 ad nauseam. The loop then seems to be locked in some kind of infinite loop of running, outputting in a random order until I close the page 10-20 minutes later.
I have obviously made a big mistake! But I have no idea what I have done wrong to have caused this. During my attempts I have rewritten the code with new variable names, so a new text does not conflict with code that could be running in memory.
I've tried resetting variables to blank at the end of the loop in case it something was being reused that was causing a problem.
If anyone could point out any errors in my code, or suggest something for me to look in to, to handle bigger loops that would be brilliant. I am assuming my issue may be a time out or memory problem. But I don't know where to start & was hoping I'd find some suggestions here.
If it's relevant, I am using 000webhostapp.com as my host provider for now, until I get some paid for hosting.
1 ... 182 183 1 184 2 3 185 4 186 5 187 6 188 7 189 190 8 191
for ($a = 1; $a <= 2000; $a++) {
$json = "https://eu.api.battle.net/wow/recipe/".$a."?locale=en_GB&<MYPRIVATEAPIKEY>";
$contents = file_get_contents($json);
$data = json_decode($contents,true);
if (isset($data['id'])) {
$file = fopen("data.txt","a");
fwrite($file,$data['id'].",'".$data['name']."'\n");
fclose($file);
} else {
$file = fopen("data.txt","a");
fwrite($file,$a."\n");
fclose($file);
}
}
The content of the file I'm trying to access is
{"id":33994,"name":"Precise Strikes","profession":"Enchanting","icon":"spell_holy_greaterheal"}
I scrapped the original plan and wrote this instead. Thank you again who took the time out of their day to help and offer suggestions!
$b = $mysqli->query("SELECT id FROM `static_recipes` order by id desc LIMIT 1;")->fetch_object()->id;
if (empty($b)) {$b=1;};
$count = $b+101;
$write = [];
for ($a = $b+1; $a < $count; $a++) {
$json = "https://eu.api.battle.net/wow/recipe/".$a."?locale=en_GB&apikey=";
$contents = #file_get_contents($json);
$data = json_decode($contents,true);
if (isset($data['id'])) {
$write [] = "(".$data['id'].",'".addslashes($data['name'])."','".addslashes($data['profession'])."','".addslashes($data['icon'])."')";
} else {
$write [] = "(".$a.",'a','a','a'".")";
}
}
$SQL = ('INSERT INTO `static_recipes` (id, name, profession, icon) VALUES '.implode(',', $write));
$mysqli->query($SQL);
$mysqli->close();
$write = [];
for ($a = 1; $a <= 2000; $a++) {
$json = "https://eu.api.battle.net/wow/".$a."?locale=en_GB&<MYPRIVATEAPIKEY>";
$contents = file_get_contents($json);
$data = json_decode($contents,true);
if (isset($data['id'])) {
$write [] = $data['id'].",'".$data['name']."'\n";
} else {
$write [] = $a."\n";
}
}
$file = fopen("data.txt","a");
fwrite($file, implode('', $write));
fclose($file);
Also, why you are think what some IDS isn't duplicated at several "https://eu.api.battle.net/wow/[N]" urls data?
Also if you are I wasn't going to try and run ~250000 think about curl_multi_init(): http://php.net/manual/en/function.curl-multi-init.php
I can't really see anything obviously wrong with your code, can't run it though as I don't have the JSON
It could be possible that there is some kind of race condition since you're opening and closing the same file hundreds of times very quickly.
File operations might seem atomic but not necessarily so - here's an interesting SO thread:
Does PHP wait for filesystem operations (like file_put_contents) to complete before moving on?
Like some others' suggested - maybe just open the file before you enter the loop then close the file when the loop breaks.
I'd try it first and see if it helps.
There's nothing in your original code that would cause that sort of behaviour. PHP will not arbitrarily change the value of a variable. You are opening this file in append mode, are you certain that you're not looking at old data? Maybe output some debug messages as you process the data. It's likely you'd run up against some rate limiting on the API server, so putting a pause in there somewhere may improve reliability.
The only substantive change I'd suggest to your code is opening the file once and closing it when you're done.
$file = fopen("data_1_2000.txt", "w");
for ($a = 1; $a <= 2000; $a++) {
$json = "https://eu.api.battle.net/wow/recipe/$a?locale=en_GB&<MYPRIVATEAPIKEY>";
$contents = file_get_contents($json);
$data = json_decode($contents, true);
if (!empty($data['id'])) {
$data["name"] = str_replace("'", "\\'", $data["name"]);
$record = "$data[id],'$data[name]'";
} else {
$record = $a;
}
fwrite($file, "$record\n");
sleep(1);
echo "$a "; if ($a % 50 === 0) echo "\n";
}
fclose($file);
My text file sample.txt. I want to exclude the first row from the text file and store the other rows into mysql database.
ID Name EMail
1 Siva xyz#gmail.com
2 vinoth xxx#gmail.com
3 ashwin yyy#gmail.com
Now I want to read this data from the text file except the first row(ID,name,email) and store into the MYsql db.Because already I have created a filed in database with the same name.
I have tried
$handle = #fopen($filename, "r"); //read line one by one
while (!feof($handle)) // Loop till end of file.
{
$buffer = fgets($handle, 4096); // Read a line.
}
print_r($buffer); // It shows all the text.
Please let me know how to do this?
Thanks.
Regards,
Siva R
It's easier if you use file() since it will get all rows in an array instead:
// Get all rows in an array (and tell file not to include the trailing new lines
$rows = file($filename, FILE_IGNORE_NEW_LINES);
// Remove the first element (first row) from the array
array_shift($rows);
// Now do what you want with the rest
foreach ($rows as $lineNumber => $row) {
// do something cool with the row data
}
If you want to get it all as a string again, without the first row, just implode it with a new line as glue:
// The rows still contain the line break, since we only trimmed the copy
$content = implode("\n", $rows);
Note: As #Don'tPanic pointed out in his comment, using file() is simple and easy but not advisable if the original file is large, since it will read the whole thing into memory as an array (and arrays take more memory than strings). He also correctly recommended the FILE_IGNORE_NEW_LINES-flag, just so you know :-)
You can just call fgets once before your while loop to get the header row out of the way.
$firstline = fgets($handle, 4096);
while (!feof($handle)) // Loop till end of file.
{ ...
in "file.txt" i have some numbers like 1234,123456,12345678 etc.
so i was wondering how i can get dynamically just one element, for example echo just 12345,then echo 123456, but one by one ???when user comes i want to show him one element,and then next time i want to show another element on the page, it would be good also if i could erase elements that i have echo...When i manually enter position it works... Please help...
I have a following code:
function test($n){
$file=file_get_contents("file.txt");
$array = explode(",", $file);
return print_r($array[$n]);
};
The solution will need deleting the number on your filesystem or marking it as already showed either on the filesystem or on the database, after the number gets echoed.
Shuffling the numbers or generating a random index will never quite be full proof since the same number could be shown again.
First of all, this is usually a terrible way of doing this.
It's much better to use a database, for example.
But you could do the following:
<?php
function getRandomNumber() {
// retrieve and parse the numbers
$numbers = file_get_contents('file.txt');
$numbers = explode(',' $numbers);
// select a random index
$randomIndex = array_rand($numbers);
// get the chosen number
$randomNumber = $numbers[$randomIndex];
// remove it from array
unset($numbers[$randomIndex]);
// save updated file
file_put_contents('file.txt', implode(',', $numbers));
return $randomNumber;
}
Im using PHP to create a playlist. Two random songs are chosen from a directory, and their name and location are stored in an array and then written to a file via json_encode().
$arraySongs[] = array('name' => $songName , 'mp3' => $webUrl);
This works great. I can make a very long playlist, two songs at a time. Id also like to remove songs, so I have an AJAX powered delete button, that posts the id of the track to be deleted, PHP then loads the whole tracklist...
$decoded = json_decode(file_get_contents($tracklist),true);
and removes the given song from the array, then re encodes and re writes the json text file. This all works great.
The problem comes whenever I try to delete anything with a playlist of more than 10 items.
Typically, my song.json file goes [{name:song1,mp3:song url},{name:song2,mp3:song2 url}]
However, when I have a list of more than 10 items, the re encoded playlist looks like this:
[{ ... },{name:song9,mp3:song9 url}],[10,{"name":song10,mp3:song10 url}]
Why is my re-encoded array get that strange [10,{"name"... [11,{"name"... [12,{"name"...
but everything below 10 is always fine?
Thanks for reading this! Any suggestions would be greatly appreciated, this is driving me nuts!
Here is the code im Using:
<?php
$json = "song.php";
$decoded = json_decode(file_get_contents($json),true);
$playlist = array();
$names = array();
// Now Get i From Javascript
$i=$_POST['id'];
//Select i's Array
$collect_array=$decoded[$i];
while (list ($key, $val) = each ($collect_array)) {
//Remove i's Values
//echo "<br />$key -> $val <br>";
unset($decoded[$i]);
}
//Take all the remaining arrays
$collect_array=$decoded;
while (list ($key, $val) = each ($collect_array)) {
$arraySongs[] = array($key , $val);
}
// Our New Array ready for json.
$jsonData = json_encode($arraySongs);
// open song.php and scribble it down
$tracklist = $json;
$fh = fopen($tracklist, 'w') or die("can't open filename: $tracklist");
fwrite($fh, $jsonData);
fclose($fh);
?>
try removing elements with unset
debug your code (not posted in the thread, so do it yourself) by adding a line where you var_dump or print_r the whole thing before json_encode
or it's a bug in json_encode which would not be nice...
Encode the track ID on 2 or even 3 digits using the php function sprintf with parameter %02d.
This worked fine for me.
I have remade my original post as it was terribly formatted. Basically I would like some advice / tips on how to generate a line graph with 2 Y Axis (temperature and humidity) to display some information from my text file. It is contained in a textfile called temperaturedata.txt I have included a link to one of my posts from the JpGrapher forum only because it is able to display the code clearly.
I understand that since it is JpGraph problem I shouldn't post here however the community here is a lot more supportive and active. Many thanks for all your help guys in advance!
my code
I don't see any reason why you shouldn't post here about jpgraph. And I don't see why you shouldn't post your sample code and data here, either.
The code you've posted on the other site is broken. Check line #42.
Furthermore, you're passing JpGraph a single row (specifically, the last row) via $keyval. $data is where all your data is stored, though in a wrong format. A very quick fix was:
$keyval = array();
$keyval['time'] = array();
$keyval['count'] = array();
$keyval['temperature'] = array();
$keyval['humidity'] = array();
if ($file) {
while (!feof($file)) {
$line = trim(fgets($file));
if (strlen($line)) {
$fields = explode(":", $line);
$keyval['time'][] = $fields[0];
$keyval['count'][] = $fields[1];
$keyval['temperature'][] = $fields[2];
$keyval['humidity'][] = $fields[3];
}
}
fclose($file);
}
which transposed $data and renamed it $keyval. (Where it used to hold time data in $data[x]['time'], it now holds it in $keyval['time'][x].) And we're passing $keyval['temperature'], which is a simple array of temperature values.