Good morning.
I'm currently trying to build a very basic caching system for one of my scripts. The cache is JSON data and contains only 1 key and it's value, but many individual fields, something like this;
{"Item1":"Item1 Description"}
{"Item2":"Item2 Description"}
{"Item3":"Item3 Description"}
What I'm intending to do is;
First check if a cache file is available
Then check if an item exists in the cache
Then add the new item along with it's description if it's not already in the cache...
...or return the item description if it's not there.
All data being stored is strings. The cache file doesn't store any other type of data.
I've put together a basic function but I'm having trouble getting it functioning;
function ItemIsInCache($CacheFile, $ItemId) {
if(file_exists($CacheFile)) {
$json = json_decode(file_get_contents($CacheFile, true));
if(in_array($ItemId, $json)) { // <<
$itemname = array_search($ItemId, $json);
return itemname;
} else {
$item[$itemId] = GrabItemName($ItemId);
$itemname = array_search($ItemId, $json); // <<
return $itemname;
}
} else {
$item[$ItemId] = GrabItemName($ItemId);
$ejson = json_encode($item);
file_put_contents($CacheFile, $ejson);
return $item[$ItemId];
}
}
Notes
GrabItemName is a different function that returns the description data based on the $ItemId.
The warnings I'm getting are Wrong datatype for second argument in both array_search() and in_array(), on lines 4 and lines 9 respectively (those are the line numbers in the above code - due to the nature of my script these numbers are later on) -- for simplicity, I've marked the problem lines with // <<.
The function is running in a loop which I've no problems with. The problems lie within this function.
What currently happens
Right now, if the cache doesn't exist, it creates it and adds the first item from the loop to the cache file in it's respective JSON format (that fires since the cache file doesn't exist, so after the final else statement).
However, items from the loop after that don't get added, presumably because the file exists and there's something wrong with the code.
The last part of the function works exactly as I want it to but the first part does not.
Expected behaviour with fixed code
Check cache > Return description if item exists ELSE add new item to cache.
The items and their associated descriptions will NOT change, but I'm pulling them from a rate limited API, and I need to ensure I cache whatever I can for everyones benefit.
So, any ideas what I'm doing wrong with the function? I'm sure it's something incredibly simple that I'm overlooking.
Your file is not JSON for an erray. The correct JSON for an array is
[
{"Item1":"Item1 Description"},
{"Item2":"Item2 Description"},
{"Item3":"Item3 Description"}
]
You're missing the brackets around the array, so you just get a single object.
When creating the initial file, you need to do:
$ejson = json_encode(array($item));
so that it's initialized as an array of one item, not just an item.
Related
I am trying to parse a large amount of JSON data generated from a remote web service. The output produced is paginated across 500 URIs and each URI contains 100 JSON objects. I need to match a property in each JSON object, it's DOI (a digital object identifier), against a corresponding field fetched from a local database and then update the record.
The issue I am having is controlling my looping constructs to seek out the matching JSON DOI while making sure that all the data has been parsed.
As you can see I have tried to use a combination of break and continue statements but I am not able to 'move' beyond the first URI.
I later introduced a flag variable to help control the loops without effect.
while($obj = $result->fetch_object()){
for($i=1;$i<=$outputs_json['meta']['response']['total-pages'];$i++){
$url = 'xxxxxxxxxxxxxxx&page%5Bnumber%5D='."$i".'&page%5Bsize%5D=100';
if($outputs = json_decode(file_get_contents($url),true)===false){
}
else{
try{
$outputs = json_decode(file_get_contents($url),true);
$j=0;
do{
$flag = false;
$doi = trim($outputs['data'][$j]['attributes']['identifiers']['dois'][0], '"');
if(!utf8_encode($obj->doi)===$doi) continue;
}else{
$flag = true;
$j++;
}
}while($j!==101);
if($flag===true) break;
} catch(Exception $e) {
}
}
}
}
}
What is the optimal approach that guarantees each JSON object at all URIs is parsed and that CRUD operations are only performed on my database when a fetched record's DOI field matches the DOI property of the incoming JSON data?
I'm not 100% sure I understand every aspect of your question but for me it would make sense to change the order of execution
fetch page from external service
decode json and iterate through all 100 objects
get one DOI
fetch corresponding record from database
change db record
when all json-objects are progressed - fetch next url
repeat until all 100 urls are fetched
I think it's not a good idea to fetch one record from local DB and try to find it in 100 different remote calls - instead it's better to base your workflow/loops on fetched remote data and try to find the corresponding elements in your local DB
If you think that approach will fit your task - I can of course help you with the code :)
I'm trying to store multiple data and then at the end go a head and push the data into the new .ini file. I found solutions which works but I want to get all the data first and then update the file at the end but the solutions i found updates the file straight away!
A solution I liked and worked is located: https://stackoverflow.com/a/36997282/6613233
I am trying to allow it gather information and then push it to the file at the end. My own attempt at this is below but i keep getting array array in my ini file.
Code:
$fbsettingsDB = parse_ini_file("location.ini", true);
$fbsettingsDB["id"]["value"] = $_POST['fbconfigid'];
$fbsettingsDB["location"]["value"] = $_POST['fbconfigcty'];
file_put_contents('location.ini', implode("\n", $fbsettingsDB));
The above is how I want to collect data. I have a bunch of code which goes in and out of statements, I want it to go ahead.. Assign the values required and at the very end go ahead and put the contents in the file like shown above.
Using the referred code i would then have to do:
config_set("location.ini", "id", "value", $_POST['fbconfigid']);
config_set("location.ini", "location", "value", $_POST['something']);
config_set("location.ini", "result", "value", $_POST['somethingelse']);
Which overwrites the file every time which in my opinion is just crazy! Overkill for my idea, there is obviously some way that can suit my needs so i can just call the function once after making a list of edit/changes and then when i call the function it grabs all my changed data and saves the file the way i want it!
I'll try to explain first why your code doesn't work, compared to the other.
Your inifile-array is build up of a nested array, $array[section][item] = value. The first dimension has the section names. The second dimension is the name of the items in the sections. So $fbsettingsDB["location"] contains an array of items, of which "value" is one.
Implode doesn't check if the array is nested. It just takes the first dimension (the sections) and tries to treat their values as a string. Since those values are actually arrays of items, PHP just converts that to the text 'array'.
Apart from that, you can't just implode the whole array. Section names should be enclosed in square brackets, so there is a little more work to do in that regard too.
If you check the solution in the answer you referred to, you'll see that it contains a loop which takes care of the first layer, the sections.
The array of items of each section is converted separately with implode, which is then prefixed by the section name in square brackets, and the whole lot is appended to the end result.
So, your intention here: You don't want to set a value and write it back to file at once, but update multiple values and only write the end result to disk. Well, fortunately the function doesn't have to be atomic. It already performs three separate actions: loading from disk, modifying the data, and serializing it back to disk. Let's see if those can be isolated in separate functions:
Read the data. Well, hardly worth to make a function, but it may make your application somewhat more consistent if you use the same naming et cetera in a collection of related functions.
Note: I just wrote these from scratch. No PHP at hand to test, so they might contain minor syntactical errors.
So here it is:
// Loads ini file data
function config_read($config_file) {
return parse_ini_file($config_file, true);
}
Setting the config in the loaded data. Again, hardly worth to have a function, but it adds readability and hides how exactly the ini file data is built up, so you don't have to worry about implementation details when using it. Note that the array is passed by reference. The array you specify is updated. The function doesn't return a value.
// Update a setting in loaded inifile data
function config_set(&$config_data, $section, $key, $value) {
$config_data[$section][$key] = $value;
}
Then writing it:
// Serializes inifile config data back to disk.
function config_write($config_data, $config_file) {
$new_content = '';
foreach ($config_data as $section => $section_content) {
$section_content = array_map(function($value, $key) {
return "$key=$value";
}, array_values($section_content), array_keys($section_content));
$section_content = implode("\n", $section_content);
$new_content .= "[$section]\n$section_content\n";
}
file_put_contents($config_file, $new_content);
}
Note that so far I didn't modify any of the code. I just wrapped it in separate functions. If you like, you could even call those functions in another function, so you still got the shorthand to write everything to disk at once. You'll have the original functionality, but without having duplicate code:
// Short-hand function for updating a single config value in a file.
function config_set_file($config_file, $section, $key, $value) {
$config_data = config_read($config_file);
config_set($config_data, $section, $key, $value)
config_write($config_file, $section, $key, $value);
}
So, now you got this collection of functions, you can decide which to use based on the situation. If you just want to update a single value, you might as well write this:
config_set_file("location.ini", "id", "value", $_POST['fbconfigid']);
But if you have multiple configs to set, you can do this:
// Load
$config_data = config_read("location.ini");
// Set multiple values
config_set($config_data, "id", "value", $_POST['fbconfigid']);
config_set($config_data, "location", "value", $_POST['something']);
config_set($config_data, "result", "value", $_POST['somethingelse']);
// Save
config_write($config_data, $config_file);
I can imagine you can add other shorthands, like config_set_array_file, which you could call like this.. I'll leave the implementation of this one to you for exercise. ;)
array_config_set_file($config_file, array(
"id" => $_POST['fbconfigid'],
"location" => $_POST['something'],
"result" => $_POST['somethingelse']));
And after that, you can poor all this into an IniFile class to make it even nicer. :)
I am having trouble figuring out how to correct an issue I am having with the following code. I am trying to list the names and emails of all the people in my Active Directory. This code works. However, I also get the following warning when it is executed.
Warning: Cannot use a scalar value as an array.
From what I have read online I need to set " $name['mail']['0'] = "Not Found";" to an array. My question is how would I go about doing this. I have tried every way I could think of with no success. If anyone could provide me with some feedback it would be greatly appreciated.
foreach ($results as $name) {
if (!isset($name['mail']['0'])){
$name['mail']['0'] = "Not Found";
}
$allnames[$name['cn']['0']]['mail'] = $name['mail']['0'];
It looks like you are trying to assign the "Not Found" value BACK into the results of the ldap query you are running.
When I wrote code to get user information from ldap, I always inserted values into a different array (getting the fields I wanted) and then displayed THAT array out in the HTML:
foreach ($results as $name)
{
if (isset($name['mail']['0']))
// Check if it IS set, rather than not set and then append the data.
{
$allnames[$name['cn']['0']]['mail'] = $name['mail']['0'];
}
}
Then after you have all the fields you want you display the $allnames array.
Having said that, when I construct the array of user information, I make the array much much simpler - knowing what I want to display out, more along the lines of this:
foreach ($results as $name)
{
if (isset($name['mail']['0']))
// Check if it IS set, rather than not set and then append the data.
{
$allnames['mail'] = $name['mail']['0'];
}
}
You (or at least I) don't need to follow the ldap structure when making the array of information I have gathered, but rather I want to get specific fields from the directory and then display them by what they are - for example when I output the value in mail I might want to hyperlink it as an email address, so I keep it in an array element that I know by key and refer to later.
I have this while loop, that basically loops through a lot of records in a database, and inserts the data in another:
$q = $con1->query($users1) or die(print_r($con2->errorInfo(),1));
while($row = $q->fetch(PDO::FETCH_ASSOC)){
$q = $con2->prepare($users2);
$q->execute(array($row['id'], $row['username'])) or die(print_r($con2-errorInfo(),1));
}
(The script has been shortened for easy reading - the correct one has a much longer array)
I would like to do this more graphical, and show a progress bar on how far it has went, instead of just seeing a page loading for a few minutes (there are ~20.000 rows in this one - I have tables with much more data)
I get that you could get the total number from the old database, and I could also easily put the current number into a variable like this:
$q = $con1->query($users1) or die(print_r($con2->errorInfo(),1));
$i = 0;
while($row = $q->fetch(PDO::FETCH_ASSOC)){
$q = $con2->prepare($users2);
$q->execute(array($row['id'], $row['username'])) or die(print_r($con2-errorInfo(),1));
$i++;
}
But now I need to actually fetch $i and display it - or something like it.
How is this "easily" done?
The code for the progress bar can either be in the same document as the while loop, or in another if easier.
You can do a "master" file that does an ajax to this first file to run a single query. You could get all the entry id's in this master file, and then pass it as a parameter to the second file that does a single query. Store these ids in a javascript array.
Create a function that does this, and when the first ajax is done, move to the second element of the id array, and do another ajax with a second parameter. That's how magento imports are done by the way :)
If you need further explanations, let me know, I tried my best to explain, but may have not been perfectly clear.
// you generate this javascript array using php.
// let's say you have all the ids that have to be processed in $Ids php array.
Ids = [<?php echo implode(',', $Ids); ?>];
function doAjax(i) {
$.ajax({ // using jquery for simplicity
'url': "ajax.php?id=" + Ids[i],
}).done(function(){
if ( i >= 0 ) {
// at the point you know you're at ((Ids.length-i)/(Ids.length) * 100) percent of the script
// so you can do something like this:
// $('.progressbar').css('width', ((Ids.length-i)/(Ids.length) * 100) + '%');
doAjax(i-1);
}
});
}
doAjax(Ids.length); // starting from the last entry
So, just to explain what this does. It starts by declaring a global javascript array that has all the ids that will need to be changed.
Then I declare a recursive ajax function, this way we can make sure that only one ajax runs at any single time (so the server doesn't blow up), and we can have a fairly accurate progress. This ajax function does the following:
Sends a request to ajax.php?id=xxx - where xxx is one of the ids in the javascript array.
In the file, we get the id ($_GET['id']), you take it from the old database, and insert it in the new one. This is only for one entry.
when the ajax is done, it goes to the done() function. Since we start the doAjax() function with the last element, we do the next iteration doAjax(i-1). Since we're going backwards in the array, we check if the key is positive. If it's not, the script will stop.
That's about it.
You can't. The php is first interpreted by the server and then send to the user as HTML-Code.
The only possibility would be creating a html-page and call the php-script with AJAX.
I have this while loop, that basically loops through a lot of records in a database, and inserts the data in another:
$q = $con1->query($users1) or die(print_r($con2->errorInfo(),1));
while($row = $q->fetch(PDO::FETCH_ASSOC)){
$q = $con2->prepare($users2);
$q->execute(array($row['id'], $row['username'])) or die(print_r($con2-errorInfo(),1));
}
(The script has been shortened for easy reading - the correct one has a much longer array)
I would like to do this more graphical, and show a progress bar on how far it has went, instead of just seeing a page loading for a few minutes (there are ~20.000 rows in this one - I have tables with much more data)
I get that you could get the total number from the old database, and I could also easily put the current number into a variable like this:
$q = $con1->query($users1) or die(print_r($con2->errorInfo(),1));
$i = 0;
while($row = $q->fetch(PDO::FETCH_ASSOC)){
$q = $con2->prepare($users2);
$q->execute(array($row['id'], $row['username'])) or die(print_r($con2-errorInfo(),1));
$i++;
}
But now I need to actually fetch $i and display it - or something like it.
How is this "easily" done?
The code for the progress bar can either be in the same document as the while loop, or in another if easier.
You can do a "master" file that does an ajax to this first file to run a single query. You could get all the entry id's in this master file, and then pass it as a parameter to the second file that does a single query. Store these ids in a javascript array.
Create a function that does this, and when the first ajax is done, move to the second element of the id array, and do another ajax with a second parameter. That's how magento imports are done by the way :)
If you need further explanations, let me know, I tried my best to explain, but may have not been perfectly clear.
// you generate this javascript array using php.
// let's say you have all the ids that have to be processed in $Ids php array.
Ids = [<?php echo implode(',', $Ids); ?>];
function doAjax(i) {
$.ajax({ // using jquery for simplicity
'url': "ajax.php?id=" + Ids[i],
}).done(function(){
if ( i >= 0 ) {
// at the point you know you're at ((Ids.length-i)/(Ids.length) * 100) percent of the script
// so you can do something like this:
// $('.progressbar').css('width', ((Ids.length-i)/(Ids.length) * 100) + '%');
doAjax(i-1);
}
});
}
doAjax(Ids.length); // starting from the last entry
So, just to explain what this does. It starts by declaring a global javascript array that has all the ids that will need to be changed.
Then I declare a recursive ajax function, this way we can make sure that only one ajax runs at any single time (so the server doesn't blow up), and we can have a fairly accurate progress. This ajax function does the following:
Sends a request to ajax.php?id=xxx - where xxx is one of the ids in the javascript array.
In the file, we get the id ($_GET['id']), you take it from the old database, and insert it in the new one. This is only for one entry.
when the ajax is done, it goes to the done() function. Since we start the doAjax() function with the last element, we do the next iteration doAjax(i-1). Since we're going backwards in the array, we check if the key is positive. If it's not, the script will stop.
That's about it.
You can't. The php is first interpreted by the server and then send to the user as HTML-Code.
The only possibility would be creating a html-page and call the php-script with AJAX.