Delete multiple files in MongoDB GridFS in PHP - php

I am using MongoDB and GridFS in PHP, and trying to figure out how to delete multiple files by the _id.
Here is the code I have:
$ids = array("50401f40ff558cec38000061", "62401f40ff558cec38000072", "73401f40ff558cec38000083");
$mongo_ids = array();
foreach($ids as $id) {
$mongo_ids[] = new MongoId($id);
}
$mongo_grid_fs->remove(array("_id" => $mongo_ids));
Any idea what I am doing wrong?

This impossible to do with a single request due to the way that GridFS actually works.
You have two collections:
Files
Chunks
Inorder to delete a GridFs file you must query BOTH of these table. As such the remove() function actually calls the chunk collection and then removes the file from the files collection.
Since MongoDB cannot, fundamentally, query two collections in one request (joined deleted basically) you must send a delete request per file to delete otherwise you will have left over chunks taking up space in your chunks collection.
As such, taking this into consideration #ToddMoses answer is the correct one.
You can of course use: http://www.php.net/manual/en/mongogridfs.remove.php but I believe it does exactly the same thing, just abstracted so your query should have been:
$mongo_grid_fs->remove(array("_id" => array('$in' => $mongo_ids)));

First, use MongoDB::lastError() to find out what is going wrong. MongoGridFS::remove won’t present you with a message if it fails. DO something like this:
$errorArray = $db->lastError();
var_dump($errorArray);
It appears the problem is that you are not setting the criteria properly. The easiest thing to do is just use Delete instead of Remove since Delete takes an ID as its only parameter:
public bool MongoGridFS::delete ( mixed $id )
This deletes a file from the database whereas remove deletes files from the collection. Since you are looping anyway, you can do something like this:
foreach($ids as $id) {
$mongo_grid_fs->delete($id);
}

Related

SugarCRM 6.5.26 CE - contacts export using SugarBean [php]

I've got SugarCrm plugin which is exporting data to external service. I'm using logic hooks for updated/deleted/new Contacts, but I've got problem with synchronizing already existing data. I have to extract all the data from the SugarCRM and there are two SugarBean methods I've tried to use: get_full_list() and get_list(). First one gives me the full Contact list, but I need to send it in batches 1000 Contacts in one Json max, the second method returns only first page of the Contacts (depends on config settings 10 - 1000max entries).
I'm using this method ATM:
// prepare contacts data from SugarBean
$bean = BeanFactory::getBean($module);
$contactResults = $bean->get_full_list();
Then foreach on $contactResults and save the data I want to the required format and send it as a Json via postrequest. I've tried to find the solution to split it into batches, but Im stuck :( Neither get_full_list or get_list seems to work for me.
Any suggestions? Maybe someone solved this issue already?
Thanks in advance!
It sounds to me like your problem is creating batches? If not please be more specific about what isn't working.
For splitting an array into batches, you may want to have a look at https://php.net/manual/en/function.array-chunk.php
Also get_list supports retrieving later pages. It is defined like this: function get_list($order_by = "", $where = "", $row_offset = 0, $limit=-1, $max=-1, $show_deleted = 0, $singleSelect=false, $select_fields = array()).
That means for the second page you could specify $row_offset = 1000, for the third page make it 2000, etc. So basically run a loop that calls get_list with $limit = 1000 and increases an initial $row_offset of 0 by 1000 after each iteration, until less than 1000 records or null is returned by the function.
Here are some general hints if you run into problems with processing those beans:
If the problem you're having is incomplete data, try loading each bean manually by using its ID. Some Sugar functions don't load all (special) fields by default.
If things seem to just fail for no reason, make sure to check your PHP log for errors. Maybe loading as many beans at once could possibly cause problems with your PHP's max_execution_time or memory_limit.

Writing values to INI file

I'm trying to store multiple data and then at the end go a head and push the data into the new .ini file. I found solutions which works but I want to get all the data first and then update the file at the end but the solutions i found updates the file straight away!
A solution I liked and worked is located: https://stackoverflow.com/a/36997282/6613233
I am trying to allow it gather information and then push it to the file at the end. My own attempt at this is below but i keep getting array array in my ini file.
Code:
$fbsettingsDB = parse_ini_file("location.ini", true);
$fbsettingsDB["id"]["value"] = $_POST['fbconfigid'];
$fbsettingsDB["location"]["value"] = $_POST['fbconfigcty'];
file_put_contents('location.ini', implode("\n", $fbsettingsDB));
The above is how I want to collect data. I have a bunch of code which goes in and out of statements, I want it to go ahead.. Assign the values required and at the very end go ahead and put the contents in the file like shown above.
Using the referred code i would then have to do:
config_set("location.ini", "id", "value", $_POST['fbconfigid']);
config_set("location.ini", "location", "value", $_POST['something']);
config_set("location.ini", "result", "value", $_POST['somethingelse']);
Which overwrites the file every time which in my opinion is just crazy! Overkill for my idea, there is obviously some way that can suit my needs so i can just call the function once after making a list of edit/changes and then when i call the function it grabs all my changed data and saves the file the way i want it!
I'll try to explain first why your code doesn't work, compared to the other.
Your inifile-array is build up of a nested array, $array[section][item] = value. The first dimension has the section names. The second dimension is the name of the items in the sections. So $fbsettingsDB["location"] contains an array of items, of which "value" is one.
Implode doesn't check if the array is nested. It just takes the first dimension (the sections) and tries to treat their values as a string. Since those values are actually arrays of items, PHP just converts that to the text 'array'.
Apart from that, you can't just implode the whole array. Section names should be enclosed in square brackets, so there is a little more work to do in that regard too.
If you check the solution in the answer you referred to, you'll see that it contains a loop which takes care of the first layer, the sections.
The array of items of each section is converted separately with implode, which is then prefixed by the section name in square brackets, and the whole lot is appended to the end result.
So, your intention here: You don't want to set a value and write it back to file at once, but update multiple values and only write the end result to disk. Well, fortunately the function doesn't have to be atomic. It already performs three separate actions: loading from disk, modifying the data, and serializing it back to disk. Let's see if those can be isolated in separate functions:
Read the data. Well, hardly worth to make a function, but it may make your application somewhat more consistent if you use the same naming et cetera in a collection of related functions.
Note: I just wrote these from scratch. No PHP at hand to test, so they might contain minor syntactical errors.
So here it is:
// Loads ini file data
function config_read($config_file) {
return parse_ini_file($config_file, true);
}
Setting the config in the loaded data. Again, hardly worth to have a function, but it adds readability and hides how exactly the ini file data is built up, so you don't have to worry about implementation details when using it. Note that the array is passed by reference. The array you specify is updated. The function doesn't return a value.
// Update a setting in loaded inifile data
function config_set(&$config_data, $section, $key, $value) {
$config_data[$section][$key] = $value;
}
Then writing it:
// Serializes inifile config data back to disk.
function config_write($config_data, $config_file) {
$new_content = '';
foreach ($config_data as $section => $section_content) {
$section_content = array_map(function($value, $key) {
return "$key=$value";
}, array_values($section_content), array_keys($section_content));
$section_content = implode("\n", $section_content);
$new_content .= "[$section]\n$section_content\n";
}
file_put_contents($config_file, $new_content);
}
Note that so far I didn't modify any of the code. I just wrapped it in separate functions. If you like, you could even call those functions in another function, so you still got the shorthand to write everything to disk at once. You'll have the original functionality, but without having duplicate code:
// Short-hand function for updating a single config value in a file.
function config_set_file($config_file, $section, $key, $value) {
$config_data = config_read($config_file);
config_set($config_data, $section, $key, $value)
config_write($config_file, $section, $key, $value);
}
So, now you got this collection of functions, you can decide which to use based on the situation. If you just want to update a single value, you might as well write this:
config_set_file("location.ini", "id", "value", $_POST['fbconfigid']);
But if you have multiple configs to set, you can do this:
// Load
$config_data = config_read("location.ini");
// Set multiple values
config_set($config_data, "id", "value", $_POST['fbconfigid']);
config_set($config_data, "location", "value", $_POST['something']);
config_set($config_data, "result", "value", $_POST['somethingelse']);
// Save
config_write($config_data, $config_file);
I can imagine you can add other shorthands, like config_set_array_file, which you could call like this.. I'll leave the implementation of this one to you for exercise. ;)
array_config_set_file($config_file, array(
"id" => $_POST['fbconfigid'],
"location" => $_POST['something'],
"result" => $_POST['somethingelse']));
And after that, you can poor all this into an IniFile class to make it even nicer. :)

Accessing a model from another unrelated model takes a very long time

I have a query in a CakePHP 3.0 table with a formatResults() method applied to it.
In order to carry out this calculation, data is required from a table in a different database (Currencies).
I am currently using this code to gather the data and pass it to the function:
$currencies = TableRegistry::get('Currencies');
$currencyValues = $currencies
->findByCurrno($options['currency'])
->cache('currency'.$options['currency']);
$currencyValues = $currencyValues->first()->toArray()
$query->formatResults(function ($results) use ($currencyValues) {
return $results->map(function($row) use ($currencyValues) {
$row['converted_earnings'] = $row['earned'] / $currencyValues['cur'.$row['currency']];
$row['converted_advances'] = $row['advances'] / $currencyValues['cur'.$row['currency']];
return $row;
});
});
The problem is, this code seems to take a very long time to execute, even though it is only iterating through a few hundred rows of data.
Further investigation revealed that if I do not collect the data from the 'currencies' table and instead declare $currencyValues as an array with fixed numbers the code takes a full second less to execute.
Through commenting out parts of the code, I have isolated this to be the cause of the problem:
$currencyValues = $currencies
->findByCurrno($options['currency'])
->cache('currency'.$options['currency']);
If I remove this part of the code then everything runs quickly, and as soon as I add it in (even if I do not use the data it returns and use the static array) the page takes far longer to load. It should be noted that this problem occurs whether or not I use the ->cache() method, and that the query itself reports that it takes 0-1ms in the sql dump.
My question is - why is this slowing down my execution so much and how can I stop it? It is pretty much a requirement that I use the data from this table for the operation, so I am looking for a way to speed it up.

Have CakePHP saveAll function to ignore errors and continue

Trying to make it simple, I have an excel spreadsheet of people and their emails. What I want is to use saveAll to insert them all at once in the database.
Problem is that I need to re-upload the spreadsheet everytime it has some new people in it. In that case, I need saveAll to save all the new people and ignore the validation errors that will happen due to the old people already existing in the database (people emails have unique rule). Any ideas?
EDIT: What I have now is that, when the saveAll method tries to save a record that already exists (i.e. whose email is already in the database) it doesn't save anything. What I need is to save the new ones that come from the spreadsheet, and ignore the ones that already exist (i.e. not save the ones that already exist).
Set the second parameter to false so that validation is not checked.
Given your requirements, I would do this:
Loop through the data and check if that particular record exist, if it doesn't then push it to a temp array.
Now you can use that temp array as a parameter for your saveAll call.
The pseudo code would be something like this:
foreach ($originalDataToBeSaved as $something) {
if ($this->find('count', array('conditions' => $yourConditions)) == 0) {
$tempArray[] = $something;
}
}
$this->saveAll($tempArray);
Edit: I updated the code to reflect what Nunser said, new data doesn't have ID thus we need to do our search using another search criteria.
try this
foreach ($originalDataToBeSaved as $key => $something) {
if ($this->hasAny($your_condition_to_chekc_existing_user) {
unset($originalDataToBeSaved[$key]);
}
}
$this->saveAll(($originalDataToBeSaved);
if you dont want to validate then use -
$this->saveAll(($originalDataToBeSaved, array('validate' => false));

PHPFlickr script...could this be cleaner/leaner?

OK, here's my dilemma:
I've read all over about how many guys want to be able to display a set of images from Flickr using PHPFlickr, but lament on how the API for PhotoSets does not put individual photo descriptions. Some have tried to set up their PHP so it will pull the description on each photo as the script assembles the gallery on the page. However, the method has shown how slow and inefficient it can be.
I caught an idea elsewhere of creating a string of comma separated values with the photo ID and the description. I'd store it on the MySQL database and then call upon it when I have my script assemble the gallery on the page. I'd use explode to create an array of the photo ID and its description, then call on that to fill in the gaps...thus less API calls and a faster page.
So in the back-end admin, I have a form where I set up the information for the gallery, and I hand a Set ID. The script would then go through and make this string of separated values ("|~|" as a separation). Here's what I came up with:
include("phpFlickr.php");
$f = new phpFlickr("< api >");
$descArray = "";
// This will create an Array of Photo ID from the Set ID.
// $setFeed is the set ID brought in from the form.
$photos = $f->photosets_getPhotos($setFeed);
foreach ($photos['photoset']['photo'] as $photo) {
$returnDesc = array();
$photoID = $photo['id'];
$rsp = $f->photos_getInfo($photoID);
foreach ($rsp as $pic) {
$returnDesc[] = htmlspecialchars($pic['description'], ENT_QUOTES);
}
$descArray .= $photoID."|~|".$returnDesc[0]."|~|";
}
The string $descArray would then be placed in the MySQL string that puts it into the database with other information brought in from the form.
My first question is was I correct in using a second foreach loop to get those descriptions? I tried following other examples all over the net that didn't use that, but they never worked. When I brought on the second foreach, then it worked. Should I have done something else?
I noticed the data returned would be two entries. One being the description, and the other just an "o"...hence the array $returnDesc so I could just get the one string I wanted and not the other.
Second question is if I made this too complicated or not. I like to try to learn to write cleaner/leaner code, and was looking for opinions.
Suggestions on improvement are welcome. Thank you in advance.
I'm not 100% sure as I've just browsed the source for phpFlickr, and looked the the Flickr API for the getInfo() call. But let me have a go anyway :)
First off, it looks like you shouldn't need that loop, like you mention. What does the output of print_r($rsp); look like? It could be that $rsp is an array with 1 element, in which case you could ditch the inner loop and replace it with something like $pic = $rsp[0]; $desc = $pic['description'];
Also, I'd create a new "description" column in your database table (that has the photo id as the primary key), and store the description in their on its own. Parsing db fields like that is a bit of a nightmare. Lastly, you might want to force htmlspecialchars to work in UTF8 mode, cause I don't think it does by default. From memory, the third parameter is the content encoding.
edit: doesn't phpFlickr have its own caching system? Why not use that and make the cache size massive? Seems like you might be re-inventing the wheel here... maybe all you need to do is increase the cache size, and make a getDescription function:
function getDescription ($id)
{
$rsp = $phpFlickr->photos_getInfo ($id);
$pic = $rsp[0];
return $pic['description'];
}

Categories