I am a little confused. I actively use PHP RedBean as ORM within my direct mail service and I run into curious situation - I have a table with unique key constraint (i.e. subscriber_id, delivery_id) and two scripts that is writing data into this table.
There is source code that is inserting or updating table:
public static function addOpenPrecedent($nSubscriberId, $nDeliveryId)
{
$oOpenStatBean = \R::findOrDispense('open_stat', 'delivery_id = :did AND subscriber_id = :sid', array(':did' => $nDeliveryId, ':sid' => $nSubscriberId));
$oOpenStatBean = array_values($oOpenStatBean);
if (1 !== count($oOpenStatBean)) {
throw new ModelOpenStatException(
"Ошибка при обновлении статистики открытий: пара (delivery_id,
subscriber_id) не является уникальной: ($nDeliveryId, $nSubscriberId).");
}
$oOpenStatBean = $oOpenStatBean[0];
if (!empty($oOpenStatBean->last_add_dt)) {
$oOpenStatBean->precedent++;
} else {
$oOpenStatBean->delivery_id = $nDeliveryId;
$oOpenStatBean->subscriber_id = $nSubscriberId;
}
$oOpenStatBean->last_add_dt = time('Y-m-d H:i:s');
\R::store($oOpenStatBean);
}
It is called both from two scripts. And I have issues with corruption unique constraint on this table periodically, because race conditions occurs. I know about SQL "INSERT on duplicate key update" feature. But how can I obtain same result purely using my ORM?
Current, that I know if, Redbean will not issue an
INSERT ON DUPLICATE KEY UPDATE
as the discussion of this cited in the comments above indicates that Redbean's developer considers upsert to be a business logic thing that would pollute the ORM's interphase. This being said, it is most likely achievable if one were to extend Redbean with a custom Query Writer or plugin per the Documentation. I haven't tried this because the method below easily achieves this behavior without messing with the internals and plugins of the ORM, however, it does require that you use transactions and models and a couple of extra queries.
Basically, start your transaction with either R::transaction() or R::begin() before your call to R::store(). Then in your "FUSE"d model, use the "update" FUSE method to run a query that checks for duplication and retrieves the existing id while locking the necessary rows (i.e. SELECT FOR UPDATE). If no id is returned, you are good and just let your regular model validation (or lack thereof) continue as usual and return. If an id is found, simply set $this->bean->id to the returned value and Redbean will UPDATE rather than INSERT. So, with a model like this:
class Model_OpenStat extends RedBean_SimpleModel{
function update(){
$sql = 'SELECT * FROM `open_stat` WHERE `delivery_id`=? AND 'subscriber_id'=? LIMIT 1 FOR UPDATE';
$args = array( $this->bean->deliver_id, $this->bean->subscriber_id );
$dupRow = R::getRow( $sql, $args );
if( is_array( $dupRow ) && isset( $dupRow['id'] ) ){
foreach( $this->bean->getProperties() as $property => $value ){
#set your criteria here for which fields
#should be from the one in the database and which should come from this copy
#this version simply takes all unset values in the current and sets them
#from the one in the database
if( !isset( $value ) && isset( $dupRow[$property] ) )
$this->bean->$property = $dupRow[$property];
}
$this->bean->id = $dupId['id']; #set id to the duplicates id
}
return true;
}
}
You would then modify the R::store() call like so:
\R::begin();
\R::store($oOpenStatBean);
\R::commit();
or
\R::transaction( function() use ( $oOpenStatBean ){ R::store( $oOpenStatBean ); } );
The transaction will cause the "FOR UPDATE" clause to lock the found row or, in the event that no row was found, to lock the places in the index where your new row will go so that you don't have concurrency issues.
Now this will not solve one user's update of the record clobbering another, but that is a whole different topic.
Related
I am trying to add caching to an existing Symfony project.
But I am unsure how best to proceed
I get an array with ids. I check every id if there is an item in the cache. If not, then break and send a query to the database.
Then I'll get the result and now I can store these values in my cache. But I have no idea how to set the id as a key
public function getHelpfulResult(array $reviewIds): array
{
$helpfulCache = new FilesystemAdapter("helpful", 2 * 60 * 60, "cache");
$helpfulValues = [];
foreach ($reviewIds[0] as $id) {
$item = $helpfulCache->getItem((string)$id);
if($item->isHit()) {
array_push($helpfulValues, $item);
} else {
break;
}
}
$repo = $this->getDoctrine()->getRepository('Project:Test\Helpful', 'reviews');
$query = $repo->createQueryBuilder('helpful')
->where("helpful.parentId IN (:parentIds) AND helpful.type = 'review'")
->setParameter('parentIds', $reviewIds)
->getQuery();
$result = $query->getResult();
foreach($result as $item) {
$cache->set($item);
}
$cache->save();
return $result;
}
Regarding the basic: "how do I set the 'id' for a cache item" question:
You basically don't. You try to retrieve a cache item by an ID, and then the object you have already has that ID, was it hit or not.
You save it, and a new cache item with ID "x" exists now.
E.g.
$cacheItem = $cacheyAdapter->getItem('some-arbitrary-id');
if (!$cacheItem->isHit()) {
$cacheItem->set('some new value');
$cacheAdapter->save($cacheItem);
}
With the above code you'd check if some-arbitrary-id existed in the cache layer, and if it didn't set a value for it and stored it back.
Regarding the rest of the code in the question, I'm afraid the whole thing is rather poorly designed, with a single element being stale on the cache forcing you to make the whole query again and retrieve everything from the DB.
I'm not going to spend time on that, since it's out of scope for this question (and too broad and opinionated for SO), some basic ideas:
Serialize the result of the whole query and cache that
Exclude the ids from you hit on the cache from the second query (remove them from the array $reviewIds). Also, keep the non-hit cache items in another array so that when you get them from the DB you update those in the cache layer. Finally merge results from cache and DB before returning.
I am trying to implement the "Edit Application Settings" feature. After a bit of thinking, my configuration values are stored in the DB with key -> value structure, like this:
id
key
value
1
logo_path
img/logo.png
As you can see, for each setting, there is only a key & value column. I made an App Service provider to cache them forever, and a helper function (config('setting_key')) to get the value, but now I'd like to update it in the most efficient way.
The user interface consists of the <form action="post" ...> and input with a corresponding name, like this: <input name="setting_key_name" ... />. As you can see, the name attribute here has the value of the key column value and the actual value of the input would be the value column value (a bit of confusion here).
First thing that came to my mind, was to make a foreach loop and find & update every row in DB, but IMHO it is very unoptimized way, cause if the page has a form with 10 values, it is 10 SQL queries. But till now, this is what I've done:
$keys = collect($request->except('_token'))->keys()->toArray();
// get all settings if the key name matches the request's input name
$setting = Setting::whereIn('key', $keys)->get();
$logo = self::GENERAL_APP_LOGO; // contant with a key-name (general_application_logo);
if($request->has(self::GENERAL_APP_LOGO) && $request->$logo) {
// Processing uploaded image here;
$this->uploadLogo($image, self::LOGO_IMAGE_PATH, $name); // Using an upload trait
$setting->where('key', $logo)->value = self::LOGO_IMAGE_PATH . $name; // just a try to update the DB this way
}
foreach ($keys as $key) {
$setting->where('key', $key)->value = $request->$key; // putting all request's input values to corresponding key
}
$setting->save(); // saving the DB.
As you can see, this won't work and will throw an Exception, like Call to undefined method ...\Eloquent\Builder::save(). I tried the same code with an update, but the difficult part here is to update it multiple times (since the if section should have the update as well, for the logo), as well as binding the key to value.
So, a little bit of your help would be appreciated - what the logic should be here? How can I update a DB rows with corresponding column's value? I mean - like this (update where key = 'general_app_name' set value, 'some_setting_value'), but using the optimized and clear way?
Working solution
As #miken32 stated in his answer, I used hid version of code, but with slight changes:
// Changed the $request->settings->keys() to PHP native method array_keys():
$settings = Settings::whereIn('key', array_keys($request->settings))->get()->groupBy('id');
// Also, here I changed the `whereIn('id', ...)` to `whereIn('key', ...)`, since it was my primary index.
foreach ($request->settings as $k=>$v) {
if ($k === self::GENERAL_APP_LOGO_ID) {
// not sure about this one, but I think this is
// how you'd access a file input in an array
$image = $request->file('settings')[$k];
$this->uploadLogo($image, self::LOGO_IMAGE_PATH, $name);
$v = self::LOGO_IMAGE_PATH . $name;
}
// take the Setting object out of the list we pulled
// Here I added the ->first() to get the first element from the retrieved collection;
$setting = $settings->get($k)->first();
$setting->value = $v;
$setting->save();
}
Since I was fetching the configuration values via helper, that only returns the value of the current key (and no id column), I changed the id to key and made the key as my PK in a model. Works like a charm!
With each setting in a separate row, there's no way to avoid multiple database queries – one to get the current values for all settings, and other to update each one. Looking up items by primary key is more efficient, so I'd recommend putting the contents of the id column in your blade view, like this:
<label for="setting_{{$setting->id}}">{{$setting->key}}</label>
<input name="settings[{{$setting->id}}]" id="setting_{{$setting->id}}" value="{{$setting->value}}"/>
Now in your controller, $request->settings will be an array you can loop through. You can continue treating your file upload separately, but now you've got the id column to look up, so change your constant to that.
$settings = Settings::whereIn('id', $request->settings->keys())->get()->groupBy('id');
foreach ($request->settings as $k=>$v) {
if ($k === self::GENERAL_APP_LOGO_ID) {
// not sure about this one, but I think this is
// how you'd access a file input in an array
$image = $request->file('settings')[$k];
$this->uploadLogo($image, self::LOGO_IMAGE_PATH, $name);
$v = self::LOGO_IMAGE_PATH . $name;
}
// take the Setting object out of the list we pulled
$setting = $settings->get($k);
$setting->value = $v;
$setting->save();
}
Note that Laravel does offer methods to bulk-update multiple models at once, but they are doing separate queries to the database in the background. IIRC, the save() method doesn't do anything if the value hasn't changed, which will spare you some hits.
You could try creating a text field, or a json field if your database supports it, and storing all of your settings as a JSON string in that field.
id
settings
1
{ "logo_path" : "img/logo.png", "foo" : "bar", "thing_count" : 17 }
2
{ "logo_path" : "img/logo2.png", "foo" : "baz", "thing_count" : 4 }
In your Laravel model, you can cast it as an array
protected $casts = ["settings" => "array"];
and then use it from the model
echo $theModel->settings['logo'];
echo $theModel->settings['foo'];
or you can cast it as a fully fledged object if you need to using value object casting.
One gotcha that can be confusing for people is the setting of the values in the array to update it. This will not work:
$theModel->settings['foo'] = "boz";
The reason is due to the way the Laravel mutators work. Instead, you make a value copy of the settings, change that, and reassign it to the model:
$settings = $theModel->settings;
$settings['foo'] = "boz";
$theModel->settings = $settings;
This approach has the capacity to infinitely expandable in the future as you just add new keys to your json. Be sure to do checks on the settings array to ensure fields you are looking for are set (which is why value objects can be very handy to do validation).
It also solves your database query problem - it's only ever one.
You don't need to put
$setting->where('key', $logo)->value = ...;
Just call
$setting->where('key', $logo)->update($request->toArray());
$setting->save(); called when you instantiated setting class like :
$setting = new Setting();
Or
$setting = Setting::whereIn('key', $keys)->get()->first();
Then
$setting->val = ...;
$setting->save(); // then it work's
I am facing a problem. I have to update a field in database by using cron function in moodle. I am using update query in cron function to update value. But It doesn't work. I am using this function to update value:
function activitysetmodule_cron ()
{
global $CFG, $DB;
$DB->update_record("activitysetmodule",)
$sql="update {$CFG->prefix}activitysetmodule as asm set status = 1 where exists (select 1 from {$CFG->prefix}course_modules as cm where (module=asm.activityset OR module=asm.activityset2 ) AND completion=1 AND asm.course =cm.course ");
return true;
}
Please help to sought it out.
Take a look at the documentation https://docs.moodle.org/dev/Data_manipulation_API#Updating_Records
$DB->update_record takes 2 params, the name of the table to update the record in and an object containing the updated data.
e.g.
$obj = new stdClass();
$obj->id = $id_of_object_to_update;
$obj->status = 1;
$DB->update_record('tablename', $obj);
It looks like you should refactor your code to get a list of records to update, then call $DB->update_record on each in turn (or $DB->set_field, if there is only one field to update). Alternatively, you could use the $DB->execute($sql) function to directly run some SQL on the server, e.g.
$DB->execute("UPDATE {activitysetmodule} asm SET status = 1 WHERE EXISTS (SELECT 1 FROM {course_modules} cm WHERE (module=asm.activityset OR module=asm.activityset2 ) AND completion=1 AND asm.course = cm.course)");
Note the use of {tablename} rather than {$CFG->prefix}tablename and the removal of the 'AS' keyword, as that is not allowed on all DB engines.
Note also, if you haven't done so already, turning on debugging (http://docs.moodle.org/en/Debugging) will give you much more helpful error messages.
I have run into a problem I have never seen before. The code is too much and large and too many classes, so pardon me for not going into much detail.
The interesting bit is this:
$copy = clone $this;
$copy->workerid = $new_workerid;
if ( $copy->mysql_create_row( $this->pending_table ) )
return $copy;
This runs as expected. When I go into the database, I see a new entry in table Pending.
When however I run multiple queries in succession:
$copy = clone $this;
$copy->workerid = $new_workerid;
if ( $copy->mysql_create_row( $this->pending_table ) )
if ( $this->mysql_delete_row( $this->pending_table ) )
if ( $this->mysql_create_row( $this->cancelled_table ) )
return $copy;
Then for some unexplainable reason, the wrong id column is inserted at one of the tables.
In specific method $copy->mysql_create_row, among other things, does this:
if ( $con = db_connect() ) {
$res = mysqli_query( $con, "INSERT INTO $table (`workerid`,`time`,`location`,`tasks`) VALUES ('$this->workerid','$this->datetime','$this->location','$this->tasks');" )
or error_log( "mysql_create_row Error: " . mysqli_error( $con ) );
if ( $res ) {
$this->contactid = mysqli_insert_id( $con );
foreach ( $this->attendees as $att ) {
$att->__setContactId( $this->contactid );
if ( !$att->mysql_create_row( $this->attendee_table ) ) {
error_log( " Contact->mysql_create_row Error: Did not create attendee MySQL row" );
return false;
}
Delegation through object method $att->mysql_create_row( $this->attendee_table ) works the same way, it is merely an INSERT query.
Now this is where the things make no sense:
Whereas the newly inserted $copy object inserts the correct id at $copy->mysql_create_row, the $att->mysql_create_row does not !!!
It inserts the old id.
Furthermore, when I echo out the query from $att->mysql_create_row, it echoes out the correct (new id)!!!
But when I go into the actual database, it has inserted the old id. This makes absolutely no sense to me.
I have tried to create custom __clone() methods, try to make sure that I have copies and not references (damn you php) but it still produces the same error.
The problem seems to be with the successive queries, as when I do not do the delete or the new insert, I do not have any problems.
The delete works fine, and the second insertion also works fine, without any errors whatsoever.
What could possibly force the MySQLi api to insert the wrong id, while the query echoes the correct one ?
I could only thing of some form of background/multithreaded corruption, but since I have no control over this sort of thing, I do not even know how to tackle the problem.
When echoing out the query from $att->mysql_create_row, I get:
before query id: 305
'INSERT INTO XXXXXXXX.attendees
(contactid,title,forenames,surname,relationship,gender,age,arrived,left,supervised,ics_number)
VALUES ('305','Mr',
'Alex','Werner','','M','29/08/1981','0','','','0');'
after query id: 305
But when I actually go into the database, the value inserted is the old id value (say 294).
The table column is not Auto-incrementing, is not unique or primary, it is simply a int(11).
What on earth could be causing this ?
Your problem could be that you are using mysql_last_insert_id to get the id of the last inserted row. As you are inserting multiple rows, it could be possible that you don't get the correct id.
Replace mysql_last_insert_id call with a select query that returns the id of your last insertion and look if your problem is solved.
I've probably murdered the whole concept of MVC somewhere along the line, but my current situation is thus:
I have participants in events and a HABTM relationship between them (with an associated field money_raised). I have a controller that successfully creates new HABTM relationships between pre-existing events and participants which works exactly as I want it to.
When a new relationship is created I wish to set the flash to include the name of the participant that has just been added. The actually addition is done using ids, so I've used the following code:
public function addParticipantToEvent($id = null) {
$this->set('eventId', $id);
if ($this->request->is('post')) {
if ($this->EventsParticipant->save($this->request->data)) {
$participant_id = $this->request->data['EventsParticipant']['participant_id'];
$money_raised = $this->request->data['EventsParticipant']['money_raised'];
$participant_array = $this->EventsParticipant->Participant->findById($participant_id);
$participant_name = $participant_array['Participant']['name'];
$this->Session->setFlash('New participant successfully added: ' . $participant_name . ' (' . $participant_id . ') ' . '— £' . $money_raised);
} else {
$this->Session->setFlash('Unable to create your event-participant link.');
}
}
}
This works, but generates the following SQL queries:
INSERT INTO `cakephptest`.`cakephptest_events_participants` (`event_id`, `participant_id`, `money_raised`) VALUES (78, 'crsid01', 1024) 1 1 0
SELECT `Participant`.`id`, `Participant`.`name`, `Participant`.`college` FROM `cakephptest`.`cakephptest_participants` AS `Participant` WHERE `Participant`.`id` = 'crsid01' LIMIT 1 1 1 0
SELECT `Event`.`id`, `Event`.`title`, `Event`.`date`, `EventsParticipant`.`id`, `EventsParticipant`.`event_id`, `EventsParticipant`.`participant_id`, `EventsParticipant`.`money_raised` FROM `cakephptest`.`cakephptest_events` AS `Event` JOIN `cakephptest`.`cakephptest_events_participants` AS `EventsParticipant` ON (`EventsParticipant`.`participant_id` = 'crsid01' AND `EventsParticipant`.`event_id` = `Event`.`id`)
This final one seems superfluous (and rather costly) as the second should give me all that I need, but removing $this->EventsParticipant->Participant->findById($participant_id) takes out both the second and third queries (which sort of makes sense to me, but not fully).
What can I do to remedy this redundancy (if indeed I'm not wrong that it is a redundancy)? Please tell me if I've made a complete hash of how these sorts of things should work – I'm very new to this.
This is probably due to the default recursive setting pulling the relationship. You can remedy this by setting public $recursive = -1; on your model (beware this will affect all find calls). Or, disable it temporarily for this find:
$this->EventsParticipant->Participant->recursive = -1;
$this->EventsParticipant->Participant->findById($participant_id);
I always suggest setting public $recursive = -1; on your AppModel and using Containable to bring in the related data where you need it.