MongoDb Collection 300K Objects Making an A to Z page - php

We have a website with about 300,000 people on it - what we want to do is make an page with all the peoples name that starts with A, B, or C.
The challenge is the speed.
How would you set up your database. Do you make a cache collection. Do you use regex or something else.
What i've done is the following;
$letter = 'A';
$where = array();
$where['name'] = new MongoRegex("/^" . $letter . "/i");
$sort = array('name' => 1);
if($hasImage){
$where['images.profiles.0'] = array('$exists' => true);
}
$fields = array('name' => 1, 'images.profiles' => 1);
$this->mdb->data_people->ensureIndex(array('name' => 1, 'images.profiles' => 1), array('background' => true) );
$people = $this->mdb->data_people->find( $where, $fields );
$people = $people->sort( $sort );
$page['total'] = 100;
$page['current'] = 1;
$page['perPage'] = 20;
if(isset($this->domain->getQuery['_page']) && $this->domain->getQuery['_page'] > 1){
$page['current'] = $this->domain->getQuery['_page'];
}
$data['pages'] = $this->pageNavigation->setNavigation((int) $page['total'], (int) $page['perPage'], (int) $page['current'] );
$data['pages']['page'] = $this->domain->menu_reverse[0];
$data['pages']['path'] = $this->domain->path;
$data['data'] = $people->limit($page['perPage'])->skip( ($page['perPage']*($page['current']-1)) );

You are pretty much doing the right thing.
You are even using the right regexp since it look only for the start of the string, and so Mongodb can use the name index.
see bottom of this page: http://docs.mongodb.org/manual/reference/operator/query/regex/)
I see only two potential problems (but as always: don't do premature optimization, you'll code can be just fine as is):
the case-insensitive modifier in the regexp: try using a strtolower-ed field insted (create a nameCanonical field with the lowercase version of the name field.)
the pagination, which is always slow: try using a range-based pagination http://docs.mongodb.org/manual/reference/method/cursor.skip/

Related

Update One JSON Field in Database - CodeIgniter

I have a JSON field called 'spec' and there are about 10 other items in this in JSON format. I need to only update the quantity.
Although when I try this method, it deletes everything else in it and just sets spec = quantity.
Heres what I have so far.
$pass_coupon_id = $this->pass_coupon_id();
$coupon_array = $this->db->query("SELECT * FROM coupon WHERE coupon_id='$pass_coupon_id'")->result_array();
foreach ($coupon_array as $row) {
$spec = json_decode($row['spec'], true);
}
$quantity_new = $spec['quantity'] - 1;
$data2 = array(
'spec' => json_encode(array(
'quantity'=> $quantity_new
)));
$this->db->where('coupon_id', $pass_coupon_id);
$this->db->update('coupon', $data2);
You need to overrite only this one field and update whole field in query.
<?php
$pass_coupon_id = $this->pass_coupon_id();
$coupon_array = $this->db->query("SELECT * FROM coupon WHERE coupon_id='$pass_coupon_id'")->result_array();
// i don't know what you're using, but using foreach to extract single row isn't good solution. Look for sth like result_row() maybe.
$coupon = $coupon_array[0];
$spec = json_decode($coupon, true);
$new_quantity = $spec['quantity'] - 1;
$spec['quantity'] = $new_quantity;
$new_spec = json_encode($spec);
$this->db->where('coupon_id', $pass_coupon_id);
$this->db->update('coupon', $new_spec);
Depending on the database, the best solution would be using specific function to ommit updating whole structure - https://stackoverflow.com/a/34987329/2926214

Is it beneficial to reduce number of DB calls by merging conditions covered by PHP and MySql

I've given a very ugly piece of code which combines PHP loops and SQL queries to retrieve some IDs and I have to refractor. My plan is to merge the PHP iteration to the queries, but I was wondering if a. doing this affect the result and b. it really has good impact on the performance.
The following is a gesture of the current state of the script:
$additional_conditions = [
"test_column1 = 'b1' OR test_column1 = 'b2' ",
"test_column2 = 'c1' OR test_column2 = 'c2' ",
"test_column3 = 'd1' OR test_column3 = 'd2' ",
"test_column4 = 'e1' OR test_column4 = 'e2' ",
"test_column5 = 'f1' OR test_column5 = 'f2' ",
];
$query = 'SELECT * from test_table WHERE test_column = 1 AND %s';
foreach ($additional_conditions as $additional_condition)
{
$result = mysqlQuery(sprintf($query, $additional_condition));
while ($data = mysqli_fetch_assoc($result)) {
// do something with the data
}
}
which I'm thinking to change it to:
$query = <<<QUERY
SELECT * from test_table WHERE test_column = 1 AND (
(test_column1 = 'b1' OR test_column1 = 'b2') OR
(test_column2 = 'c1' OR test_column2 = 'c2') OR
(test_column3 = 'd1' OR test_column3 = 'd2') OR
(test_column4 = 'e1' OR test_column4 = 'e2') OR
(test_column5 = 'f1' OR test_column5 = 'f2')
)
QUERY;
$result = mysqlQuery($query);
while ($data = mysqli_fetch_assoc($result)) {
// do something with the data
}
Your database engine will optimize the query to make it more efficient. It is usually better to perform a heavy database query than a number of dummy queries.

Multi undefined variables from a form and subsequent sql statement

The outcome I seek is to loop through items taken from a form that isset or true and then have a sql statement query the database. The complication comes from I have 9 fields that the user could choose one or more or all fields (It is a search database but every time you choose an extra field it refines the search). The part of the code to return true I can get my head around by using a foreach loop but how would you link it to SQL query if the outcome is varied?
You generate a different query depending on which fields are entered.
Luckily this isn't too hard in SQL: all those fields are in the WHERE clause:
$where = [ 'foo' => 'bar', 'baz' => 0 ];
$sth = $db->prepare( "SELECT * FROM $table WHERE " .
implode( " AND ",
array_map( function($i) { return "$i=?"; }, array_keys( $where ) ) )
);
$sth->execute( array_values( $where ) );
Of course, if there are relationships between the fields, the query may become more complicated, but this is the gist of it.
Learning this takes time and patience I have cut past the variables from the form
if(isset($_POST['Search'])){
$packID = $_POST['packID'];
$supplier_name = $_POST['supplier_name'];
$timber_species = $_POST['timber_species'];
$timber_product = $_POST['timber_product'];
$timber_grade = $_POST['timber_grade'];
$timber_finish = $_POST['timber_finish'];
$timber_treatment = $_POST['timber_treatment'];
$width = $_POST['width'];
$thickness = $_POST['thickness'];
$length = $_POST['length'];
$markup = $_POST['markup'];
} else{
$packID="";
$supplier_name="";
$timber_species="";
$timber_product="";
$timber_grade="";
$timber_finish="";
$timber_treatment="";
$width= "";
$thickness= "";
$length="";
}
How would you write this Kenney when the variables may or may not be set. I must admit I am a novice and keen to learn. It takes time and patience.

Filling a two dimensional array with mysql fetch results in php

I just had a framework for creating charts and this is how it works normally.
$p = new chartphp();
$p->data = array(array(
array("A",2),
array("B",3),
array("C",23),
array("D",10)
));
$p->chart_type = "bar";
// Common Options
$p->xlabel = "My X Axis";
$p->ylabel = "My Y Axis";
$out = $p->render('c1');
This way it works perfectly fine, now I need to get results from a sql query and fill the array.
$query ="SELECT t.date AS dates,COUNT(t.id) AS trans FROM Gab AS g, Transaction AS t WHERE t.date BETWEEN '2015-07-30' AND '201-07-10' AND g.TID = '1401009' ORDER BY DATES";
$ask = mysql_query($query) or die("Error");
//Now I try to load the results into the array to be integrated into the API.
$p = new chartphp();
$p->data = array(array(
while($recon = mysql_fetch_array($ask)
{
array($recon['dates'],recon['trans']),
}
));
$p->chart_type = "bar";
// Common Options
$p->xlabel = "My X Axis";
$p->ylabel = "My Y Axis";
$out = $p->render('c1');
I tried this but it does not work, the array dont seem to be loaded !
I'm actually not sure what nesting a while like you have would do and I'm unable to experiment at the moment, but something like this should get you in the right direction:
$p->data = array(array());
while($recon = mysql_fetch_array($ask))
{
$p->data[0][] = array($recon['dates'], $recon['trans']);
}
Initializing the array and then appending the elements in the loop.

Example of a Multi Condition Delete with Zend framework

Can someone give me an example of how I would delete a row in mysql with Zend framework when I have two conditions?
i.e: (trying to do this)
"DELETE FROM messages WHERE message_id = 1 AND user_id = 2"
My code (that is failing miserably looks like this)
// is this our message?
$condition = array(
'message_id = ' => $messageId,
'profile_id = ' => $userId
);
$n = $db->delete('messages', $condition);
Better to use this:
$condition = array(
'message_id = ?' => $messageId,
'profile_id = ?' => $userId
);
The placeholder symbols (?) get substituted with the values, escapes special characters, and applies quotes around it.
Instead of an associative array, you should just be passing in an array of criteria expressions, ala:
$condition = array(
'message_id = ' . $messageId,
'profile_id = ' . $userId
);
(and make sure you escape those values appropriately if they're coming from user input)
Use this , it is working...
$data = array(
'bannerimage'=>$bannerimage
);
$where = $table->getAdapter()->quoteInto('id = ?', 5);
$table->update($data, $where);

Categories