PHP automatically removes the zeros that i need in an identification string - php

In a table, the primary field is a Char(12) field called ribiid, whose format is RB##########,
It needs to auto-increment it self, and for that i have prepared the following code:
function getid() {
global $connection;
$idquery = "SELECT ribiid FROM systems ORDER BY ribiid DESC LIMIT 1";
$idsave = mysqli_query($connection, $idquery);
$idresult = mysqli_fetch_assoc($idsave);
$idalpha = substr($idresult['ribiid'], 0, 2);
$idnumeric = substr($idresult, 2);
$newidnumeric = $idnumeric + 1;
$newid = $idalpha . $newidnumeric;
return $newid;
}
Now for testing I manually entered a row in cmd with id = RB0000000000, the next entry that I submit through my webpage using php, should have been RB0000000001, but it is coming RB1.
How can I fix this, this is my first web database. Thanks

Your problem is that when adding 1 to $idnumeric PHP needs to treat it as a number. Leading zeroes in numbers do not make sense, so they are discarded.
To keep the zeroes you can use sprintf format the resulting (incremented) number:
$newid = sprintf("%s%010d", $idalpha, $newidnumeric);
However, using code like this is not a really good idea
There's an issue with this code though: it's subject to a race condition. Consider what could happen if two instances of the script run in parallel:
Instance A Instance B
T |
i | Reads ribiid RB..001 Reads ribiid RB..001
m | Generates next id RB..002 Generates next id RB..002
e v Writes RB..002 to DB
Writes RB..002 to DB => OOPS
As you see this situation will result in instance B failing to insert a record due to the use of a duplicate primary key. To solve this problem you need to eliminate the race condition, which you could do in one of several ways:
Use an AUTO_INCREMENT column for the PK instead of manually inserting values. Although this means you can no longer have the "RB" prefix as part of the key, you can move it to a different column and have the PK be a combination of these two columns.
LOCK TABLES ribiid while the insertion is taking place (note that the lock needs to cover all of the process, not just the getid function). Locking tables is something you normally want to avoid, but if inserts are not frequent it's a usable practical solution.

You could try something like this:
$newid = $idalpha . str_pad($newidnumeric, 10, '0', STR_PAD_LEFT);
This will add zeros to reach the ten chars.

You can padd the numeric string again using the following function:
function pad_number($number, $pad=10){
$pad_zero = $pad - strlen($number.'');
$nstr = '';
for($i =0; $i< $pad_zero; $i++){
$nstr .="0";
}
$nstr .= $number;
return $nstr;
}
You can use in your code this function as:
$newid = $idalpha . pad_number($newidnumeric);

Related

PHP MySQL - Update 6.5m rows performance issues

I am working with a MySQL table and I need to increment a value in one column for each row, of which there are over 6.5m.
The col type is varchar and can contain an integer or a string (i.e. +1). The table type is MyISAM.
I have attempted this with PHP:
$adjust_by = 1;
foreach ($options as $option) {
$original_turnaround = $option['turnaround'];
$adjusted_turnaround = $option['turnaround'];
if (preg_match('/\+/i', $original_turnaround)) {
$tmp = intval($original_turnaround);
$tmp += $adjust_by;
$adjusted_turnaround = '+'.$tmp;
} else {
$adjusted_turnaround += $adjust_by;
}
if (!array_key_exists($option['optionid'], $adjusted)) {
$adjusted[$option['optionid']] = array();
}
$adjusted[$option['optionid']][] = array(
'original_turn' => $original_turnaround,
'adjusted_turn' => $adjusted_turnaround
);
}//end fe options
//update turnarounds:
if (!empty($adjusted)) {
foreach ($adjusted as $opt_id => $turnarounds) {
foreach ($turnarounds as $turn) {
$update = "UPDATE options SET turnaround = '".$turn['adjusted_turn']."' WHERE optionid = '".$opt_id."' and turnaround = '".$turn['original_turn']."'";
run_query($update);
}
}
}
For obvious reasons there are serious performance issues with this approach. Running this in my local dev environment leads to numerous errors and eventually the server crashing.
Another thing I need to consider is when this is run in a production environment. This is for an ecommerce store, and I cannot have a huge update like this lock the database or cause any other issues.
One possible solution I have found is this: Fastest way to update 120 Million records
But creating another table comes with it's own issues. The codebase is not in a good state, similar queries are run on this table in loads of places so I would have to modify a large number of queries and files to make this approach work.
What are my options (if there are any)?
You can do this task with SQL.
With CAST you can convert a string into integer.
With IF and SUBSTR you can check if string contains +.
With CONCAT you will add (merge a two values into one string) + to your calculated result (if it will be necessary).
Just try this SQL:
"UPDATE `options` SET `turnaround` = CONCAT(IF(SUBSTR(`turnaround`, 1, 1) = '+', '+', ''), CAST(`turnaround` AS SIGNED) + " + $adjust_by + ") WHERE 1";
can't you just say
UPDATE whatevertable SET whatever = whatever + 1?
Try it and see, I'm pretty sure it will work!
EDIT: You have strings OR integers? Your DB design is flawed, this probably won't work, but would have been the correct answer had your DB design been more strict.
You probably don't have, but need, this 'composite' index (in either order):
INDEX(optionid, turnaround)
Please provide SHOW CREATE TABLE.
Another, slight, performance boost is to explicitly LOCK TABLE WRITE before that update loop. And UNLOCK afterwards. Caution: This only applies to MyISAM.
You would be much better off with InnoDB.

How to loop my function based on query result

I wrote a function which makes a random id makeid(); Just to ensure the id is unique I have a SQL statement which checks if the id already exists.
$does_id_exist = mysql_query("SELECT COUNT(*) AS count FROM signups WHERE affid='$affid'");
if(mysql_num_rows($does_id_exist) == 1)
{
#loop function and perform query again
}
else
{
#insert record
}
So I'm having trouble with looping the function. How do I loop my function makeid() and perform the $does_id_exist check to ensure that each ID is unique.
--UPDATE-- Just to clarify- My code makes an id like YES#281E But before I INSERT this id into the users record. I just need to verify IF any other user already has this id. IF another user has this id that event must trigger my function to create a new id e.g. WOW!29E3 and again check the sql/query to ensure no other user has that id. Continue to loop if fails or end and INSERT if the id is available.
You can either just use a primary key on your database table, or something like this:
<?php
// the id to insert
$newId = null;
// populate with results from a SELECT `aff_id` FROM `table`
$currentIds = array();
// prepopulate
for( $i=0; $i<100000; $i++ )
{
$currentIds[] = "STRING_" + rand();
}
// generate at least one id
do
{
$newId = "STRING_" + rand();
}
// while the id is taken (cached in $currentIds)
while( in_array($newId, $currentIds) );
// when we get here, we have an id that's not taken.
echo $newId;
?>
Output:
STRING_905649971 (run time 95ms);
I'd definitely not recommend running the query repeatedly. Perhaps a final check before you insert, if your traffic volume is high enough.
Do not do COUNT(*), because you do not need to know how many rows is there (it should be 0 or 1 as you need Id unique), so even DB finds your row it will still be checking for the whole table to count. You really care if you got 1 row, so just select for row with that ID and this sufficient. You should also avoid using rand() - this does not help as you see and you cannot predict how many loops you can do before you find "free slot". use something predictable, like date prefix, or prefix incremented each day. anything that would help you narrow the data set. But for now (pseudocode!):
$id = null;
while( $id == null ) {
$newId = 'prefix' . rand();
mysql_query("SELECT `affid` FROM `signups` WHERE `affid`='${newId}'");
if( mysql_num_rows() == 0) {
$id = newId;
break;
}
}
Ensure you got DB indexed, to speed things up.
EDIT: I do agree that any cache would be useful to speed things up (you can add it easily yourself based on #Josh example), still, I think this is fixing at wrong place. If possible rethink the way you generate your ID. It does not really need to be auto increment, but something more predictable than rand() would help you. If your ID does not need to be easily memorable and it is not any security concern to have them sequential, maybe use numbers with other base than 10 (i.e. using 26 would use all digits + letters so you'd end with PREFIX-AX3TK, so string as you want, and at the same time you would easily be able to quickly generate next Id

Looking for help creating a unique ID and making sure it doesn't exist in the DB

I'm trying to create a function that, when run, creates numerical ID, checks to see if that ID exists in the DB and if it does exists then create another ID and check again until it has a unique ID. I'm stuck on how to loop in the functions withing a function
function createUniqueID() {
function buildUnique() {
$uniqueID = rand(100000000000,999999999999);
return $uniqueID;
}
function compareWithDB($uniqueID) {
$s = "SELECT id FROM table WHERE id='{$uniqueID}'";
$r = mysql_query($s);
return $r;
}
function countDBRows($r) {
if(mysql_num_rows($r) >0){
$f = false; // found
} else{
$f = true;
}
}
$uniqueID = buildUnique();
$r = compareWithDB($uniqueID);
$f = countDBRows($r);
if (!$f) {
$uniqueID = 'nope';
}
return $uniqueID;
}
You're much better off to call MySQL's UUID() function, and store & return the value of that, unless the value absolutely must be numeric.
SELECT UUID();
If you want a unique number, then just use AUTO_INCREMENT
If you want a unique random number (why?) create a unique index on the ID column and keep trying until you get no errors. This is better for concurrency: multiple concurrent calls can have the same number and pass the test. Better still, let the DB engine do it with RAND...
Theres another way too,
You can use the sha1() function in conjuction with an static variable as shown:
function GetUniqueID() {
static $salt=1;
$id = sha1("somestring".(string)$salt);
$salt++;
return $id;
}
Here $salt being an static one retains its value between all calls incrementing ensuring unique ID's (hash of SHA1).for more security "somestring" can be made randomized also .
Might I suggest the much easier and more efficient uniqid function which will do this for you. It generates guaranteed unique IDs based on the timestamp in milliseconds. The generated Id is 13 digits long unless you decide to add a prefix or use extra 'entropy' (more uniqueness).
Edit: Note that this is both numbers and letters returned by the function.
http://php.net/manual/en/function.uniqid.php
Edit 2:
Using your method here is how to nest the loops...
while (true) {
$uniqueID = buildUnique();
$r = compareWithDB($uniqueID)
if (countDBRows($r)) { break; }
}
Your unique ID is then stored in $uniqueID. I do however discourage this because it's bulky and inefficient... but there you go! :)
I'm answering my own question because I have found a solution of creating a big number that (likely) will never duplicate. I use three random 2-digit variables and three different date syntax.
Putting them together makes it clear that they will never duplicate unless someone happens to post during the same year, week and with the same number of seconds in the minute an also have 3 different random number all be the same.
I think the chances of this happening would be in the millions and if there are millions of posts happening then i'm sure I will have more resources to solve this issue.
$rand1 = rand(10,99);
$rand2 = rand(10,99);
$rand3 = rand(10,99);
$date1 = date("s"); // seconds
$date2 = date("y"); // 2 digit year
$date3 = date("W"); // week number (out of 52)
$uniqueID = $date1.$rand1.$date2.$rand2.$date3.$rand3;
return $uniqueID;

Performance of recursively searching for indices in large sets of data

I came across a question today of search efficiency for large sets today and I've done by best to boil it down to the most basic case. I feel like this sort of thing probably relates to some classic problem or basic concept I'm missing, so a pointer to that would be great.
Suppose I have a table definition like
CREATE TABLE foo(
id int,
type bool,
reference int,
PRIMARY KEY(id),
FOREIGN KEY(reference) REFERENCES foo(id),
UNIQUE KEY(reference)
) Engine=InnoDB;
Populated with n rows where n/2 are randomly assigned type=1. Each row references another with its same type except for the first, which has reference=null.
Now we want to print all items with type 1. I assume that at some point, it will be faster to recursively call something like
function printFoo1($ref){
if($ref==null)
return;
$q = 'SELECT id, reference FROM foo WHERE id='.$ref;
$arr = mysql_fetch_array( mysql_query($q) );
echo $arr[0];
printFoo1($arr[1]);
}
As opposed to
function printFoo2($ref){
$q = 'SELECT id FROM foo WHERE type=1';
$res = mysql_query($q);
while( $id = mysql_fetch_array($res) ){
echo $id[0];
}
}
The main point here being that function 1 searches for "id", which is indexed, whereas function 2 has to make n/2 comparisons that don't result in a hit, but that the overhead of multiple queries is going to be significantly greater than the single SELECT.
Is my assumption correct? If so, how large of a data set would we need before function 1 outperforms function 2?
Your example is a bit difficult to parse, but ill start at the top:
Your first function does not return all of the elements with type = 1. It returns all of the elements that are dependent (based on references) to the element you pass in. From the PHP standpoint, since the link/handle is already open there is a non-trivial overhead from your function call with each successive request, not to mention the string concatenation incurring a cost with each execution of that line.
Typically it is better to use the second function styling because it only queries the database one time and will return the elements you are requesting without further work. It will come down to a profiler of course, to determine which is going to return faster, but from my tests the second is hands down the better choice:
This was executed with n = 5000 elements in the db (n/2 = 2500 type 1 and passing in reference = highest id with type = 1 from query of db).
printFoo1: 3.591840 seconds
printFoo2: 0.010340 seconds
It wouldn't really make sense for it to work any other way. If you were able to do what you propose that would make JOIN calls have to perform less efficient as well.
Code
$res = mysql_query('SELECT MAX( id ) as `MAX_ID` FROM `foo` WHERE `type` = 1', $link);
$res2 = mysql_fetch_assoc($res);
$id = $res2['MAX_ID'];
// cleanup result and free resources here
echo "printFoo1: ";
$start = microtime(true);
printFoo1($id);
echo microtime(true) - $start;
echo '<br />';
echo "printFoo2: ";
$start = microtime(true);
printFoo2();
echo microtime(true) - $start;
mysql_close($link);
All of this was tested on PHP 5.2.17 running on Linux

Checking mySQL db for duplicate uid

I am trying to implement a check in my PHP code, that checks if there is a duplicate uid in the database, and if so, to assign a new uid, and check again, but I am having trouble nailing the logic, here is what I have thus far,
function check($uid){
$sql = mysql_query("SELECT * FROM table WHERE uid='$uid'");
$pre = mysql_num_rows($sql);
if($pre >= 1){
return false;
}else{
return true;
}
}
And then using that function I thought of using a while loop to continue looping through until it evaluates to true
$pre_check = check($uid);
while($pre_check == false){
//having trouble figuring out what should go here
}
So basically, once I have a usable uid, write everything to the database, else keep generating new ones and checking them till it finds one that is not already in use.
It is probably really simple, but for some reason I am having trouble with it.
Thanx in advance!
$uid = 100; // pick some other value you want to start with or have stored based on the last successful insert.
while($pre_check == false){
$pre_check = check(++$uid);
}
Of course ths is exactly what 'auto incrementing' primary keys are useful for. Are you aware of 'auto incrementing' primary keys in mysql?
EDIT
In light of your comment regarding maintaining someone else's code that uses the random function like that (ewwwww)... I would use the method I suggest above and store the last inserted id somewhere you can read it again for the next user. This will allow you to "fill-in-the-blanks" for the uids that are missing. So, if for example you have uids 1, 2, 5, 9, 40, 100... you can start with $uid = 1; Your while loop will return once you get to 3. Now you store the 3 and create the new record. Next time, you start with $uid = 3; and so on. Eventually you will have all numbers filled in.
It is also important to realize that you will need to do the inserts by either locking the tables for WRITES. You don't want to get into a race condition where two different users are given the same uid because they are both searching for an available uid at the same time.
Indeed the best is to use autoincrement ids, but if you don't have the choice, you can do a reccursive function like that:
function find_uid() {
$new_uid = rand(1000000000, 9999999999);
$sql = mysql_query("SELECT COUNT(*) AS 'nb' WHERE uid=".$new_uid.";");
$row = mysql_fetch_assoc();
$pre = $row['nb'];
return ($pre >= 1 ? find_uid() : $new_uid);
}
COUNT(*) should be more performant because the count is made by MySQL and not php.
By the way, if you need a new uid shouldn't the condition be ($pre > 0) instead of ($pre > 1) ?

Categories