I have a PHP file that inserts data from a CSV file into the MYSQL DB using the LOAD DATA INFILE sql function.
If the CSV data is a duplicate it is not inserted because of the DB table indexing rules (unique).
$sql = "LOAD DATA LOW_PRIORITY LOCAL INFILE
'" . $makes_file . "' IGNORE
INTO TABLE make
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n' (make, img_url)";
$link->query($sql) or die(mysqli_error($link));
For some reason the Auto Increment of the table is not correct from this process.
So I have used the following code to correct this.
$get_max = $link->query("SELECT max(id) as max FROM `make` LIMIT 1");
while ($r = $get_max->fetch_assoc()) {
$link->query("ALTER TABLE `make` AUTO_INCREMENT = " . ($r['max'] + 1)) or
die(mysqli_error($link));
}
So if anybody knows:
Why the Auto Increment is incorrect from the LOAD DATA sql
or
If there is a 'prettier' way to set the auto increment to the max(id) +1
Thanks
For MyISAM tables you can just set AUTO_INCREMENT to 0 and MySQL will use the current maximum value plus one.
$link->query('ALTER TABLE make AUTO_INCREMENT = 0');
From the docs:
You cannot reset the counter to a value less than or equal to any that
have already been used. For MyISAM, if the value is less than or equal
to the maximum value currently in the AUTO_INCREMENT column, the value
is reset to the current maximum plus one.
This answer is about a prettier way to set the auto increment to the max(id) +1, You can do this with one Query only :
$link->query("ALTER TABLE `make` AUTO_INCREMENT = ( SELECT max(id)+1 FROM `make` LIMIT 1 )" );
But your first solution should work if AUTO_INCREMENT is unique , check your database sheme
Check your last id in your database manually and get it
Then insert new record and set id to
Your last id plus say 3 or 4
Example
If your last taken id is 5000
Insert new record and set id to 5003
Then auto increment will works again from 5003.
Related
I Have some question about the Seq in PHP.
I create 1 tabel to maintain the sequence table,
ex. table is sequence and the field is seq(start from 1)
The sequence should be unique value.
The problem is.
there is 2 process which is run parallel that will use it.
and I got the error message that say duplicate value.
The question.
How to lock the table from another select query?
this is my code
//select seq
$sqlSeq = "select seq from sequence for update";
$resultSeq = pg_query($sqlSeq);
$rowSeq = pg_fetch_assoc($resultSeq);
$seqCif = $rowSeq['seq'];
//INSERT
$sqlInsert1 ="insert into TEST (customer_id) values( '".$seqCif."')";
//UPDATE
$sqlInsert1 .= "update sequence set seq=seq+1;" ;
Can you help me for this case?
Many Thanks befor..
One simple way to make the operation atomic and to guarantee an incremental id without gaps is to turn the statement to an insert ... select ... query:
insert into test(customer_id)
select coalesce(max(customer_id), 0) + 1 from test
I currently have a script that runs every 5 minutes and selects the data from a table on server 1 and an identical table on server2. This is a workaround for replication, essentially, since we don't have that option currently.
The script is successful but I've realized that it misses records sometimes, for whatever reason. The current script selects all records from the destination table, stores the max primary key, selects all data from the source table and then inserts anything with a greater Primary key into the dest. table.
I'd like to modify the script slightly and instead of using max id, just say "if a row has an primary key that doesn't exist in the destination table, insert that row there."
Again these are cloned tables so the structure is the same and they both use AI Primary Keys.
Here's the current working script:
$latest_result = $conn2->query("SELECT MAX(`SESSIONID`) FROM
`ambition`.`session`");
$latest_row = $latest_result->fetch_row();
$latest_session_id = $latest_row[0];
//Select All rows from the source phone database
$source_data = mysqli_query($conn, "SELECT * FROM
`cdrdb`.`session` WHERE `SESSIONID` > $latest_session_id");
// Loop on the results
while($source = $source_data->fetch_assoc()) {
// Check if row exists in destination phone database
$row_exists = $conn2->query("SELECT SESSIONID FROM
ambition.session WHERE SESSIONID = '".$source['SESSIONID']."' ") or
die(mysqli_error($conn2));
//if query returns false, rows don't exist with that new ID.
if ($row_exists->num_rows == 0){
//Insert new rows into ambition.session
$stmt = $conn2->prepare("INSERT INTO ambition.session (SESSIONID,
SESSIONTYPE,CALLINGPARTYNO,FINALLYCALLEDPARTYNO,
DIALPLANNAME,TERMINATIONREASONCODE //etc. There are a lot of columns so I
ommitted the others
Is there a way I can slightly modify this to just insert what doesn't exist rather than relying on the MAX ID?
Or is there something here that would be a culprit as to why it's missing records?
You could use INSERT INTO SELECT and check if value is already in target:
INSERT INTO trg_table (cols)
SELECT cols
FROM src_table s
WHERE NOT EXISTS (SELECT 1 FROM trg_table t WHERE t.id = s.id);
Following is my database in mysql:
Id Username Password
1 admin admin
2 jay jay1
3 suman xyza
4 chintan abcde
This is my code in php:
$fetchid = mysql_query(" SELECT MAX(Id) As max From user;");
$row = mysql_fetch_array($fetchid);
$largest = $row['max'];
$largest++;
$user= $_POST['username'];
$pass= $_POST['password'];
$result = mysql_query(" INSERT INTO `proshell`.`user` (
`Id` ,
`Username` ,
`Password`
)"."
VALUES (
'".$largest."', '".$user."', '".$pass."'
);");
Problem:
Now if I delete row with Id=1 and then re-enter the data then it should use ID=1 then Again I reinsert the data it use ID=5
It works like this:
if I delete row with Id=1 and then re-enter the data the Id it gets is 5 but then 1 is free so,
What should I write to perform that task.
First, if you set your Id column to AUTO_INCREMENT you don't need the following part in your code at all:
$fetchid = mysql_query(" SELECT MAX(Id) As max From user;");
$row = mysql_fetch_array($fetchid);
$largest = $row['max'];
$largest++;
Because AUTO_INCREMENT will automatic add new value to your ID colume.
But if you don't set it to AUTO_INCREMENT, the above code will grab the MAXIMUM ID value (in this case, 4).
When you re-enter your data again after you delete the row 1, the MAXIMUM ID still 4, so your new ID value will be 5 (from $largest++;).
.....
If you really need to use consecutive ids as you PK, you need to re-write you code but I suggest you to use UUID for you ID column instead.
You can easily generate UUID by using uuid().
How about the UUID performance? Refer to Dancrumb's answer about this:
A UUID is a Universally Unique ID. It's the universally part that you should be considering here.
Do you really need the IDs to be universally unique? If so, then UUIDs
may be your only choice.
I would strongly suggest that if you do use UUIDs, you store them as a
number and not as a string. If you have 50M+ records, then the saving
in storage space will improve your performance (although I couldn't
say by how much).
If your IDs do not need to be universally unique, then I don't think
that you can do much better then just using auto_increment, which
guarantees that IDs will be unique within a table (since the value
will increment each time)
see. UUID performance in MySQL?
EDIT: I don't suggest you run query on the whole table just to find the MAX ID value before inserting new value everytime, because it will give you a performance penalty (Imagine that if you have million rows and must query on them everytime just to insert a new row, how much workload causes to your server).
It is better to do the INSERT just as INSERT, no more than that.
EDIT2:
If you really want to use consecutive ids, then how about this solution?
Create new TABLE just for store the ids for insert (new ids and the ids that you deleted).
For example:
CREATE TABLE cons_ids (
ids INT PRIMARY KEY,
is_marker TINYINT DEFAULT 0
);
then initial ids with values from 1-100 and set marker to be '1' on some position, e.g. 80th of whole table. This 'marker' uses to fill your ids when it's nearly to empty.
When you need to INSERT new Id to your first table, use:
$result = mysql_query("SELECT ids, marker FROM cons_ids ORDER BY ids ASC LIMIT 1;");
$row = mysql_fetch_row($result);
and use $row[0] for the following code:
INSERT INTO yourtable (Id, Username, Password)
VALUES ($row[0], $username, $password);
DELETE FROM cons_ids
WHERE ids = $row[0];
This code will automatically insert the lowest number in cons_ids as your Id and remove it from the cons_ids table. (so next time you do insert, it will be the next lowest number)
Then following with this code:
if ($row[1] == 1) {
//add new 100 ids start from the highest ids number in cons_ids table
//and set new marker to 80th position again
}
Now each time you delete a row from your first table, you just add the Id from the row that you deleted to cons_ids, and when you do INSERT again, it will use the Id number that you just deleted.
For example: your current ids in cons_ids is 46-150 and you delete row with Id = 14 from first table, this 14 will add to your cons_ids and the value will become 14, and 46-150. So next time you do INSERT to your first table, your Id will be 14!!.
Hope my little trick will help you solve your problem :)
P.S. This is just an example, you can modify it to improve its performance.
First of all, as I understand, you are selecting highest column ID which should be always the last one (since you set auto-increment on ID column).
But what are you trying to do is actually filling up holes after delete query, right?
If you are really looking for such approach, try to bypass delete operation by making new boolean column where you flag record if it is active or not (true/false).
SQL table change:
Id Username Password Active
1 admin admin false
2 jay jay1 true
3 suman xyza false
4 chintan abcde true
PHP request:
$fetchid = mysql_query(" SELECT MIN(Id) As min FROM user WHERE active = false;");
$result = mysql_query(" INSERT INTO `proshell`.`user` (
`Id` ,
`Username` ,
`Password`
`Active`
)"."
VALUES (
'".$largest."', '".$user."', '".$pass."', 'true'
);");
I'm trying to obtain the max primary key in the REDES table in order to add one but it's not working. I was also wondering if this would be the best way to do it and if it's secure.
// mysql inserting a new REDES field
//$redes_maxid = mysql_query("SELECT MAX(id_redessociales) AS id FROM REDES");
//$newmaxid = $redes_maxid + 1;
$redes_maxid = mysql_fetch_row(mysql_query("SELECT max(id_redessociales) AS id FROM REDES"));
$newmaxid = $redes_maxid[0] + 1;
mysql_query("INSERT INTO REDES(id_redessociales, facebook, twitter, linkedin, pinterest) VALUES ('$newmaxid', NULL, NULL, NULL, NULL");
Thanks!
What you're doing is very unsafe and wrong.
This is why we have auto increment fields in databases, just set the id field in your table to auto increment then when inserting a new row simply pass in NULL for the id, MySQL it will automatically do this increment for you safely.
here's a link to the documentation
You can get last id by ordering in descending order and limiting 1 row result
replace this line
$redes_maxid = mysql_fetch_row(mysql_query("SELECT max(id_redessociales) AS id FROM REDES"));
with
$redes_maxid = mysql_fetch_row(mysql_query("SELECT id_redessociales AS id FROM REDES order by id_redessociales DESC limit 1"));
exactly what A.B said
But shouldn't you be making the ID's row in the database auto incremented?
you can set it to auto increment and when you insert a record just make the inserted ID NULL, the database will automaticly number it to the last ID+1
if you are ysing mySql then you can use this SQL statement
ALTER TABLE `myTable` CHANGE `id` `id` TINYINT( 3 ) PRIMARY KEY UNSIGNED NOT NULL AUTO_INCREMENT
I have my database with table test1.
It has a primary id "Id" which is auto-increment.
Now the id is in the format 1,2,3.. . .Is it possible to store the primary Id as
PNR1,PNR2,PNR3 .. . . and so on(with auto-increment).
No. Either add the prefix in the query, or use a view instead.
Not really, but you can use another column (but a view)
this is already covered here:
MySQL Auto Increment Custom Values
Yes you can do it if you have INT prefix. You have id as INT in table
// START PREFIX
$query = mysql_query("SELECT id FROM `table_name` ORDER BY id DESC LIMIT 1");
// GET THE LAST ID MAKE SURE IN TABLE YOU 9991
while ($row = mysql_fetch_object($query)) {
$lastId = $row->id;
}
list($prefix,$Id) = explode('999',$lastId );
$Id = ($Id+1);
$new_id = '999'.$Id;
// END PREFIX
$insertQuery = mysql_query("INSERT INTO `table_name` SET id = '".$new_id."',...");
Hi, I made it work in this way :
Products Table (products):
id_prod(varchar(11), NOT NULL, PK), name(varchar(40))
Products Sequence Table (productidseq):
id(AI, PK, NOT NULL)
Before Insert Trigger in Products Table:
CREATE DEFINER=`root`#`localhost` TRIGGER `dbname`.`products_BEFORE_INSERT` BEFORE INSERT ON `products` FOR EACH ROW
BEGIN
insert into productidseq (id) values(NULL);
set new.id_prod = concat('PROD or any prefix here',last_insert_id());
set #productId = new.id_prod; -- To use outside of trigger this variable is useful.
END
When you run below query :
insert into products (name) values('Bat');
data inside tables will we be like this :
products:
id | name
---|-----
1 | Bat
productidseq:
id
---
1
If any better way than this or any cons with this, please comment below. Thanks.