mysql database structure (unique id) - php

I think most people had serious problems with inserting a value/data into a database (mysql). When I'm inserting a value into a database, I assign an unique id (INT) for that line. When I query the database, easily I can read/modify/delete that line.
With function for() (in php) I easily can read values/data from the database. The problem occurs when I delete a line (in the middle for example).
E.g:
DB:
ID | column1 | column2 | ... | columnN
--------------------------------------
1 | abcdefgh | asdasda | ... | asdasdN
2 | asdasddd | asdasda | ... | asdasdN
...
N | asdewfew | asddsad | ... | asddsaN
php:
for($i = 0; $i <= $n; $i++){
$sql = mysql_query("SELECT * FROM db WHERE ID = '$i' ");
//Code;
}
*$n = last column value from ID
Am I need to reorganize the entire database to have a correct "flow" (1, 2, 3, .. n)? Or am I need to UPDATE the each cell?

What you're doing here is unnecessary thanks to AUTO_INCREMENT in mysql. Run this command from PhpMyAdmin (or another DB management system):
ALTER TABLE db MODIFY COLUMN ID INT NOT NULL AUTO_INCREMENT;
Now when insert a row into db mysql will assign the ID for you:
INSERT INTO db (id, column1, column2) VALUES(NULL, 'abc', 'def');
SELECT * FROM db;:
+--+-------+-------+
|id|column1|column2|
+--+-------+-------+
|1 |old |oldrow |
+--+-------+-------+
|2 |abc |def | <--- Your newly inserted row, with unique ID
+--+-------+-------+
If you delete a row, it is true that there will be an inconsistency in the order of the ID's, but this is ok. IDs are not intended to denote the numeric position of a row in a table. They are intended to uniquely identify each row so that you can perform actions on it and reference it from other tables with foreign keys.
Also if you need to grab a group of ID's (stored in an array, for example), it is much more efficient to perform one query with an IN statement.
$ids = array(1,2,3,4,5,6);
$in = implode(',', $ids);
mysql_query('SELECT * FROM db WHERE id IN ('.$in.')');
However, if you want all rows just use:
SELECT * FROM dbs;
But be weary of bobby tables.

You can select all the rows with query
SELECT * FROM db
and do whatever you want after

You can never ensure, that the ids are continous. As you noticed yourself there are gaps after you delete a row, but even for example when you insert rows within a transaction and don't commit it, because something failed and you need to revert the transaction. An ID is an identifier and not row number.
For example if you want to select X items from somewhere (or such) have a look at LIMIT and OFFSET
SELECT * FROM mytable ORDER BY created DESC LIMIT 10 OFFSET 20;
This selects the rows 21 to 30 ordered by their creation time. Note, that I don't use id for ordering, but you cannot rely on it (as mentioned).
If you really want to fetch rows by their ID you definitely need to fetch the IDs first. You may also fetch a range of IDs like
SELECT * FROM mytable WHERE id IN (1,2,3,4);
But don't assume, that you will ever receive 4 rows.

Ids are surrogate keys–they are in no way derived from the data in the rest of the columns and their only significance is each row has a unique one. There's no need to change them.
If you need a specific range of rows from the table, use BETWEEN to specify them in the query:
SELECT id, col1, col2, ..., coln
FROM `table`
WHERE id BETWEEN ? AND ?
ORDER BY id

If you need all the rows and all columns for that table use:
$query = mysql_query("SELECT * FROM table_name");
and then loop through each row with a while statement:
while ($row = mysql_fetch_array($query ))
{
echo $row['column_name'];
}

You do not have to reorganize the entire database in order to keep the index. But if you feel like it, you'd have to update each cell.
BTW, look at mysql_fetch_array(), it will ease the load on the SQL server.

Related

Add column of data to an existing mySQL table

I have a basic SQL problem that's been driving me mad. If I have a mySQL table e.g below.
How would I add another 80+ values to Column 2 starting from the first empty row (in this example row 3).
I've been trying a number of queries using INSERT or UPDATE but the closest I've got is to add the values to column 2 starting from the last defined ID value (e.g. row 80ish).
ID | Column 2 |
--------------------------------
1 | value |
2 | value |
3 | |
4 | |
5 | |
etc
The real table has around 10 columns, all with data in but I just need to add content (a list of around 80 different strings in CSV format to one of the columns)
I'd appreciate it if anyone could point me in the right direction.
I'd load the data into a separate table with the same structure and then update the target table using join or subquery to determine which columns are currently empty.
i.e. load interim table and then:
update target_table set column2 = (select column2 from interim_table where ...
where column2 is null
(slow but intuitive)
update target table, interim_table
set target table.column2 = interim_table.column2
where target table... = interim_table...
and target_table.column2 is null
(better performance)
Why don't you first run a query to find out the first empty row ID number? you can use SELECT COUNT(*)
FROM TABLE_NAME for that.
then you create a for loop and inside run a INSERT query, starting with the value returned by the previous query. just a scratch:
for(var id = last; id < totalOfQueries; id++)
{
var query = new MysqlCommand("INSERT INTO table VALUES ('" + id + "',....);
}

Deleting Duplicate Rows from MySql Table

I have a script to find duplicate rows in my MySql table, the table contains 40,000,000 rows. but it is very slow going, is there an easier way to find the duplicate records without going in and out of php?
This is the script i currently use
$find = mysql_query("SELECT * FROM pst_nw ID < '1000'");
while ($row = mysql_fetch_assoc($find))
{
$find_1 = mysql_query("SELECT * FROM pst_nw add1 = '$row[add1]' AND add2 = '$row[add2]' AND add3 = '$row[add3]' AND add4 = '$row[add4]'");
if (mysql_num_rows($find_1) > 0) {
mysql_query("DELETE FROM pst_nw WHERE ID ='$row[ID]'}
}
You have a number of options.
Let the DB do the work
Create a copy of your table with a unique index - and then insert the data into it from your source table:
CREATE TABLE clean LIKE pst_nw;
ALTER IGNORE TABLE clean ADD UNIQUE INDEX (add1, add2, add3, add4);
INSERT IGNORE INTO clean SELECT * FROM pst_nw;
DROP TABLE pst_nw;
RENAME TABLE clean pst_nw;
The advantage of doing things this way is you can verify that your new table is correct before dropping your source table. The disadvantage is it takes up twice as much space and is (relatively) slow to execute.
Let the DB do the work #2
You can also achieve the result you want by doing:
set session old_alter_table=1;
ALTER IGNORE TABLE pst_nw ADD UNIQUE INDEX (add1, add2, add3, add4);
The first command is required as a workaround for the ignore flag being .. ignored
The advantage here is there's no messing about with a temporary table - the disadvantage is you don't get to check that your update does exactly what you expect before you run it.
Example:
CREATE TABLE `foo` (
`id` int(10) NOT NULL AUTO_INCREMENT,
`one` int(10) DEFAULT NULL,
`two` int(10) DEFAULT NULL,
PRIMARY KEY (`id`)
)
insert into foo values (null, 1, 1);
insert into foo values (null, 1, 1);
insert into foo values (null, 1, 1);
select * from foo;
+----+------+------+
| id | one | two |
+----+------+------+
| 1 | 1 | 1 |
| 2 | 1 | 1 |
| 3 | 1 | 1 |
+----+------+------+
3 row in set (0.00 sec)
set session old_alter_table=1;
ALTER IGNORE TABLE foo ADD UNIQUE INDEX (one, two);
select * from foo;
+----+------+------+
| id | one | two |
+----+------+------+
| 1 | 1 | 1 |
+----+------+------+
1 row in set (0.00 sec)
Don't do this kind of thing outside the DB
Especially with 40 million rows doing something like this outside the db is likely to take a huge amount of time, and may not complete at all. Any solution that stays in the db will be faster, and more robust.
Usually in questions like this the problem is "I have duplicate rows, want to keep only one row, any one".
But judging from the code, what you want is: "if a set of add1, add2, add3, add4 is duplicated, DELETE ALL COPIES WITH ID < 1000". In this case, copying from the table to another with INSERT IGNORE won't do what you want - might even keep rows with lower IDs and discard subsequent ones.
I believe you need to run something like this to gather all the "bad IDs" (IDs with a duplicate, the duplicate above 1000; in this code I used "AND bad.ID < good.ID", so if you have ID 777 which duplicates to ID 888, ID 777 will still get deleted. If this is not what you want, you can modify that in "AND bad.ID < 1000 AND good.ID > 1000" or something like that).
CREATE TABLE bad_ids AS
SELECT bad.ID FROM pst_nw AS bad JOIN pst_nw AS good
ON ( bad.ID < 1000 AND bad.ID < good.ID
AND bad.add1 = good.add1
AND bad.add2 = good.add2
AND bad.add3 = good.add3
AND bad.add4 = good.add4 );
Then once you have all bad IDs into a table,
DELETE pst_nw.* FROM pst_nw JOIN bad_ids ON (pst_nw.ID = bad_ids.ID);
Performances will greatly benefit from a (non_unique, possibly only temporary) index on add1, add2, add3, add4 and ID in this order.
Get the duplicate rows using "Group by" operator. Here is a sample that you can try :
select id
from table
group by matching_field1,matching_field2....
having count(id) > 1
So, you are getting all the duplicate ids. Now delete them using a delete query.
Instead of using "IN", use "OR" operator as "IN" is slow compared to "OR".
Sure there is. Note however that with 40 million records You most probably will exceed max php execution time. Try following
Create table temp_pst_nw like pst_nw;
Insert into temp_pst_nw select * from pst_nw group by add1,add2,add3,add4;
Confirm that everything is ok first!!
Drop table pat_nw;
Rename table temp_pst_nw to pst_nw;
Try creating a new table that has the same definitions. i.e. "my_table_two", then do:
SELECT DISTINCT unique_col1, col2, col3 [...] FROM my_table INTO
my_table_two;
Maybe that'll sort it out.
Your code will be better if you don't use select *, only select columns (4 address) you want to compare. It should have limit clause in my sql. It can avoid state not response when you have too large nums rows like that.

How can I SELECT the last row that NO ID increment

How can I SELECT the last row in a MySQL table?
I'm INSERTING data and I need to retrieve a column value from the previous row.
(I'm using PHP by the way.)
the table1 something like this
table1
******************
cate_id | task_id | start_date | end_date | line |
1 2 30/04/2012 26/06/2012 text
3 1 26/06/2012 27/06/2012 text
2 1 27/06/2012 01/01/9999 text
There'sNO an auto_incrementin that table.
And my case is to update the existing last row in table and then insert a new one.
You've edited question so, here's update
SELECT MAX(cate_id) AS last_cate_id FROM table;
or you can get next ID by:
SELECT MAX(cate_id)+1 AS next_cate_id FROM table;
Without transactions this is very vulnerable for inserting same cate_id!
If you cant use them, for example because of MyISAM, you could insert with select.
INSERT INTO table
(cate_id, task_id ..)
VALUES
( (SELECT MAX(cate_id)+1 AS next_cate_id FROM table), 1 )
if you don't have order by you wont be able to get the "LAST" value or the first, because the order will not be the same (necessarily), if you don't have auto increment how can you know which one is the first or the last?, if you are working with date or auto increment you will be able to get that, however, lets say that you have a order by 'column1' you can do something like:
select * from table1 order by `column1` desc limit 1

How can I insert a row if it doesn't already exist while updating multiple rows?

I have a MySQL query that looks like this:
UPDATE `Table` SET `Column` =
CASE
WHEN `Option Id` = '1' THEN 'Apple'
WHEN `Option Id` = '2' THEN 'Banana'
WHEN `Option Id` = '3' THEN 'Q-Tip'
END
An my table currently looks like this:
Option Id | Column
1 | x
2 | x
I'd like it to result in:
Option Id | Column
1 | Apple
2 | Banana
3 | Q-Tip
But it doesn't insert the Q-Tip row. I've looked up and read a bit about INSERT ON DUPLICATE KEY UPDATE and REPLACE, but I can't find a way to get those to work with this multiple row update using CASE. Do I have to write a separate query for each row to get this to work, or is there a nice way to do this in MySQL?
Option Id is not a Key itself, but it is part of the Primary Key.
EDIT Some more info:
I'm programming in PHP, and essentially I'm storing an array for the user. Option Id is the key, and Column is the value. So for simplicities sake, my table could look like:
User Id | Option Id | Value
10 | 1 | Apple
10 | 2 | Shoe
11 | 1 | Czar
...
That user can easily update the elements in the array and add new ones, then POST the array to the server, in which case I'd like to store it in the table. My query above updates any array elements that they've edited, but it doesn't insert the new ones. I'm wondering if there is a query that can take my array from POST and insert it into the table without me having to write a loop and have a query for every array element.
This should work, if Option_Id is a primary key:
REPLACE INTO `Table` (`Option_Id`, `Column`) VALUES
(1, 'Apple'),
(2, 'Banana'),
(3, 'Q-Tip');
The statement means: Insert the given rows or replace the values, if the PK is already existing.
Of course it does not insert. As there is no such value, it cannot get updated.
I suppose you are normalizing a database by putting in the values already present and now want to add the required mapping for every valid value.
So it would be better to start from scratch and just do INSERTs.
You could always query the database for entries and then choose update or insert based on yor results
I see no point in such updating.
Why don't you have a separate table with option ids and corresponding values, leaving only option ids linked to user ids in this one?

PHP mysql ordering rows

For example, I have a table which looks like this :
id | name
1 | Mike
2 | Adam
3 | John
4 | Sarah
...
Now, when I execute query select * from table order by id desc it will output something like this:
4 | Sarah
3 | John
2 | Adam
1 | Mike
Now what do I do if I want to move John's row up or down, or move Adam's row up or down ( with a MySQL query ( I need basic one, just to know from where to start )).
My solution :
First of all, I created another column named orderID which has the same value as id.
Here is an example which moves up a user:
$query = "
SELECT (
SELECT orderID
FROM test WHERE id = 'user id that i want to move up'
) AS user_order,
(
SELECT orderID
FROM test WHERE orderID > user_order
ORDER BY orderID
LIMIT 0,1
) AS nextUser_order
";
$result = mysql_query($query);
$data = mysql_fetch_assoc($result);
$query = "
UPDATE test SET orderID = IF(orderID='{$data[nextUser_order]}',
'{$data[user_order]}', '{$data[nextUser_order]}')
WHERE orderID IN ('{$data[nextUser_order]}', '{$data[user_order]}');
";
$result = mysql_query($query);
Is there a better way to do that?
You have to switch IDs, or to order it by another column. That's the only way.
Changing the id is not what you want to do. You never want to mess with your primary key especially because later down the road it would be easier (and take up much less space, one is an int the other a varchar) to reference your users using their id rather than their name from other tables, it is nice to have a field that you know will never change.
Make another field such as order as a floating point number.
When you move foo between bar and foobar, set foo's order to the average of bar and foobar's order.
You can put arbitrary values into an order by clause in a query, but none will work easily for a simple "move up/down a row" type things. You can force certain values to sort first or last, but not "put this value after that value, but let that value go into its natural place". You'd need to have an extra field to specify sorting order.
SQL tables aren't inherently ordered - they effectively behave like a "bag of rows". If you want the results in a specific order, you will need to sort them (using ORDER BY ...) when you pull them out of the bag -- otherwise, the SQL server will return them in whatever order it feels is easiest. (In this case, they're coming out in the reverse order you inserted them, but that's not guaranteed at all.)
You should def be using another column which holds the order of the display. id is just a unique identifier. On a relational database moving up and down rows might result in a lot of queries because of the updates on the related tables so I stick with the idea of defining a special row for this purpose.

Categories