Working on some postgreSQL queries. As I said in a previous question..my knowledge of SQL logic is quite limited..
I have this query that inserts a row.
$timestamp = date('Y-m-d G:i:s.u');
$check_time = "start"; //can also be stop
$check_type = "start_user"; //can also be stop_user
$insert_query = "INSERT INTO production_order_process_log (
production_order_id,
production_order_process_id,
$check_time,
$check_type)
VALUES (
'$production_order_id',
'$production_order_process_id',
'$timestamp',
'$user')
";
Unfortunately, that is adding a new row every time. I would like to add conditional SQL so that
if the production_order_process_id doesn't exist, do the INSERT as it's written in the query above. That is, add the new row with all the new information
but if the production_order_process_id does exist and the check_type is stop_user then UPDATE the row to fill the column stop with the $timestamp and fill the column stop_user with $user.
I understand this is complicated.. Or, at least for me it is ^^ Thanks much for the help!
This is usually called MERGE or upsert. PostgreSQL doesn't have explicit support for this operation.
The best article I've seen on the topic of MERGE in PostgreSQL is this one by depesz .
After insert type ON DUPLICATE KEY UPDATE...
Use MERGE statement
Here is the usage
MERGE INTO table [[AS] alias]
USING [table-ref | query]
ON join-condition
[WHEN MATCHED [AND condition] THEN MergeUpdate | DELETE]
[WHEN NOT MATCHED [AND condition] THEN MergeInsert]
MergeUpdate is
UPDATE SET { column = { expression | DEFAULT } |
( column [, ...] ) = ( { expression | DEFAULT } [, ...] ) }
[, ...]
(yes, there is no WHERE clause here)
MergeInsert is
INSERT [ ( column [, ...] ) ]
{ DEFAULT VALUES | VALUES ( { expression | DEFAULT } [, ...] )
[, ...]}
(no subquery allowed)
I'm sure you'll find more articles/examples if you search for it.
It would be good if you can create a stored procedure and call while insert new record.
DELIMITER $$
DROP PROCEDURE IF EXISTS `DB`.`InsertNewRow` $$
CREATE PROCEDURE `db`.`InsertNewRow` ()
BEGIN
DECLARE V_EXIST INT DEFAULT 0;
DECLARE V_check_type VARCHAR(20);
SELECT production_order_process_id,check_type INTO V_EXIST,V_check_type FROM production_order_process_log;
IF V_EXIST=0 THEN
INSERT INTO production_order_process_log (
production_order_id,
production_order_process_id,
$check_time,
$check_type)
VALUES (
'$production_order_id',
'$production_order_process_id',
'$timestamp',
'$user');
ELSEIF V_check_type='stop_user' THEN
/* UPDATE QUERY HERE */
END IF;
END $$
DELIMITER ;
Just ad a WHERE CLAUSE to the insert:
INSERT INTO production_order_process_log
( production_order_id, production_order_process_id, check_time, check_type)
VALUES ( '$production_order_id', '$production_order_process_id', '$timestamp', '$user')
WHERE NOT EXISTS ( SELECT *
FROM production_order_process_log nx
--
-- assuming production_order_id is the Primary Key, here
--
WHERE nx.production_order_id = '$production_order_id'
);
UPDATE: I was confused by the parameters and the VALUE() . The fragment below works without parameters, but
with immediate values:
INSERT INTO tmp.production_order_process_log
( production_order_id, production_order_process_id, check_time, check_type)
SELECT 1, 2, '2012-07-19 12:12:12', 'Lutser'
WHERE NOT EXISTS ( SELECT *
FROM tmp.production_order_process_log nx
--
-- assuming production_order_id is the Primary Key, here
--
WHERE nx.production_order_id = 1
);
(you'll have to change it a bit to re-add the parameters)
Related
I have a table with a column 'search_text' type text.
In that field I have values:
1. 'MyBook MyBook PDF PDF',
2. 'Example 1 Example 2 Example 3'
3. 'John Snow John Snow'
I would like to distinct clean these fields.
Expected result:
1. 'MyBook PDF',
2. 'Example 1 2 3'
3. 'John Snow'
The approach I came up with goes as follows:
read the field for each record, split it by space (' '), put each text in array, do array_unique in PHP, then put the array back to string with join in PHP.
The thing is, this is a PHP based solution, I would like to have an MySQL solution for this. I have over 180.000 records I need to clean, I don't know what impact it would have to run this on PHP.
I have found a solution for MS SQL: Remove duplicate values in a cell SQL Server
Help greatly appreciated.
SQL of my test data:
CREATE TABLE IF NOT EXISTS `test` (
`id` int(10) unsigned NOT NULL,
`search_text` text COLLATE utf8_unicode_ci NOT NULL
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
INSERT INTO `test` (`id`, `search_text`) VALUES
(1, 'MyBook MyBook PDF PDF'),
(2, 'Example 1 Example 2 Example 3'),
(3, 'John Snow John Snow'),
(4, 'test test test test formula test test test formula test test test formula test test test formula test test test formula test test test formula '),
(5, '');
ALTER TABLE `test`
ADD PRIMARY KEY (`id`);
ALTER TABLE `test`
MODIFY `id` int(10) unsigned NOT NULL AUTO_INCREMENT,AUTO_INCREMENT=6;
Try this to sort by count :)
SELECT DISTINCT SUBSTRING_INDEX(SUBSTRING_INDEX(test.search_text, ' ', numbers.n), ' ', - 1) col_name
FROM (
SELECT 1 n
UNION ALL
SELECT 2
UNION ALL
SELECT 3
UNION ALL
SELECT 4
) numbers
INNER JOIN test ON CHAR_LENGTH(test.search_text) - CHAR_LENGTH(REPLACE(test.search_text, ' ', '')) >= numbers.n - 1
ORDER BY col_name;
You will need to write a MySQL function to do this for you. I would think that a PHP page will be just fine. 180,000 records isn't that many and it should (unless you are using a low spec server) run without putting much strain on anything else.
I wrote 2 for you that you might be able to make use of:
DROP PROCEDURE IF EXISTS explode;
DELIMITER //
CREATE PROCEDURE explode(str_string TEXT)
NOT DETERMINISTIC
BEGIN
DROP TABLE IF EXISTS explosion;
CREATE TABLE explosion (id INT AUTO_INCREMENT PRIMARY KEY NOT NULL, word VARCHAR(100));
SET #sql := CONCAT('INSERT INTO explosion (word) VALUES (', REPLACE(QUOTE(str_string), " ", '\'), (\''), ')');
PREPARE myStmt FROM #sql;
EXECUTE myStmt;
END //
DELIMITER ;
This procedure creates an "explode" function for use in MySQL. It uses a temporary table and explodes the words, separated by spaces into it
Then this function will read that table in, and put them into another temporary table with the duplicates removed:
DROP PROCEDURE IF EXISTS removeDuplicates;
DELIMITER //
CREATE PROCEDURE removeDuplicates(str TEXT)
BEGIN
DECLARE temp_word TEXT;
DECLARE last_word TEXT DEFAULT "";
DECLARE result TEXT;
DECLARE finished INT DEFAULT false;
DECLARE words_cursor CURSOR FOR
SELECT word FROM explosion;
DECLARE CONTINUE handler FOR NOT found
SET finished = true;
CALL explode(str);
DROP TABLE IF EXISTS temp_words;
CREATE TABLE temp_words (id INT AUTO_INCREMENT PRIMARY KEY NOT NULL, t VARCHAR(100));
OPEN words_cursor;
loop_words: LOOP
FETCH words_cursor INTO temp_word;
IF finished THEN
LEAVE loop_words;
END IF;
IF last_word = "" THEN
INSERT INTO temp_words (t) VALUES (temp_word);
SET last_word = temp_word;
ITERATE loop_words;
END IF;
IF last_word = temp_word THEN
SET last_word = temp_word;
ITERATE loop_words;
END IF;
INSERT INTO temp_words (t) VALUES (temp_word);
END LOOP loop_words;
CLOSE words_cursor;
END //
DELIMITER ;
So all you need to do is work out how to get the records in temp_words into your current database table.
I went for the PHP solution here:
$s = 'John Snow John Snow';
//remove duplicate values in string
$tmpArray = explode(" ", $s);
$tmpArray = array_unique($tmpArray);
$s = join(" ", $tmpArray);
Which is run before INSERT, and it does what I wanted.
I am learning SQL and I am in situation where I have to INSERT values into database only if it do not exist already.
My tableview structure is like this:
+----------+-----------+-----+-----+
| first_id | second_id | timestamp |
+----------+-----------+-----------+
I Want to insert only if there is not same first_id and second_id e.g if in table there is first_id 1 and second_id 2 and I am adding it again, I do not want to add it anymore. So if the first_id and second_id rows already have values 1 and 2 then do not add but if first_id is 3 and second_id is 1 then I would allow the inserting.
This is my query ATM:
INSERT INTO `testtable`.`ids` (`first_id`, `second_id`) VALUES (:first_id, :second_id)
And like this I tried with NOT EXISTS but it is not working:
NOT EXISTS (SELECT first_id, second_id FROM `testtable`.`ids` WHERE first_id = : first_id AND second_id = : second_id) INSERT INTO `testtable`.`ids` (`first_id `, `second_id `) VALUES (: first_id, : second_id)
The last mentioned query gives me Syntax error but once I even got the integrity violation and it told me to check documentation.
I am executing my queries using PHP ->query(""); function.
I tried to do like IF NOT EXISTS and NOT EXISTS but those didn't work. How should I approach to this?
This is simple. Declare first_id and second_id as composite key. I would prefer not to make any changes in your PHP Code but make your DB structure versatile so that it doesn't accept any duplicate values how-so-ever you are inserting it.
CREATE TABLE `demo` (
`first_id` smallint(6) DEFAULT NULL,
`second_id` smallint(6) DEFAULT NULL,
`timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
UNIQUE KEY `first_id` (`first_id`,`second_id`)
)
Now first_id and second_id will never accept duplicate values.
^ table contains values (1,2). Now inserting (1,3).
^the table accepts (1,3). Now inserting (1,2) again.
the insert statement throws an error. Now the table will never accept duplicate values for the key (first_id,second_id).
If the table already exists and you're not creating it from scratch, simply execute:
alter table `table_name` add unique key (first_id, second_id);
This will prevent the duplicate values henceforth.
You may try this if you are using PHP and MySql:
<?php
//Added database connection code here
$first_id = $_POST['first_id '];
$second_id = $_POST['second_id '];
$sql = "select * from ids where first_id = ".$first_id ." and second_id ='".$second_id."'" ;
$result = $mysqli->query($sql);
$row = $result->fetch_row();
if($row[0]) {
$mysqli->close();
} else {
//preapare an insert statement
$stmt = $mysqli->prepare(" INSERT INTO `ids` (first_id, second_id) VALUES (?,?)");
$stmt->bind_param("ii", $first_id, $second_id);
//execute the statement
if( $stmt->execute() ) {
unset($first_id);
unset($second_id);
} else {
echo $mysqli->error;
}
//close statement
$stmt->close();
$mysqli->close();
}
?>
Checking for existence for the first and second ids and additionally declare 2 sql variables for first and second ids and set them as per required.
DECLARE #Exists int; #first_id int; #second_id int;
SET #first_id = 1;
SET #second_id = 2;
SELECT #Exists = COUNT(*) FROM [testtable] where [first_id] = #first_id and [second_id] = #second_id;
Condition to insert if count of matched records is 0:
IF(#Exists = 0)
BEGIN
INSERT INTO [testtable](first_id, second_id)
VALUES(#first_id,#second_id)
END
I am doing an insert condition in oracle that when the record based on job and subjob doesnt exists, it shall insert otherwise, if it exists then it should update the rest of the value.
this is my procedure,
CREATE OR REPLACE PROCEDURE WELTESADMIN.SP_JOB_INS
(
JOB_V VARCHAR2,
SUBJOB_V VARCHAR2,
STARTDATE_V DATE,
ENDDATE_V DATE,
JOBWEIGHT_V NUMBER
)
AS BEGIN INSERT INTO PROJECT_SPAN (JOB, SUBJOB, STARTDATE, ENDDATE, WEIGHT) VALUES (JOB_V, SUBJOB_V, STARTDATE_V, ENDDATE_V, JOBWEIGHT_V);
EXCEPTION WHEN DUP_VAL_ON_INDEX THEN
UPDATE PROJECT_SPAN SET STARTDATE = STARTDATE_V, ENDDATE = ENDDATE_V, WEIGHT = JOBWEIGHT_V WHERE JOB = JOB_V AND SUBJOB = SUBJOB_V;
END;
/
and this is from PHP Call,
$insertJobSpanSql = "BEGIN SP_JOB_INS(:JOB, :SUBJOB, :SDATE, :EDATE, :WT); END;";
$insertJobSpanParse = oci_parse($conn, $insertJobSpanSql);
oci_bind_by_name($insertJobSpanParse, ":JOB", $jobValue);
oci_bind_by_name($insertJobSpanParse, ":SUBJOB", $subJobValue);
oci_bind_by_name($insertJobSpanParse, ":SDATE", $startDateValue);
oci_bind_by_name($insertJobSpanParse, ":EDATE", $endDateValue);
oci_bind_by_name($insertJobSpanParse, ":WT", $jobWeightValue);
$insertJobSpanRes = oci_execute($insertJobSpanParse);
if ($insertJobSpanRes){
oci_commit($conn);
} else {
oci_rollback($conn);
}
problem is it keeps inserting new row with the same job and subjob value. it should be an update to the new value.
First of all, I recommend to use MERGE in such cases:
CREATE OR REPLACE PROCEDURE WELTESADMIN.SP_JOB_INS
(
JOB_V VARCHAR2,
SUBJOB_V VARCHAR2,
STARTDATE_V DATE,
ENDDATE_V DATE,
JOBWEIGHT_V NUMBER
) AS
BEGIN
merge into PROJECT_SPAN ps
using (select JOB_V, SUBJOB_V, STARTDATE_V, ENDDATE_V, JOBWEIGHT_V
from dual) new_val
on (ps.SUBJOB = new_val.SUBJOB_V and ps.JOB = new_val.JOB_V)
when matched then update
set STARTDATE = new_val.STARTDATE_V,
ENDDATE = new_val.ENDDATE_V,
WEIGHT = new_val.JOBWEIGHT_V
when not matched then insert (JOB, SUBJOB, STARTDATE, ENDDATE, WEIGHT)
values (new_val.JOB_V, new_val.SUBJOB_V, new_val.STARTDATE_V,
new_val.ENDDATE_V, new_val.JOBWEIGHT_V );
END;
/
If it still not updating values, use package DBMS_OUTPUT or logging into a table to make sure, that new and old JOB and SUBJOB really the same.
Oracle raises DUP_VAL_ON_INDEX when a DDL statement violates a primary key or unique key constraint. So for your code to work you need to define a primary key on (job, subjob) or else build a unique index on that pair.
Oracle provides an easier way of implementing upserts, in the form of the MERGE statement:
CREATE OR REPLACE PROCEDURE WELTESADMIN.SP_JOB_INS
(
JOB_V VARCHAR2,
SUBJOB_V VARCHAR2,
STARTDATE_V DATE,
ENDDATE_V DATE,
JOBWEIGHT_V NUMBER
)
AS BEGIN
merge into project_span ps
using ( select job_v, subjob_v, startdate_v, enddate_v, jobweight_v from dual ) q
on (ps.job = q.job_v
and ps.subjob = q.subjob_v)
when not matched then
insert (job, subjob, startdate, enddate, weight)
values (q.job_v, q.subjob_v, q.startdate_v, q.enddate_v, q.jobweight_v);
when matched then
update
set startdate = q.startdate_v
, enddate = q.enddate_v
, weight = q.jobweight_v
;
END;
The MERGE statement is in the documentation. Find out more.
Can anybody help me to get the sub_cat(csv) values which are not in id column from same table.
(sub_cat) comma separated values are the ids of same table, i need to get the values which are not in id column. Like 2,3,7 are present in id column whereas 20,24 are not. I need to get 20,24 only.
As I have elaborated in this post, I recommend not storing data in CSV format.
This gives trouble accessing and updating it.
I am not sure about this, but you might be able to simply use:
SELECT sub_cat FROM table_name WHERE id NOT IN (
SELECT sub_cat FROM table_name
)
However, I always prefer to store only one ID per row.
To do this purely in MySQL required writing a stored procedure; you'll need to change the database name test to the actual name of your database. It would have been a whole lot easier to do it with PHP - but where's the fun in that?
DELIMITER //
CREATE PROCEDURE test.check_subcats(
IN s_delimiter VARCHAR(30)
)
DETERMINISTIC
READS SQL DATA
BEGIN
DECLARE s_csv TEXT;
DECLARE i_subcat_index INT(10) unsigned DEFAULT 1;
DECLARE i_subcat_count INT(10) unsigned;
DECLARE l_category_done INT(10) DEFAULT FALSE;
DECLARE c_category CURSOR FOR SELECT category.sub_cat FROM category;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET l_category_done = TRUE;
-- create a temporary table to hold every csv value
CREATE TEMPORARY TABLE IF NOT EXISTS tmp_csv( cvalue VARCHAR(10) NOT NULL );
OPEN c_category;
l_category: LOOP
FETCH c_category INTO s_csv;
IF l_category_done THEN
LEAVE l_category;
ELSE
-- determine the number of sub-categories
SELECT (LENGTH(s_csv) - LENGTH(REPLACE(s_csv, s_delimiter, ''))) + 1 INTO i_subcat_count;
-- loop to store all csv values
WHILE i_subcat_index <= i_subcat_count DO
INSERT INTO tmp_csv ( cvalue ) (
SELECT REPLACE(SUBSTRING(
SUBSTRING_INDEX(s_csv, s_delimiter, i_subcat_index),
LENGTH(SUBSTRING_INDEX(s_csv, s_delimiter, i_subcat_index - 1)) + 1
), s_delimiter, '')
);
SET i_subcat_index = i_subcat_index + 1;
END WHILE;
END IF;
SET i_subcat_index = 1;
END LOOP;
CLOSE c_category;
SELECT DISTINCT tmp_csv.cvalue FROM tmp_csv WHERE tmp_csv.cvalue NOT IN ( SELECT category.id FROM category );
DROP TEMPORARY TABLE IF EXISTS tmp_csv;
END //
DELIMITER ;
I'm not 100% certain as to how robust it is but it was working with your data on my dev box.
You specify the delimiter for your CSV data when you call the procedure thus:
CALL `check_subcats`(',');
Essentially this loops through the category table to read sub_cat. It then splits the sub_cat value into chunks using the delimiter provided (much like PHPs explode() function) and writes every one of those values to a temporary table.
This then gives you a temporary table holding all your CSV data in individual bits and it's then a simple matter of selecting everything from that data that's NOT IN the category.id list.
I have below data in Mysql Column (storing all data in serialize form - with comma separated into column) and i want to get/fetch this column data in Mysql stored procedure and want to loop for each data and insert into another trans table.
So if my data like below then i want to insert 7 record in trans table.
{"FormBuilderId":"5","vAnswer":"Develeop"},
{"FormBuilderId":"15","vAnswer":"Search Engine"},
{"FormBuilderId":"13","vAnswer":"10-15"},
{"FormBuilderId":"6","vAnswer":"Tester entered"},
{"FormBuilderId":"1","vAnswer":"Female"},
{"FormBuilderId":"14","vAnswer":"Moon.jpg"},
{"FormBuilderId":"12","vAnswer":"TV,dancing and modeling"}
My table structure & data is like below in table:
CREATE TABLE IF NOT EXISTS `idea_history` (
`iIdeaHistoryId` int(11) NOT NULL AUTO_INCREMENT,
`iApplicationId` tinyint(11) unsigned DEFAULT '0',
`tAnswerData` text COMMENT 'all answer data store here in json format',
`dAddedDate` datetime NOT NULL,
PRIMARY KEY (`iIdeaHistoryId`),
KEY `iApplicationId` (`iApplicationId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='history comes here' AUTO_INCREMENT=57 ;
--
-- Dumping data for table `idea_history`
--
INSERT INTO idea_history (
iIdeaHistoryId,
iApplicationId,
tAnswerData,
dAddedDate
)
VALUES
(
53,
2,
'{"FormBuilderId" : "2","vAnswer":"Environmental Group"},{"FormBuilderId" : "11","vAnswer":"Satelite"},{"FormBuilderId" : "3","vAnswer":"HB"},{"FormBuilderId" : "4","vAnswer":"Dev"},{"FormBuilderId" : "7","vAnswer":"HB"},{"FormBuilderId" : "8","vAnswer":"Balaji Satellite"},{"FormBuilderId" : "10","vAnswer":""}',
'2014-07-05 19:20:56'
),
(
54,
1,
'{"FormBuilderId":"5","vAnswer":"Hello krishna|kanth double"},{"FormBuilderId":"15","vAnswer":"Website"},{"FormBuilderId":"6","vAnswer":"need to check"},{"FormBuilderId":"13","vAnswer":"20-25"}',
'2014-07-05 19:20:56'
),
(
55,
2,
'{"FormBuilderId":"11","vAnswer":"comapnay"},{"FormBuilderId":"8","vAnswer":"here am|chw "},{"FormBuilderId" : "10","vAnswer":""},{"FormBuilderId":"9","vAnswer":"Business"}',
'2014-07-05 19:20:56'
) ;
I will pass iIdeaHistoryId in stored procedure and it will fetch value of tAnswerData field and part this value and insert into another trans table.
Could you please guide what i have to change in stored procedure ?
you can do it like this
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(your_variable,'{"',-1),'":"',1) INTO arg_1;
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(your_variable,'":"',-2),'","',1) INTO arg_2;
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(your_variable,'","',-1),'":"',1) INTO arg_3;
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(your_variable,'":"',-1),'"}',1) INTO arg_4;
INSERT INTO trans VALUES (arg_1,arg_2,arg_3,arg_4);
or if you want to do it in store procedure add your data here
DELIMITER $$
DROP PROCEDURE IF EXISTS trans$$
CREATE PROCEDURE trans()
BEGIN
DECLARE arg_1,arg_2,arg_3,arg_4 VARCHAR(100);
DECLARE finished INTEGER DEFAULT 0;
DECLARE cursor_name CURSOR FOR YOUR_SELECT_statement;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
OPEN cursor_name;
my_loop: LOOP
FETCH cursor_name INTO your_variable;
IF finished = 1 THEN
LEAVE my_loop;
END IF;
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(your_variable,'{"',-1),'":"',1)
INTO arg_1 FROM your_table;
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(your_variable,'":"',-2),'","',1)
INTO arg_2 FROM your_table;
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(your_variable,'","',-1),'":"',1)
INTO arg_3 FROM your_table;
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(your_variable,'":"',-1),'"}',1)
INTO arg_4 FROM your_table;
INSERT INTO trans VALUES (arg_1,arg_2,arg_3,arg_4);
END LOOP my_loop;
CLOSE cursor_name;
END$$
DELIMITER ;
for one column you use this
DELIMITER $$
DROP PROCEDURE IF EXISTS trans$$
CREATE PROCEDURE trans(IN in_iIdeaHistoryId INT)
BEGIN
DECLARE arg_1 VARCHAR(100);
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(REPLACE(`tAnswerData`,' ',''),'{"',-1),'":"',1)
INTO arg_1 FROM `idea_history` WHERE iIdeaHistoryId=in_iIdeaHistoryId;
INSERT INTO trans VALUES (arg_1);
END$$
DELIMITER ;
and call it like this
CALL `trans`(53);
This looks like JSON. Use php function json_decode() to store the json data into an array and then you can use foreach loop on that array. Inside the loop you can place condition and when its met, you could use insert query to store it in your database.
Does this help you?
If you can explain a bit further what you're trying to do exactly and what are your condition, then i Can probably give you a better answer.