Just wondering what the best way to update an entire (large) database would be.
Bascially I have inherited a DB which had character issues, with some help I have sorted the character issues going forward (being written in) however the existing data needs cleaning up.
There was a good suggestion that I could use utf_decode to clean all this up - I have tried this on a wrong value in the page itself (when pulled in) and it works great.
As there seems to be a lot of tables, and alot of data, what's the best / quickest way to sweep all the data in the entire DB by using utf_decode ?
Thanks
Thanks for the comments, I can't seem to comment directly so dropping as message in here - I will have a look through and give them a go ! thanks.
Have you tried using the MySQL function CONVERT? Depending on your data, you may be able to update tables in a single statement, such as "UPDATE mytable SET myfield = CONVERT(myfield USING utf8)".
http://dev.mysql.com/doc/refman/5.0/en/charset-convert.html
Fetch all the data, convert it and insert it as:
INSERT INTO table VALUES (id, text)
(1, 'utf8'), (2, 'utf8'), (3, 'utf8')
etc.
Goes faster then to do a php loop with multi INSERT queries.
EDIT:
If you use a nice array, you could run a smooth system to do this:
$arr = array('users' => array('user_id', 'text', 'username', 'first_name')));
foreach(array_keys($arr) as $h) {
$query = mysql_query("SELECT * FROM {$h}");
while($row = mysql_fetch_object($query)) {
// Loop thingies, utf8_decode then and stuff
}
// Then implode them nicely and use above query
}
Tell me if you need more code example.
Related
I have stored the datas in db with the addslashes when i submit the form.
Im stored values using the function like below
addslashes(trim($data[1]));
I want to check existing record in that table but its not working when it has value like
Regional Sales Director - Americas\'
Its checking existing values in table without those shlashes
\'
My query is
$query = $this->db->query("select * from tbl_contacts where contact_name='".$name."' and contact_company='".$company."' and contact_designation='".$designation."'");
$result1 = $query->result();
I'm not sure i should answer to this, but i feel i should because what you are doing is wrong in so many ways...
You should never ever do things like that if you want to insert data - especially if you use a framework which can do the job for you...
First of all you've to understand how Codeigniter inserts Data
Use the query builder
an example for inserting data would be
$arrData = [
'contact_name' => $this->input->post('contact_name'),
'contact_company' => $this->input->post('contact_company')
];
$this->db->insert('tbl_contacts', $arrData);
please read carefully the section in the CI Documentation here
and your select query is a disaster because you don't protect anything - you are widely open to any sort of attacks as Alex already said in the comments
Instead you should try the following:
$query = $this->db
->select('*')
->from('tbl_contacts')
->where('contact_name', $name)
->where('conatct_company', $company)
->where('contact_designation', $designation)
->get();
$result1 = $query->result();
Furthermore, please study the documentation, below are some links which are mandatory
Form Validation
Security Class
Input Class
Query Builder Class
If you've accidentally escaped your data with addslashes on top of existing database escaping: you may be able to methodically remove those back slashes with an update and replace to fix your data.
UPDATE tableName
SET columnName = replace(columnName, '\\', '');
But do be very careful, back up all your data first and test on a sample.
Then in the future, do not use addslashes on top of your database library's escape mechanism for updates or inserts.
Someone retired in our group and I'm trying to figure out what his merge statement (and associated code) does so I can determine how to convert some (not all) values to integer before sending up. See comments below for questions. I am an absolute newbie with Microsoft SQL and took a class in php a few years ago, but don't have much experience. I've tried googling the merge command but I'm having trouble with a couple parts in it. See my questions below. (// ?)
I've looked at:
http://php.net/manual/en/pdo.query.php
http://stackoverflow.com/questions/4336573/merge-to-target-columns-using-source-rows
http://pic.dhe.ibm.com/infocenter/iseries/v7r1m0/index.jsp?topic=%2Fsqlp%2Frbafymerge.htm
I realize these are basic questions but I'm trying to figure it out and nobody around here knows.
function storeData ($form)
{
global $ms_conn, $QEDnamespace;
//I'm not sure what this is doing?? I thought this was where it was sending data up??
$qry = "MERGE INTO visEData AS Target
USING (VALUES (?,?,?,?,?,?,?,?,?,?))
AS Source (TestGUID,pqID, TestUnitID, TestUnitCountID,
ColorID, MeasurementID, ParameterValue,
Comments, EvaluatorID, EvaluationDate)
ON Target.pqID = Source.pqID
AND Target.MeasurementID=Source.MeasurementID //what is this doing?
AND Target.ColorID=Source.ColorID //what is target and source?
WHEN MATCHED THEN
UPDATE SET ParameterValue = Source.ParameterValue,
EvaluatorID = Source.EvaluatorID, //where is evaluatorID and source? My table or table we're send it to?
EvaluationDate = Source.EvaluationDate,
Comments = Source.Comments
WHEN NOT MATCHED BY TARGET THEN
INSERT (TestGUID,
pqID, TestUnitID, TestUnitCountID,
ColorID, MeasurementID,
ParameterValue, Comments,
EvaluatorID, EvaluationDate, TestIndex, TestNumber)
VALUES (Source.TestGUID, Source.pqID,
Source.TestUnitID,
Source.TestUnitCountID,
Source.ColorID, Source.MeasurementID, Source.ParameterValue,
Source.Comments, Source.EvaluatorID, Source.EvaluationDate,?,?);";
$pqID = coverSheetData($form);
$tid = getBaseTest($form['TextField6']);
$testGUID = getTestGUID($tid);
$testIndex = getTestIndex ($testGUID);
foreach ($form['visE']['parameters'] as $parameter=>$element)
{
foreach ($element as $key=>$data)
{
if ( mb_ereg_match('.+evaluation', $key) === true )
{
$testUnitData = getTestUnitData ($form, $key, $tid, $testGUID);
try
{
//I'm not sure if this is where it's sent up??
//Maybe I could add the integer conversion here??
$ms_conn->query ($qry, array(
$testGUID, $pqID,
$testUnitData[0], $testUnitData[1], $testUnitData[2],$element['parameterID'], $data, $element['comments'] $QEDnamespace->userid, date ('Y-m-d'), $testIndex, $tid));
}
catch (Zend_Db_Statement_Sqlsrv_Exception $e)
{
dataLog($e->getMessage());
returnStatus ("Failed at: " . $key);
}
}
}
}
}
This is a bit long for a comment. If you are using SQL Server, then look at the SQL Server documentation on merge. All the SQL Server documentation is on line, and it is very easy to find via Google (and perhaps even easier using Bing).
The purpose of the MERGE command is to do both inserts and updates in one step. Basically, you have a table that has new data ("source") and a table to be updated ("target"). When a record matches, then update the existing record in the target with matching record in source. When a record doesn't match, then insert it into target.
The main advantage of MERGE over two statements is not necessarily the elegant and intuitively obvious syntax. The main advantage is that all the operations occur in a single transaction, so either they all succeed or all fail as one.
The syntax actually isn't that bad. I would recommend that you set up a test database and try a few examples on your own, so you at least understand the syntax. Then, return to this code. When doing so, print out the resulting merge statement and put it in SQL Server Management Studio, where you will have nice color coded key words for the statement. Then go through it step by step, and you'll probably find that it makes lots of sense.
i might be doing some idiot mistake, but i could not figure that out. i have some values coming from html and wanna insert into mysql db. problem is, the very same query does not work in regular php file (that includes other queries), but when i try on an independent php file, it does. here is a sample of the code:
$sql15="insert into body
(Article_ID, Article_Title)
values
('$article_id', '".$_POST['Article_Title']."') ";
mysql_query($sql15);
as i mentioned, the very same code works when i just copy this snippet to a new php file, and it works smoothly.. as you see, there are 20+ insert with the same php, because there are 25+ tables, but data is not much. first 14 query and following 7 queries do work by the way.
do you have any ideas?
There are some things to check and do.
Sanitize user input:
"('$article_id', '".mysql_real_escape_string($_POST['Article_Title'])."')";
You might also want to check if the value is what you expect.
Is your $article_id correct for column Article_ID?
Are your table and column names correct?
Check for errors:
$res = mysql_query($sql15);
if (!$res)
echo mysql_errno($link) . ": " . mysql_error($link);
Show us you complete query:
echo $sql15;
First of all i would suggest you to write your insert query like below
$sql15="insert into body SET Article_ID = '$article_id', Article_Title = '".$_POST['Article_Title']."'";
echo $sql15;
mysql_query($sql15);
so that each time when you add new column to database it would be easy for u to change insert query. echo your query and see it in browser. in it seems to o.k then copy it and paste it in SQL section under your phpmyadmin (see you are choosing proper database) and run it. if one row inserted successfully then your query is alright.
I hope this would help you a little.
$sql15="insert into body
(Article_ID, Article_Title)
values
('$article_id', '".$_POST['Article_Title']."') ";
mysql_query($sql15) or die(mysql_error());
use like this u will be get the error. then u will be find the issue
I think using mysql_real_escape_string may solve your problem.I also recommend you to store your form data in a string.
$article_title= mysql_real_escape_string($_POST['Article_Title']);
$sql15="insert into body
(Article_ID, Article_Title)
values
('$article_id', '$article_title') ";
mysql_query($sql15) or die(mysql_error());
I am using a classified scripts and saves user_meta data in the wp_usermeta table.
The meta_key field is called user_address_info and in it there are all the data like below :
s:204:"a:7:{s:9:"user_add1";s:10:"my address";s:9:"user_add2";N;s:9:"user_city";s:7:"my city";s:10:"user_state";s:8:"my phone";s:12:"user_country";N;s:15:"user_postalcode";s:10:"comp phone";s:10:"user_phone";N;}";
I am not using all the fields on the script but user_add1, user_city, user_state and user_postalcode
I am having trouble to get the data using SQL like the example below (wordpress) :
$mylink = $wpdb->get_row("SELECT * FROM $wpdb->links WHERE link_id = 10", ARRAY_A);
I would like some help here so that I will display anywhere (I dont mind using any kind of SQL queries) the requested info e.g. the user_city of current author ID (e.g. 25)
I was given the following example but I want something dynamic
<?php
$s = 's:204:"a:7:{s:9:"user_add1";s:10:"my address";s:9:"user_add2";N;s:9:"user_city";s:7:"my city";s:10:"user_state";s:8:"my phone";s:12:"user_country";N;s:15:"user_postalcode";s:10:"comp phone";s:10:"user_phone";N;}"';
$u = unserialize($s);
$u2 = unserialize($u);
foreach ($u2 as $key => $value) {
echo "<br />$key == $value";
}
?>
Thank you very much.
No, you can't use SQL to unserialize.
That's why storing serialized data in a database is a very bad idea
And twice as bad is doing serialize twice.
So, you've got nothing but use the code you've given.
I see not much static in it though.
do you experience any certain problem with it?
Or you just want to fix something but don't know what something to fix? Get rid of serialization then
i have found that the serialize value stored to database is converted to some other way format. Since the serialize data store quotes marks, semicolon, culry bracket, the mysql need to be save on its own, So it automatically putting "backslash()" that comes from gpc_magic_quotes (CMIIW). So if you store a serialize data and you wanted to used it, in the interface you should used html_entity_decode() to make sure you have the actual format read by PHP.
here was my sample:
$ser = $data->serialization; // assume it is the serialization data from database
$arr_ser = unserialize(html_entity_decode($ser));
nb : i've try it and it works and be sure avoid this type to be stored in tables (to risky). this way can solve the json format stored in table too.
I'm downloading large sets of data via an XML Query through PHP with the following scenario:
- Query for records 1-1000, download all parts (1000 parts has roughly 4.5 megs of text), then store those in memory while i query the next 1001 - 2000, store in mem (up to potentially 400k)
I'm wondering if it would be better to write these entries to a text field, rather than storing them in memory and once the complete download is done trying to insert them all up into the DB or to try and write them to the DB as they come in.
Any suggestions would be greatly appreciated.
Cheers
You can run a query like this:
INSERT INTO table (id, text)
VALUES (null, 'foo'), (null, 'bar'), ..., (null, 'value no 1000');
Doing this you'll do the thing in one shoot, and the parser will be called once. The best you can do, is running something like this with the MySQL's Benchmark function, running 1000 times a query that inserts 1000 records, or 1000000 of inserts of one record.
(Sorry about the prev. answer, I've misunderstood the question).
I think write them to database as soon as you receive them. This will save memory and u don't have to execute a 400 times slower query at the end. You will need mechanism to deal with any problems that may occur in this process like a disconnection after 399K results.
In my experience it would be better to download everything in a temporary area and then, when you are sure that everything went well, to move the data (or the files) in place.
As you are using a database you may want to dump everything into a table, something like this code:
$error=false;
while ( ($row = getNextRow($db)) && !error ) {
$sql = "insert into temptable(key, value) values ($row[0], $row[1])";
if (mysql_query ($sql) ) {
echo '#';
} else {
$error=true;
}
}
if (!error) {
$sql = "insert into myTable (select * from temptable)";
if (mysql_query($sql) {
echo 'Finished';
} else {
echo 'Error';
}
}
Alternatively, if you know the table well, you can add a "new" flag field for newly inserted lines and update everything when you are finished.