Could please somebody help me find out how to iterate these raw txt data to mysql. The format is
user id | item id | rating | timestamp
and i want to insert these data to my table in MySql (using PHPmyAdmin), let's say the table structure is : user_id (int), item_id(int), rating(int), timestamp(int) with its name "Rating".
So, i want to know how to insert these data to my table, i'm fine with php, or if there are easier way to do this.
If you want to generate raw SQL queries, you can do so by using find and replace in your text editor (that looks like Notepad++). I'm guessing that your delimiters are tabs.
Find and replace all tab characters and replace them with a comma. We do not need to quote anything as all of your fields are integers.
Find and replace all newline characters and replace them with a SQL query.
Execute these commands in regular expression mode:
Columns
Find: \t
Replace: ,
Rows
Find: \r\n (if that doesn't find anything, look for \n)
Replace: );\r\nINSERT INTO Rating (user_id, item_id, rating, timestamp) VALUES (
On the first row, insert the text INSERT INTO Rating (user_id, item_id, rating, timestamp) VALUES ( to make the row a valid SQL statement.
On the last row, remove any trailing portions of SQL query after the last semicolon.
Copy and paste this into your PHPMyAdmin and it should be all good.
The simplest way I have found for doing similar is to use Excel. Import the text file into a new document - judging by the look it should be easy to seperate the columns as they appear to be tab delimited. Once you have the required columns set up a string concatenation to include the values... kind of like
=CONCATENATE("INSERT INTO Rating SET user_id='",A1,"', item_id='",B1,"', rating='",C1,"', timestamp='",D1,"';")
Then repeat for all rows, copy and paste into sql client
you can use toad for mysql , import wisard and you create a table with the same structure (user id | item id | rating | timestamp) of you file after import all data you export the sql insert of you new table.
Related
I'm using implode to insert few values into one row in MySQL database.
implode(' ', $_POST['tag']);
Assuming that I have table named product with row named tags with 3 different values that inserted inside like this:
usb adapter charger
I have tried using this method using like operator (%), but that didn't worked.
$sql = "SELECT * FROM product WHERE tags='%usb%'";
How can I extract only one value from the imploded array using WHERE in mysql query?
I agree with the comments about re-designing the database. At first read it seems that using LIKE would definitely get the result you want but after reading #Patrick Q's pan - panther example, it makes a lot sense that LIKE is not really a good solution. There are ways to get exactly the tag string you're looking for but it may hurt the performance and the query will be longer and complex. Hence the following are to demonstrate how the query would look like with your current tags data value:
MySQL query:
SELECT tags,
SUBSTRING_INDEX(SUBSTRING_INDEX(tags,' ',FIND_IN_SET('usb',REPLACE(tags,' ',','))),' ',-1) v
FROM mytable
HAVING v = 'usb';
As you can see, there are a few functions being used just to get the exact string from the data cell. Since your example data was separating with spaces and FIND_IN_SET identify value separation by comma, REPLACE take place on the tags column first to replace spaces with comma. Then with SUBSTRING_INDEX twice to get the string using the location extracted in FIND_IN_SET. Finally at the end HAVING to get only the tag you're looking for.
Further demo here : https://www.db-fiddle.com/f/joDa7MNcQL2RakTgBa7qBM/3
Using PHP a secure user will enter a Ref (ex. NB093019) a query will be used to determine which PO(s) have that Ref and if they have any quantity. The issue is that we have 86 columns to check if that Ref is in and then once it finds what column it is in how to check the corresponding column that contains that quantity( the table cannot be edited).
I can make this work with 86 if else statements in PHP and then more if else statements inside of each PHP statement. I have no launching point once i do the initial query.
select 'remainder'as prefix, po, *comments,*GuideRef, *Qty
from remainder
where ('NB092419')IN (NWANTcomments,NWANTGuideRef,NWANTpreviouscomments,
NWANTpreviousGuideRef,NWANTprevious2comments,
NWANTprevious2GuideRef, BPrev2GuideRef,
BPrev2comments, BPrevGuideRef, BPrevcomments,
aGuideRef, Mcomments,MGuideRef,acomments,
MAGuideRef,BOGuideRef )
group by po
I have removed some of the in() information so it is not so long also the *comments, *GuideRef, *Qty would be decided by which one of the columns in the IN() statement returns information. Is this even possible
You could perhaps write an SQL that writes an SQL:
select REPLACE(
'SELECT ''{colstub}GuideRef'' as which, {colstub}Qty FROM remainder WHERE {colstub}Ref like ''%somevalue%'' UNION ALL',
'{colstub}',
REPLACE(column_name, 'GuideRef', '')
)
FROM information_schema.columns
WHERE table_name = 'remainder' and column_name LIKE '%Ref'
It works like "pull all the column names out of the info schema where the column name is like %guideref, replace guideref with nothing to get just the fragment of the column name that is varied: NWANTguideref -> NWANT, NWANTpreviousguideref -> NWANTprevious ... then uses this stub to form a query that gives a string depicting the column name, the qty from the quantity column, where the relevant guideref column is LIKE some value"
If you run this it will produce a result set like:
SELECT 'aGuideRef' as which, aQty FROM table WHERE aGuideRef LIKE '%lookingfor%' UNION ALL
SELECT 'bGuideRef' as which, bQty FROM table WHERE bGuideRef LIKE '%lookingfor% ...
So it's basically utputted a load of strings that are SQLs in themselves. It might need a bit of fine tuning, and hopefully all your columns are reliably and rigidly like xQty, xGuideRef, xComments triplets, but it essentially writes most the query for you
If you then copy the result set out of the results grid and paste it back into the query window, remove the last UNION ALL and run it, it will search the columns and tell you where it was found as well as the quantity
It's not too usable for a production system, but you could do the same in php- run the query, get the strings into another sql command, re-run it..
I would suggest you consider changing your table structure though:
prefix, qty, guideref, comments
You shouldn't have 86 columns that are the mostly same thing; you should have one column that is one of 86/3 different values then you can just query the guideref and the type. If this were an address table, I'm saying you **shouldn't* have HomeZipcode, WorkZipcode, UniversityZipcode, MomZipcode, DadZipcode.. and every time you want to store another kind of address you add more columns (BoyfriendZipcode, GirlfriendZipcode, Child1Zipcode...). Instead if you just had an "addresstype" column then you can store any number of different kinds of addresses without recompiling your app and changing your db schema
You can use this technique to re-shape the table - write an SQL that writes a bunch of UNION ALL sqls (without WHERE clauses), one of the columns should be the "recordtype" column (from colstub) and the other columns should just be "qty", "guide", "comments". Once you have your result set with the unions you can make a table to hold these 4 things, and then place INSERT INTO newtable at the head of the block of unions
How to insert multiple strings into specific column with space?
Example:
$this->database[ABB]->doQuery('INSERT INTO MARKET (Account, User, Rank, ChangeTime) VALUES (?,?,?, GetDate())', $_SESSION['Account'], $user, $rank);
The values are sanitized in my full code above this query however I want to combine Account & User columns into one (AccUserInfo) and when they are inserted, it should insert them like that with space between: AccEx UserEx
Is it possible to do that in my php script when its inserted to the MSSQL Server 2005 table?
I will change the db structure, just wanna know how to include both strings into the query with space between them?
to join any 2 stings together with a space:
$foo=$cat.' '.$dog;
your particular case:
$AccUserInfo=$_SESSION['Account'].' '.$user;
I have a database table with fields like:
username, description, password.
Sometimes, members copy the description from one-another, to save time
So I have this:
John - John's description - John's password
Michael - John's description - Michael's password
Is there a mysql query that searches for duplicate field entries and deletes them?
How about deleting the entire row of data while we're at it?
Create a new table, move non-duplicate entries in it, remove the old table and rename a new one.
Example:
CREATE TABLE `new_table` as
SELECT * FROM `old_table` WHERE 1 GROUP BY [columns to remove duplicates by];
DROP TABLE `old_table`;
RENAME TABLE `new_table` TO `old_table`;
However, it's good for periodically uses only, and it doesn't check is there are any duplicates. It just groups unique entries and moves them to another table. It's useful when you want to filter your entries.
Another way is to check like this:
SELECT `name` FROM `table` WHERE `description` LIKE '%descriptiontexthere%'
Then, if some results are found, it's a duplicate.
However, there's a big disadvantage: user can change only 1 letter and query will fail.
That disadvantage, however, can be avoided (not fully) by splitting a description into an array. For example, split on every 100 characters and then check like in above example, but with multiple conditions (e.g. description LIKE 'first100chars' OR description LIKE 'second100chars') .
The third way is to split a description to an array of words and then select rows with too many same words. Row with X same words could be a possible duplicate. You can set the treshold based on length of an entered description.
You can never be sure if it's duplicate or not, unless it's completely the same entry.
I am trying to understand how to write a query to import my goodreads.com book list (csv) into my own database. I'm using PHP.
$query = "load data local infile 'uploads/filename.csv' into table books
RETURN 1 LINES
FIELDS TERMINATED BY ','
ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
(Title, Author, ISBN)";
This results in data in the wrong columns, I also can't get the ISBN, no matter what I try, guessing the equals (=) sign is causing some problems.
Here's the first two lines of the CSV:
Book Id,Title,Author,Author l-f,Additional Authors,ISBN,ISBN13,My Rating,Average Rating,Publisher,Binding,Number of Pages,Year Published,Original Publication Year,Date Read,Date Added,Bookshelves,Bookshelves with positions,Exclusive Shelf,My Review,Spoiler,Private Notes,Read Count,Recommended For,Recommended By,Owned Copies,Original Purchase Date,Original Purchase Location,Condition,Condition Description,BCID
11487807,"Throne of the Crescent Moon (The Crescent Moon Kingdoms, #1)","Saladin Ahmed","Ahmed, Saladin","",="0756407117",="9780756407117",0,"3.87","DAW","Hardcover","288",2012,2012,,2012/02/19,"to-read","to-read (#51)","to-read",,"",,,,,0,,,,,
What I want to do is only import the Title, Author, and ISBN for all books to a table that has the following fields: id (auto), title, author, isbn. I want to exclude the first row as well because I dont want the column headers, but when I have tried that it fails every time.
What's wrong with my query?
Unfortunately, in case you want to use the native MySQL CSV parser, you will have to create a table that matches the column count of the CSV file. Simply drop the unnecessary columns when the import is finished.