I have 2 million rows in a myqsl DB which have multiple columns of contacts as phone_1, phone_2 upto phone_10.
These phone no. may or may not duplicate.
I intend to group them together..as
ID Contact_1 Contact_2 Contact_3
P1 1 2 3
P2 5 6 7
P3 2 8 9
result should be:
ID Contact_1 Contact_2 Contact_3 Group
P1 1 2 3 1
P2 5 6 7 2
P3 2 8 9 1
P3 11 12 13 3
P3 7 21 22 2
Now where should I do the processing part ...PHP/Python or mysql.
i.e. select the entire data in php script and create an arrray and process tha array and then use insert query.
OR
select the entire data in php script and then use UPDATE(with a logic to create groups) query.
??
I have group field in DB table.
It depends, I'd say mostly on how comfortable you are with each language. I would probably do this with PHP, but that's the language I know best.
You can certainly do it purely with MySql, and the operation would probably run faster, but it might be easier to debug and test each step in PHP or Python.
Regardless, I'd recommend first creating a data subset, maybe 1000 or 2000 rows from the table and running everything against that until you're happy with the results. It'll be much much faster and you'll see mistakes sooner.
Also, I'd avoid running anything you're worried about being slow on a production server.
Related
The following table has the time when the user enters/quit on the company:
ID | user_id | day_unix | time
10 1 1459220400 1459293745
9 1 1459220400 1459293711
8 1 1459220400 1459293689
7 1 1459220400 1459293678
6 1 1459220400 1459293669
11 1 1459220400 1459293761
day_unix consists in the first second of the day, this way, GROUP BY can be easily used in future.
time consists in the time that the user click's on the button to start/stop working. Time also can be changed to a native DATETIME column.
I want to create a SQL query capable of summing the time between the entries. So the query must to jump "odd" entries that can be considered as the beginnig of a small coffee break and right after the "odd" entries the user has started working again, so the query must to sum the working time, excluding the coffee breaks.
Any idea? Here is what I've already got:
SELECT * FROM table GROUP BY day_unix ORDER BY time ASC
I am newbie in cakephp, and I am confused about SHOW FULL COLUMN query
I've set debug leel into 2, and I clicked on one page, its show 88 queries (even the data is NULL)
Here the queries:
SHOW FULL COLUMNS FROM item_purchase_returns 10 10 2
SHOW FULL COLUMNS FROM purchases 20 20 2
SHOW FULL COLUMNS FROM outcomes 13 13 1
SHOW FULL COLUMNS FROM transaction_categories 7 7 96
SHOW FULL COLUMNS FROM incomes 16 16 2
SHOW FULL COLUMNS FROM last_receivables 14 14 1
SHOW FULL COLUMNS FROM last_funds 7 7 1
...
etc
My question is why does cakePHP do this?
Does those queries are usefull?
Is cakePHP calls those queries on each page?
Thanks!
CakePHP calls those queries in order to build it's internal model of your database tables. It does call them quite often in development mode (the cache is flushed every 10 seconds) in case you make changes to your database, but if the debug level is set to production (0), it will assume you aren't making any database changes and only flush the cache every 99 years (essentially never). Thus, if you wish to make changes to the database of an app in production, you either have to delete your cached models in /app/tmp/ or temporarily change to development mode.
Since it looks like recursive queries aren't possible in MySQL, I am wondering if there is a solution to get the same information that also limits the number of queries I make to the database. In my case I have what amounts to a tree and given a node, I make a path back to the root and save the name of the nodes as I go. Given a table like this:
id | parent
-------------
1 |
2 | 1
3 | 1
4 | 2
5 | 2
6 | 5
I want to select all ids on the path from 6 back to 1 (6,5,2,1). Since the total length of the path is unknown I would assume that the only way to do this is taking the results from one query and build a new query until I am back at the root. Then again it has been a couple years since I last used MySQL so it wouldn't surprise me if I am a little out of touch. Any help would be appreciated.
Since it looks like recursive queries aren't possible in mySQL
mySQL doesn't support the 'CONNECT BY' operator, true - but you can implement recursive procedures/functions using mysql and return result sets from them.
I have this table:
id track_name datetime weight
1 aName 2010-06-01 09:00:00 1
2 theName3 2010-07-01 11:00:00 2
3 heyThere 2010-08-01 16:00:00 3
4 abcd 2010-08-01 22:44:00 4
5 g123go 2010-08-01 22:00:50 5
6 foobar 2010-09-01 13:11:00 6
7 barfoo 2010-11-01 12:00:55 7
8 barbar 2010-12-01 11:11:00 8
The weight determines the row record order. It is used for ordering a playlist. And the user can move items up and down, thus reordering in the simple fashion, in which works great.
Now I wonder if there is possible to write a single query that can change the 'weight' value based on the 'date' column, ordering by either DESC or ASC. The same for the 'track_name' column.
Example pseudo query:
UPDATE table SET weight (start from 1) ORDER BY datetime ASC
My alternative is to fetch all rows and process each and everyone of them on the web server, which I doubt is the most effecient way, if there are thousands of records.
I don't think you can do it with a single query. You need a counter, this means you need a loop. If you want to do it with MySQL only, you can create a stored procedure. If not, just write a PHP script (witch might will be a bit slower). Logic is the same:
Get all the data from the table;
Loop through every record in the
order you need and update weight
parameter.
You could use a temporary table with an auto incremented id and select insert into it using your order.
I have a table which has data from a graph.
for example
index value
0 3
1 5
2 7
3 6
4 8
5 9
6 12
7 11
8 10
9 14
10 13
I need to a query that returns the results where the value is at a local maximum, i.e. the value at a particular index is greater than the value at index+1 and index-1.
So for the example set, it should return the list of indexes: 2, 6, 9 corresponding to values 7, 12, 14.
I'm using PHP with SQLite.
I can do it with a foreach-loop in php, but was wondering if there's an easy way to do it using just SQL commands.
Any input would be appreciated.
Or use sub-queries (this is tested):
select ind
from tmp1 t1
where val > (select val from jdoyle.tmp1 t2 where t2.ind = t1.ind-1)
and val > (select val from jdoyle.tmp1 t2 where t2.ind = t1.ind+1);
Doing this with a single loop in PHP is likely to be much faster than shoehorning this into an SQL query, but if you really want to you could self-join the table with itself with something like:
SELECT b.index
FROM points AS a, points AS b, points AS c
WHERE a.index = b.index-1 AND c.index = b.index+1
AND a.value < b.value AND c.value < b.value
(Untested, so *cross fingers*.)
You could write your loop inside a stored procedure in SQL if your SQL database supports stored procedures. However, I don't think sqlite has rich enough stored procedures to do something like this, you would need to use mysql, postgresql or similar.
Well, if you're using sqlite on production I imagine that you don't have a huge bunch of data. Considering this, the best solution really is to solve it at php level.