Hello I have a mysql database and all I want is basically to get a value on the second table from a first table query
I have figured something like this but is not working.
select src, dst_number, state, duration
from cdrs, area_code_infos
where SUBSTRING(cdrs.src,2,3) = area_code_infos.`npa`;
Please help me figure out this. I have tried in PHP to have multiple queries running one after the other but when I loaded the page after 45 minutes of wait time I gave up.
Thanks,
I assume the tables are farily big, and you are also doing an unindexed query.. basically substring has to be calculated for every row.
Whenever you do a join, you want to make sure both of the joined fields are indexed.
An option would be to create another column containing the substring calculation and then creating an index on that.
However, a better option would be to have an areaCodeInfosID column and set it as a foreign key to the area_code_infos table
Related
Let's say I have dynamic numbers with unique id's to them.
I'd like to insert them into database. But if I already have that certain ID (UNIQUE) I need to add to the value that already exists.
I've already tried using "ON KEY UPDATE" ,but it's not really working out. And selecting the old data so we could add to it and then updating it ,is not efficient.
Is there any query that could do that?
Incrementing your value in your application does not guarantee you'll always have accurate results in your database because of concurrency issues. For instance, if two web requests need to increment the number with the same ID, depending on when the computer switches the processes on the CPU, you could have the requests overwriting each other.
Instead do an update similar to:
UPDATE `table` SET `number` = `number` + 1 WHERE `ID` = YOUR_ID
Check the return value from the statement. An update should return the number of rows affected, so if the value is 1, you can move on happy to know that you were as efficient as possible. On the other hand, if your return value is 0, then you'll have to run a subsequent insert statement to add your new ID/Value.
This is also the safest way to ensure concurrency.
Hope this helps and good luck!
Did something different. Instead of updating the old values ,I'm inserting new data and leaving old one ,but using certain uniques so I wouldn't have duplicates. And now to display that data I use a simple select query with sum property and then grouping it by an id. Works great ,just don't know if it's the most efficient way of doing it.
I have a data set with more than 10000 (this will be more in future) records as below:
[[name=>'name1',url=>'url1', visit=>120],
[name=>'name2',url=>'url2'], visit=>250,
..........
]
It is possible to have duplicate values for the key combination name,url. In such situations I need to get the sum of each records have the duplicate name,url.
Finally I want insert this values into a database. When I do this I have two method to do this:
Create another array with unique combination (name,url) and sum of visit
Update/insert db for each record in a loop.
What is the optimal solution to do this or is there better way to do this?
I know there will be memory issues for a large data set in the first method. In second method there are many db hits and I need to know the disadvantage(s) if I follow 2nd way.
Any help or insight would be appreciated.
I do some big database update like this myself and spent ages trying different solutions.
Instead of:
Check if record exists, eg select count(id) from data where
name='name' and url='url'
Not found, insert record
Found, sum result
I would try this
Set the unique primary keys on your data table on the url and name field.
Try to do a normal insert and see if you get a successful result.
On unsuccessful result (there already is value for name and url because these 2 fields must be unique), sum the result.
I use while $1<=N loop to get database row from table TTT and run certain operation with it, rinse and repeat. I use $i to retrieve the next consecutive id row from database on each loop go.
I used to run SELECT query to find the duplicates in table TTT, but now with current size of TTT it is too slow, so I decided to go with Unique index.
Everything works fine and no more duplicates, however there are now gaps in my id values because I use INSERT IGNORE, which breaks my script depending on id's to be consecutive.
So how do I adjust my code to get the same functionality with unique index?
I was thinking to create temp table with ordered ids, but that was bad idea.I was trying to get next id in previous loop go with (id > $i), but it would only work for first gap...
Ok so for my situation where only one script inserts into TTT, my solution is to enable
innodb_autoinc_lock_mode=0
to get the desired functionality back.
More info on different ways to handle the issue - http://dev.mysql.com/doc/refman/5.1/en/innodb-auto-increment-handling.html
There is a large table that holds millions of records. phpMyAdmin reports 1.2G size for the table.
There is a calculation that needs to be done for every row. The calculation is not simple (cannot be put in set col= calc format), it uses a stored function to get the values, so currently we have for each row a single update.
This is extremely slow and we want to optimize it.
Stored function:
https://gist.github.com/a9c2f9275644409dd19d
And this is called by this method for every row:
https://gist.github.com/82adfd97b9e5797feea6
This is performed on a off live server, and usually it is updated once per week.
What options we have here.
Why not setup a separate table to hold the computed values to take the load off your current table. It can have two columns: primary key for each row in your main table and a column for the computed value.
Then your process can be:
a) Truncate computedValues table - This is faster than trying to identify new rows
b) Compute the values and insert into the computed values table
c) So when ever you need your computed values you join to the computedValues table using a primary key join which is fast, and in case you need more computations well you just add new columns.
d) You can also update the main table using the computed values if you have to
Well, the problem doesn't seem to be the UPDATE query because no calculations are performed in the query itself. As it seems the calculations are performed first and then the UPDATE query is run. So the UPDATE should be quick enough.
When you say "this is extremely slow", I assume you are not referring to the UPDATE query but the complete process. Here are some quick thoughts:
As you said there are millions of records, updating those many entries is always time consuming. And if there are many columns and indexes defined on the table, it will add to the overhead.
I see that there are many REPLACE INTO queries in the function getNumberOfPeople(). These might as well be a reason for the slow process. Have you checked how efficient are these REPLACE INTO queries? Can you try removing them and then see if it has any impact on the UPDATE process.
There are a couple of SELECT queries too in getNumberOfPeople(). Check if they might be impacting the process and if so, try optimizing them.
In procedure updateGPCD(), you may try replacing SELECT COUNT(*) INTO _has_breakdown with SELECT COUNT(1) INTO _has_breakdown. In the same query, the WHERE condition is reading _ACCOUNT but this will fail when _ACCOUNT = 0, no?
On another suggestion, if it is the UPDATE that you think is slow because of reason 1, it might make sense to move the updating column gpcd outside usage_bill to another table. The only other column in the table should be the unique ID from usage_bill.
Hope the above make sense.
I have some issues with sugarcrm.
As you know that sugarcrm table, they do have ID (which is a unique string), they not run by sequential. e.g
4bab37e4-798a-e01c-75de-4e4397f358b7
For example, I would like to copy the table sugarcrm.accounts to something.accounts, in something.accounts I added some custom file on it for another PHP process usage.
Now the problem is, my sugarcrm table got huge records there, I plan to run it batch by batch, each time I would like to copy 10,000 records to my somthing.accounts.
However, sugarcrm.accounts, their ID, not in sequential, how do I know offset parameter?
I do not want to amend sugarcrm table / or add a temporary table in sugarcrm. (e.g sugarcrm.account_index), it might caused me having problem to do the upgrade.
So anyone have any idea, how do I get the index number? Is MySQL got hidden index?
Or anyone have better idea to do the database table copy another database table?
One way is the following:
- Select all rows from sugarcrm.accounts and order by date_created ascending.
- Use limit to only select a subset of the rows (store the offset from batch to batch)
- Copy the subset of rows to something.accounts
If new records are added later they will still be copied, since they will be last in the set. However, if you delete records in sugarcrm.accounts while running the batch jobs, then you need to change the offset as well, since you might omit some rows.
Another way, if the two databases/tables are in the same MySQL instance, is to join the two tables, and select the next 10.000 which doesn't exist in something.accounts.