The title of question may sound weird but with my english could not get better title.
A have created chat aplication in my website. Now I want to add notifications.
When submitting a new message I am checking if other user has read previous message in given conversation. If he has then I write a new notification. If he has not seen previous then I do not write a new notification.
I use mysql count() function to count fields and then do the php logic. In CI it looks like this:
public function ifUnreadMsgs($con_id, $sender_id)
{
$this->db->where('conversation_id', $con_id);
$this->db->where('sender_id', $sender_id);
$this->db->where('seen IS NULL', null, false);
$this->db->from('messages');
$count = $this->db->count_all_results();
if($count > 0){
return true;
}else{
return false;
}
}
My question is about optimization. I do know that with time I will have a lot of messages. Lets say I have 1 000 000 messages stored in database. I also know that the one with possible "NULL" in seen will be with msg_id of approx. 999 995. I have to use this query often and user waits for ajax response so I want to reduce the time for query as much as possible.
Is it possible to run query backward and stop as I hit the value I was looking for? I thought about using DISTICT or LIMIT keyword for stopping but how to run it backwards?
EDIT:
Actually I need to start looping through messages table starting from last row, stop at "conversation_id" and look if "seen" is NULL or not.
you can use ORDER BY for both ascending & descending order. by default ORDER BY starts from the first, but you can start it by adding DESC. Please check this & that
Related
I accessed my table of database by a PHP script and I get continuous repeat results sometimes.
I ran this query:
$query ="SELECT Question,Answer1,Answer2 FROM MyTable ORDER BY RAND(UNIX_TIMESTAMP(NOW())) LIMIT 1";
Before of this query, I tried just with ORDER BY RAND(), but it gave me a lot of continuous repeat results, that's why I decided to use ORDER BY RAND(UNIX_TIMESTAMP(NOW())).
But this last one still give me continuous repeat results( but less).
Im going to write a example to explain what I mean when I said "continuous repeat results" :
Image that I have 100 rows in my table: ROW1,ROW2,ROW3,ROw4,ROW5...
well, when I call my script PHP 5 times continuosly I get 5 results:
-ROW2,ROW20,ROW20,ROW50,ROW66
I don't want same row continuously two times.
I would like it for example: -ROW2,ROW20,ROW50,ROW66,ROW20
I just want to fix it some easy way.
https://dev.mysql.com/doc/refman/5.7/en/mathematical-functions.html#function_rand
RAND() is not meant to be a perfect random generator. It is a fast way
to generate random numbers on demand that is portable between
platforms for the same MySQL version.
If you want 5 results, why not change the limit to 5 ? This will ensure that there are no duplicates
The other option is read all of the data out, and then use shuffle in php ?
http://php.net/manual/en/function.shuffle.php
Or select the max and use a random number generated from PHP
http://php.net/manual/en/function.mt-rand.php
This is not doable by just redefining the query. You need to change the logic of your PHP script.
If you want that the PHP script (and the query) returns exactly ONE row per execution, and you need a guarantee that repeated executions of the PHP scrips yield different rows, then you need to store the previous result somewhere, and use the previous result in the WHERE condition of the query.
So your PHP script becomes something like (pseudocode):
$previousId = ...; // Load the ID of the row fetched by the previous execution
$query = "SELECT Question,Answer1,Answer2
FROM MyTable
WHERE id <> ?
ORDER BY RAND(UNIX_TIMESTAMP(NOW()))
LIMIT 1";
// Execute $query, using the $previousId bound parameter value
$newId = ...; // get the ID of the fetched row.
// Save $newId for the next execution.
You may use all kinds of storages for saving/loading the ID of the fetched rows. The easiest is probably to use a special table with a single row in the same database for this purpose.
Note that you may still get repeated sequential rows if you call your PHP script many times in parallel. Not sure if it matters in your case.
If it does, you may use locks or database transactions to fix this as well.
I'm sorry for stupid question, but I have one of these days, where I feel like the dumbest programmer. I need your help. I'm currently developing with PHP and MYSQL, where I'm like super low skilled and I'm working on inherited project.
I have database table with almost 6k records in it, let's say TABLE_A, and I need to iterate through the records in TABLE A and for every record create two new records in TABLE B where the PK from TABLE_A(Id) is FK in TABLE_B. Nothing special right? So I have one more thing, this is happening, don't blame please, in production DB. So I got a request to run the insertion into table B only for 10 records every 1 second. Furthermore, I have list of Ids which looks like this: 1,2,4,6,7,8,9,11,12,15,16,.. to 6k. So I cannot basically do:
for ($i = 1; $i <= MAX(id); $i++) {
//create two new records in TABLE B
}
I have spent some time with the research and I need to talk about it with you guys, to come up with some ideas. I don't want from you the exact solution, but I want to learn how to think about that and how to come up with the solution. I was thinking about it on my way home. So I just created the algorithm in my head. Here is step-by-step process in my head about what I know and what I will probably use:
I know that I can run just 10 inserts per 1 second - so I need to limit the select from TABLE A for just 5 rows in one batch.
So I can probably use MySQL syntax: LIMIT and OFFSET, for example: select * from t LIMIT 5 OFFSET 0
This means that I have to store the id of the last record from the previous batch.
After finishing current batch, I need to wait for 1 second( I'm think about using PHP method sleep()) before starting new batch.
I need loop
The exact number of rows in TABLE_A is for now unusable
The insertion of new records is simple. Focus on the iteration.
So here is something I have on the paper and I'm not quite sure if it is going to work or not, because I really want to learn something from this problem. I will skip the things around, like connect DB,etc and will focus just on the algorithm and will write in some hybrid PHP/Mysql/Pseudo code.
$limit=5
$offset=0;
function insert($limit, $offset){
$stm = $db->prepare("SELECT id FROM tableA LIMIT :limit OFFSET :offset");
$stm->execute(array('limit' => $limit, 'offset' => $offset));
while($stm->rowCount() > 0){
$data = $stm->fatchAll();
foreach($data as $row){
// insert into TABLE_B
}
sleep(1);
$offset +=5;
$this->insert($limit, $offset);
}
}
I'm not totally sure, if this recursion will work. On paper it looks feasible. But what about performance? It's a problem in this case?
Maybe the main question is: Am I over thinking this? Do you know about better solution how to do that?
Thank you for any comments, thoughts, suggestions, ideas and detail descriptions of your procedure how to come up with feasible solution. Probably I should dig more into some algorithm analysis and design. Do you know any good resources?
(Sorry for grammar mistakes, I'm not a native speaker)
I don't know why you have to insert into table B for 10 records per 1 second, but let's assume that this condition can not be changed.
Your sources code are right, however recursion is not necessary here, we should do something like that.
limit=5
offset=0
while (itemsA = fetch_from_a(limit, offset)) {
# you should do a batch insertion here, see MySQL's documentation.
insert_into_B(itemsA);
sleep(1);
offset += 5;
}
# prototype
# fetch some records from table A, return array of found items
# or an empty array if nothing was found.
function fetch_from_a(limit, offset);
So I have a variable and a recordset:
$firstRecordID = 1;
$records = Recordset::all();
I want to filter the recordset:
$filteredRecords = $records->find(function($record) use ($firstRecordID){
return ($record->id == $firstRecordID);
});
In this example, assuming the record id is a primary key, there will only be one row that will get returned, however, I think the filter will keep on going.
My question is, how do I force the filter to stop after a certain condition is met?
edit: I'll add another more complex example:
$filteredRecords = $records->find(function($record) use ($lastRecordID){
$conditions == $records->conditions;
// in here I would find some value
// based on some other unknown record based on the $conditions
// if the value matches any of the $conditions return the row
// if the value matches a specified stop condition
// (which is defined by the user) stop retrieving rows.
});
I think the solution is to use first($filter)
$filteredRecords = $records->first(function($record) use ($firstRecordID){
return ($record->id == $firstRecordID);
});
http://li3.me/docs/lithium/util/Collection::first
Short answer
Throw an exception inside the filter and catch it on the outside. You will probably have to collect the included items in an array yourself since you won't have access to find's return value.
Long answer
You are focusing too much on a particular tactic (stopping find using a filter) when it might do you some good to step back and consider what you are trying to accomplish.
It honestly sounds like you are trying to eat your cake and have it too: you want to be flexible and specify your search conditions in a function you can easily change, but you also want to be efficient and not pull more data out of the database than you have to.
The thing is, passing a filter function into find is already as inefficient as you can get it, because li3 will have to pull every record from the database before your filter is even called. That's where the inefficiency is. Stopping the filtering process once you hit 10 items or whatever won't make much of a difference (unless your filtering is also very expensive).
I would recommend thinking of the following:
Do you really need to allow any filter method at all? Is it possible to use declarative conditions? If so, you can supply limit options and let the database do all the work.
If you can't, consider using find to fetch a small batch of records and collect the ones you want to keep in an array. Repeat with another small batch until you've checked every item in the database or filled your array.
We currently have some php code that allows an end user to update records in our database by filling out a few fields on a form. The database has a field called sequentialNumber, and the user can enter a starting number and ending number, and an update query runs in the background on all records with a sequentialNumber between the starting and ending numbers. This is a pretty quick query.
We've been running into some problems lately where people are trying to update records that don't exist. Now, this isn't an issue on the database end, but they want to be notified if records don't exist, and if so, which ones don't exist.
The only way I can think of to do this is to run a select query in a loop:
for ($i=$startNum; $i<=$endNum; $i++) {
//run select query: select sequentialNumber from myTable where sequentialNumber = $startNum;
}
The problem is, our shared host has a timeout limit on scripts, and if the sequentialNumber batch is large enough, the script will time out. Is there a better way of checking for the existence of the records before running the update query?
EDIT:
It just occurred to me that I could do another kind of test: get the number of records they're trying to update ($endNum - $startNum), and then do a count query:
select count(sequentialNumber) where sequentialNumber between $startNum and $endNum
If the result of the query is not the same as the value of the subtraction, then we know that all the records aren't there. It wouldn't be able to tell us WHICH ones aren't there, but at least we'd know something wasn't as expected. Is this a valid way to do things?
You could try
select sequentialNumber from myTable where sequentialNumber between $startNum and $endNum
This will return all known numbers in that range. Then you can use the array_search function to find out if a certain number is known or not. This should be faster than doing a lot of queries to the db.
var count = mysql_fetch_array(mysql_query('SELECT count(*) FROM x WHERE id>startNum and id<endNum'));
var modified = mysql_affected_rows(mysql_query('UPDATE x SET col='value' WHERE id>startNum and id<endNum'));
if (count[0] > modified) {
// panic now
}
Ok, I have two tables that I am joining and doing a select on to get three fields back (timestamp, message, and articleNum). Now, this data is basically a log of when an article was created and modified, I have to write the logic to process those three fields I got back. In order to figure out when an article was created and/or modified last, I need to look at the message and look for the keywords ("added" for created and "updated" for modified). I have the results of the query back in an assoc array and I would eventually like to have the end result be in an assoc array with the articleNum being the key and a two value array (created, modified) being the key. Sometimes though, there won't always be a modified value, but there will always be a created. Any idea on how I would even start a problem like this?
EDIT
From what I can tell, it looks like the date is stored as a bigint in Unix seconds. Clarification: the created and modified values are not fields from the table, I need to figure it out from the message field. There will always be one added time but sometimes there could be 0 or more updated messages and I would need to figure out the latest.
EDIT 2
Ok, sorry about the wording of the question. After looking at the problem a little longer I realized I could do this all in two SQL statements. For finding the added date I used:
"SELECT MIN(action_logs.time_added), article.number
FROM action_logs
JOIN proposal ON action_logs.article_num = article.number
WHERE action_logs.message LIKE '%added%'
GROUP BY article.number"
Could probably do the same thing for last modified, except with a MAX. Thanks for the suggestions though.
This is a poorly worded question. It smells a bit of this
What are the two tables? What is the message field? Presumably events?
Why don't you just have 'created' and 'updated' in the articles tables?
Are you sure your design is sound?
I'm making some assumptions here:
If you're table stores a row for each of your messages (added/updated), then this query would return one row per article, with the columns you need (articleNum, Created, Updated):
SELECT
A.articleNum,
MCreated.Timestamp AS Created,
MUpdated.Timestamp Updated
FROM Articles A
JOIN Messages MCreated
ON MCreated.articleNum = A.articleNum
AND MCreated.Message = 'added'
LEFT JOIN Messages MUpdated
ON MUpdated.articleNum = A.articleNum
AND MUpdated.Message = 'updated'
Try something like this:
$articles = array();
while ($row = mysql_fetch_assoc($result)) {
$articles[$row['articleId']] = array(
'created' => $row['created'],
'modified' => $row['modified']
);
}
Not sure I understand the question..
You want to look in message and extract Created Time and Modified Time?
If that's the case, what is the date/time format? It should be very easy to find that information using a regular expression.
Ian
Edit -
If you want to extract them from the message that you get out of the query, then regular expressions are a likely candidate for completing this task.
Post a sample message.