I have a stored procedure that is outputting my results in XML format; all is good with the output.
I need to however include the row count it is outputting if possible along with it.
SELECT A.[id],
A.[petName],
A.[petCaption],
B.[petType],
C.[FirstName] as ownerFirstName,
C.[LastName] as ownerLastName,
A.[imageName],
(
SELECT CONVERT(varchar(20),sum(transactionAmount), 1) as totalRaised
FROM petContestTransactionsDonations
WHERE submissionID = A.[id]
FOR XML PATH ('transactionDetails'), TYPE, ELEMENTS
)
FROM petContestSubmissions as A
LEFT OUTER JOIN petContestTypes as B
ON A.[petType] = B.[id]
LEFT OUTER JOIN EmpTable as C
ON A.[empID] = C.EmpID
LEFT OUTER JOIN petContestTransactionsEntries as E
ON E.[submissionID] = A.[id]
WHERE E.[transactionStatus] = 'completed'
ORDER BY A.[id]
OFFSET #offset ROWS
FETCH NEXT #rows ROWS ONLY
FOR XML PATH ('submission'), TYPE, ELEMENTS, ROOT ('root');
I am passing offset and rows to the stored procedure as I am using it with pagination. It is getting records for that page limiting to X rows.
Even though I say get me the next 10 records, it may only have 8 left. That's what I need to return; the total records it found in the select statement.
Is it best to do that in the stored procedure or in my php that is looping over the records?
If you want the row count on each row, then include:
count(*) over () as RowCount
in the outermost select clause.
If you want to read all the data into the application first, then do it in the application layer.
EDIT:
If you want the number of rows returned by the query, then you might as well do it in the application layer. You can also do:
select t.*, count(*) over () as RowCOunt
from (<your query here>) t;
Maybe this will meet your needs. Not in the same query, but you can try
SELECT ##ROWCOUNT AS RowsAffected
immediately after your query, you should get what you are looking for. ##ROWCOUNT is per session, so as long as you don't end your connection, should be ok.
Related
So I have the following query, which I use it to get some analytics stats.
SELECT count(*) as total,CONCAT(YEAR(created),'-',MONTH(created),'-',DAY(created))
as date_only FROM logs where action = 'banner view'
and created BETWEEN '2015-07-03 21:03'
AND '2017-08-02 21:03' group by date_only order by created asc
This works, and it gives me this:
So what I actually need is, the total count of the rows in this case is 20, this is a dummy example, but I need to use this count to check before showing the stats if the data is too big to be displayed on a graphic.
Can this be achieved?
//LE
So the process will be like this:
1. Get a count of the total rows, if the count of rows is smaller than X(number will be in config and it will be a basic if statement), then go ahread and run the above query.
More info:
I actually use this query to display the stats, I just need to adapt it in order to show the total count rows
So the result of thquery should be
total | 20 in this case
I think you would want to use a derived table. Just wrap your original query in parenthesis after the FROM and then give the derived table an alias (in this case tmp). Like so:
SELECT count(*) FROM (
SELECT count(*) as total,CONCAT(YEAR(created),'-',MONTH(created),'-',DAY(created))
as date_only FROM logs where action = 'banner view'
and created BETWEEN '2015-07-03 21:03'
AND '2017-08-02 21:03' group by date_only order by created asc
) as tmp;
If I understand what you want to do correctly, this should work. It should return the actual number of results from your original query.
What's happening is that the results of the parenthesized query are getting used as a sort of virtual table to query against. The parenthesized query returns 20 rows, so the "virtual" table has 20 rows. The outer count(*) just counts how many rows there are in that virtual table.
Based on the PHP tag, I assume you are using PHP to send the queries to MySQL. If so, you can use mysqli_num_rows to get the answer.
If your query result is in $result then:
$total = mysqli_num_rows($result);
Slightly different syntax for Object Oriented style instead of procedural style.
The best part is you don't need an extra query. You perform the original query and get mysqli_num_rows as an extra without running another query. So you can figure out pagination or font size or whatever and then display without doing the query again.
This is an small query but works fine, and give me the total number of rows, you just need add your conditions.
SELECT COUNT(*) FROM table WHERE field LIKE '%condition%'
The group by I think you need to eliminated, becouse, this instead of count the records, divide in all your group by, example: records = 4, with group by you have
1
1
1
1
I hope this help you
You can try this way .
SELECT COUNT(*) FROM ( SELECT count(*) as total,CONCAT(YEAR(created),'-',MONTH(created),'-',DAY(created))
as date_only FROM logs where action = 'banner view'
and created BETWEEN '2015-07-03 21:03'
AND '2017-08-02 21:03' group by date_only HAVING total >=20 ) temp
I have a MySQL table with over 16 million rows and there is no primary key. Whenever I try to add one, my connection crashes. I have tried adding one as an auto increment in PHPMyAdmin and in shell but the connection is always lost after about 10 minutes.
What I would like to do is loop through the table's rows in PHP so I can limit the number of results and with each returned row add an auto-incremented ID number. Since the number of impacted rows would be reduced by reducing the load on the MySQL query, I won't lose my connection.
I want to do something like
SELECT * FROM MYTABLE LIMIT 1000001, 2000000;
Then, in the loop, update the current row
UPDATE (current row) SET ID='$i++'
How do I do this?
Note: the original data was given to me as a txt file. I don't know if there are duplicates but I cannot eliminate any rows. Also, no rows will be added. This table is going to be used only for querying purposes. When I have added indexes, however, there were no problems.
I suspect you are trying to use phpmyadmin to add the index. As handy as it is, it is a PHP script and is limited to the same resources as any PHP script on your server, typically 30-60 seconds run time, and a limited amount of ram.
Suggest you get the mysql query you need to add the index, then use SSH to shell in, and use command line MySQL to add your indexes.
If you don't have duplicate rows then the following way might shed some light:
Suppose you want to update the auto incremented value for first 10000 rows.
UPDATE
MYTABLE
INNER JOIN
(SELECT
*,
#rn := #rn + 1 AS row_number
FROM MYTABLE,(SELECT #rn := 0) var
ORDER BY SOME_OF_YOUR_FIELD
LIMIT 0,10000 ) t
ON t.field1 = MYTABLE.field1 AND t.field2 = MYTABLE.field2 AND .... t.fieldN = MYTABLE.fieldN
SET MYTABLE.ID = t.row_number;
For next 10000 rows just need to change two things:
(SELECT #rn := 10000) var
LIMIT 10000,10000
Repeat..
Note: ORDER BY SOME_OF_YOUR_FIELD is important otherwise you would get results in random order. Better create a function which might take limit,offset as parameter and do this job. Since you need to repeat the process.
Explanation:
The idea is to create a temporary table(t) having N number of rows and assigning a unique row number to each of the row. Later make an inner join between your main table MYTABLE and this temporary table t ON matching all the fields and then update the ID field of the corresponding row(in MYTABLE) with the incremented value(in this case row_number).
Another IDEA:
You may use multithreading in PHP to do this job.
Create N threads.
Assign each thread a non overlapping region (1 to 10000, 10001 to
20000 etc) like the above query.
Caution: The query will get slower in higher offset.
I would like to count the number of rows in a statement returned by a query. The only solutions I found were:
sqlsrv_num_rows() This one seems a bit too complicated for such a simple task like this and I read that using this slows down the execution quite a bit
Executing a query with SELECT COUNT This method seems unnecessary, also it slows down the execution and if you already have a statement why bother with another query.
Counting the rows while generating a table As I have to generate a html table from the statemnt I could put a variable in the table generating loop and increment it by one, but this one only works when you already have to loop through the entire statement.
Am I missing some fundamental function and/or knowledge or is there no simpler way?
Any help or guidance is appreciated.
EDIT: The statement returned is only a small portion of the original table so it wouldn't be practical to execute another query for this purpose.
In sql server table rows information is stored in the catalog views and Dynamic Management Views you can use it to find the count
This method will only work for the physical tables. So you can store the records in one temp table and drop it later
SELECT Sum(p.rows)
FROM sys.partitions AS p
INNER JOIN sys.tables AS t
ON p.[object_id] = t.[object_id]
INNER JOIN sys.schemas AS s
ON t.[schema_id] = s.[schema_id]
WHERE p.index_id IN ( 0, 1 ) -- heap or clustered index
AND t.NAME = N'tablename'
AND s.NAME = N'dbo';
For more info check this article
If you don't want to execute another query then use select ##rowcount after the query. It will get the count of rows returned by previous select query
select * from query_you_want_to_find_count
select ##rowcount
Is it ok to a mysql query inside a while loop using the ID of each row passed to fetch results from another table? OR is there a better way to do it?
$q = $__FROG_CONN__->query("SELECT cms_page.id, cms_page.title, cms_page.slug, cms_page_part.* FROM cms_page LEFT JOIN cms_page_part ON cms_page_part.page_id=cms_page.id WHERE cms_page.parent_id='8'");
$r = $q->fetchAll(PDO::FETCH_ASSOC);
echo '<ul id="project-list">';
foreach ($r as $row) {
echo '<li>';
echo '<img src="<img src="phpThumb/phpThumb.php?src=public/images/'.$row[0].'/th.jpg&w=162" alt="" />';
echo '<div class="p-text">';
echo '<h4>'.$row["location"].'<span>'.$row["project_date"].'</span></h4>';
echo '<p>'.$row["body"].'</p>';
echo '</div>';
echo '</li>';
}
echo '</ul>';
I am trying to pull the project_date, body and location fields from another table where the sql query matches. The title and slug are held in another table. There should only be a maximum of eight or so results but im getting alot more.
The suggestions using IN are fine, but if you are getting the ids from another query, it might be better to combine these two queries into one query using a join.
Instead of:
SELECT id FROM users WHERE age <30
SELECT id, x FROM userinfo WHERE userid IN ($id1, $id2, ..., $idn)
do:
SELECT users.id, userinfo.x
FROM users
LEFT JOIN userinfo ON userinfo.userid = users.id
WHERE age < 30
To reduce the overhead of preforming a query, you may want to look at getting all the data in a single query. In which case you may want to take a look at IN(), e.g.
SELECT * WHERE x IN (1, 2);
There is also BETWEEN()
SELECT * WHERE x BETWEEN 1 AND 2;
See the mysql docs for more information
I would try to build the query in a way where I only need to pass it once. Something like WHERE ID=1 OR ID=2 OR ... Passing multiple queries and returning multiple recordsets is expensive in terms of processing and network traffic.
This will be very inefficient, what you want is to join the tables on the ID
SELECT * FROM table1 LEFT JOIN table2 ON (table1.ID = table2.ID) WHERE condition
Mysql join documentation
This will return one set of rows with all the information you need, returned from both tables.
In a small application / small result set, this might be okay, but it results in a lot of (small) calls to the database.
If you can find an alternative way (perhaps see Yacoby's suggestion?) in which you can do one call to the database, this is probably better.
EDIT
If you are only interested in the IDs from one table, in order to get the correct results out of another table, perhaps a JOIN is what you are looking for?
SELECT t1.fieldA, t1.fieldB
FROM table1 t1
JOIN table2 t2 ON t1.ID = t2.ID
WHERE t2.someField = 'someValue'
Is it ok to a mysql query inside a while loop using the ID of each row passed to fetch results from another table? OR is there a better way to do it?
You should reformulate your query in SQL. Say, put the ids into a memory table and use it in a JOIN:
SELECT *
FROM idtable
JOIN mytable
ON mytable.id = idtable.id
This way, MySQL will make the loops for you but will make them in (usually) more efficient way.
SQL is a language designed to work with sets.
So if you have a set of ids and a table (which is a set of tuples), your first thought should be "how do I apply the set-based operations to these sets using SQL"?
Of course it is possible to run a bunch of simple queries in a loop but this way you just do extra work which SQL engine developers most probably have already done for you (and usually have done it in more efficient way).
You may want to read this article in my blog:
Double-thinking in SQL
If you had data in table 1 that you need to use to return data in table 2 for each row returned in table 1. What is more efficient to use a set of querys in PHP one inbedded in the while loop of the other or an SQL function within a query?
for example:
$qry = mysql_query(SELECT date FROM table1)
while($res = mysql_fetch_array($qry))
{
$qry = mysql_query("SELECT name FROM table2 WHERE date=$res['date']")
}
or to do this as a function that returns the Id from table1 within the query.
A (LEFT / RIGHT) JOIN?
Unless I've misunderstood the question...
I think you're looking for JOIN sql syntax. If you have 2 tables: messages and author and you want to return messages with authors. Then you can write following SQL statement:
SELECT m.body, a.name FROM message m
LEFT JOIN author a ON (a.id=m.author_id)
This will return message body with corresponding author name
Table author:
id - primary key
name - name of the author
Table message:
body - text of the message
author_id - id of the author
UPD1:
This will be faster then looping each message and select an author. Because JOIN allows you to return all data in single query (not N x queries when using for loop).
UPD2:
With your tables the query will look like:
SELECT t1.date, t2.name FROM table1 t1 LEFT JOIN table2 t2 ON (t2.date=t1.date)
It depends on if the data is easier to find during the while loop or in the query itself.
So if the DB has to base the sub-query on the result of each row in the main query, and there are 1000 total rows and 100 results in the main query, it has to check all of the rows 100 times, so that's 100,000 sub-queries it runs.
So think it terms of the number of results of the main query. If your while loop has to query the DB 100 times while the DB can do it much faster and efficiently in one query, that's better. But if you want a subset of answers that you can say 'query only based on the last set of results' the while loop is better.
What is more efficient to use
a set of querys in PHP one inbedded in the while loop of the other
or
an SQL function within a query
Seems you answered your question yourself, didn't you?
Every query you send to the dbms has to be sent over the network, parsed, analyzed then executed. You may want to minimize the number of queries sent to the db.
There may be exceptions, for example if the processing of the data requires operations which the dbms is not capable of.