Sybase Adaptive Server IQ can't SELECT *, always limited to 30? - php

I've this problem with a Sybase IQ database (version SELECT ##version shows Adaptive Server IQ/12.5.0/0306) using the PHP SQL Anywhere extension.
I can't select all rows, i.e. SELECT * from anytable always returns 30 rows.
The only workaround I've found is using SELECT TOP 1000 * from anytable (maximum is 32767) but there are certain situations where I need all rows.
Any help is much appreciated.
EDIT: example script (much like the documentation)
$conn = sasql_connect("HOST=host:port;DBN=dbn;UID=uid;PWD=pwd");
if (!$conn) { echo "Connection failed."; die(); }
$result = sasql_query($conn, "SELECT * FROM dba.anytable" );
sasql_result_all($result); // display 30 rows in a formatted table
sasql_free_result($result);
sasql_disconnect($conn);
EDIT: specs of both machines where I'm experiencing the same exact problem:
Development machine:
Windows 7 x64
PHP 5.4.23 TS x86
SAP SQL Anywhere PHP Module for PHP 5.4
SAP SQL Anywhere Database Client 12.0.1.4134
Production sever:
CentOS 6.6 x64
PHP 5.5.23 NTS (x86_64 as per YUM)
SAP SQL Anywhere PHP Module for PHP 5.5
SAP SQL Anywhere Database Client 12.0.1.4127

Probably a stupid question, but just to be sure.
Did you check in Client --> Tools -> Sybase IQ -> Maximum number of rows to display.
(sry for posting this as an answer, but i dont have enough rep to ask you in comments.) Cheers

Well, there is probably some configurable property somewhere that will release you from the evil clutches of 30. I do not know where that property is. I hope someone finds it.
In case nobody does, however, here is a big hack which orders by the primary key of the given table, retrieves as many rows as it can given your constraints, and keeps track of the last fetched primary key in order to retrieve the next batch of rows. It would be better to adapt this to use START AT or LIMIT / OFFSET if available, but I'm assuming they're not. If one of those is available, you could use this approach for any table. If not, this approach can be adapted to any table which has a unique non-null key.
$conn = sasql_connect("HOST=host:port;DBN=dbn;UID=uid;PWD=pwd");
if (!$conn) {echo "Connection failed."; die(); }
$highest_id = -1;
$num_rows_retrieved = 0;
do {
if (!sasql_real_query($conn, "SELECT TOP 32767 * FROM dba.anytable where anytable_id > $highest_id order by anytable_id")) {
echo "Query failed.";
die();
}
$result = sasql_use_result($conn);
if (!$result) {
echo "No result set.";
die();
}
$num_rows_retrieved = 0;
$num_fields = sasql_num_fields($result);
while ($row = sasql_fetch_row($result)) {
$highest_id = $row[0]; // assumes anytable_id is the first field
$i = 0;
while ($i < $num_fields) {
echo "$row[$i]\t";
$i++;
}
$num_rows_retrieved++;
echo "\n";
}
sasql_free_result($result);
} while ($num_rows_retrieved == 32767);
sasql_disconnect($conn);

Related

MySql 8.0 Read Speed

I'm in need of some expertise here, I have a massive SQL database that I use in conjunction with my Mobile app I program. I'm getting some very long times to fetch a result from the database, at times upwards of 20 to 25 seconds. I've managed to increase the speed and it is were it is now from 40 seconds to retrieve a result. I am hoping someone may have some insight on how I can speed up the query speed and return a result faster then 20 seconds.
Main table is 4 columns + 1 for the "id" column, the database contains 15,254,543 rows of data. Currently it is setup as an InnoDB, with 4 indexes, and it about 1.3GB for size.
My server is a GoDaddy VPS, 1 CPU, with 4 GB of Ram. The is dedicated and I do not share resources with anyone else, its only purpose beside a very basic website is the SQL database.
Just to note, the database record count is not going to get any larger, I really just need to figure out a better way to return a query faster then 20 seconds.
In more detail, the Android app connects to my website via a php document to query and return the results, I have a thought that there maybe a better way to go about this and this maybe were the pitfall is. An interesting note is that when I'm in PHP My Admin, I can do a search and get a result back in under 3 seconds, which also points me to the issue might be in my php document. Here is the php document below that I wrote to do the work.
<?php
require "conn.php";
$FSC = $_POST["FSC"];
$PART_NUMBER = $_POST["NIIN"];
$mysql_qry_1 = "select * from MyTable where PART_NUMBER like '$PART_NUMBER';";
$result_1 = mysqli_query($conn ,$mysql_qry_1);
if(mysqli_num_rows($result_1) > 0) {
$row = mysqli_fetch_assoc($result_1);
$PART_NUMBER = $row["PART_NUMBER"];
$FSC = $row["FSC"];
$NIIN = $row["NIIN"];
$ITEM_NAME = $row["ITEM_NAME"];
echo $ITEM_NAME, ",>" .$PART_NUMBER, ",>" .$FSC, "" .$NIIN;
//usage stats
$sql = "INSERT INTO USAGE_STATS (ITEM_NAME, FSC, NIIN, PART_NUMBER)
VALUES ('$ITEM_NAME', '$FSC', '$NIIN', '$PART_NUMBER')";
if ($conn->query($sql) === TRUE) {
$row = mysqli_insert_id($conn);
} else {
//do nothing
}
//
} else {
echo "NO RESULT CHECK TO ENSURE CORRECT PART NUMBER WAS ENTERED ,> | ,>0000000000000";
}
$mysql_qry_2 = "select * from MYTAB where FSC like '$FSC' and NIIN like '$NIIN';";
$result_2 = mysqli_query($conn ,$mysql_qry_2);
if(mysqli_num_rows($result_2) > 0) {
$row = mysqli_fetch_assoc($result_2);
$AD_PART_NUMBER = $row["PART_NUMBER"];
if(mysqli_num_rows($result_2) > 1){
echo ",>";
while($row = mysqli_fetch_assoc($result_2)) {
$AD_PART_NUMBER = $row["PART_NUMBER"];
echo $AD_PART_NUMBER, ", ";
}
} else {
echo ",> | NO ADDITIONAL INFO FOUND | ";
}
} else {
echo ",> | NO ADDITIONAL INFO FOUND | ";
}
mysqli_close($con);
?>
So my question here is how can I improve the read speed with the available resources I have or is there an issue with my current PHP document that is causing the bottle neck here?
Instead of using LIKE you would get much faster reads by selecting a specific column that was indexed.
SELECT * FROM table_name FORCE INDEX (index_list) WHERE condition;
The other thing that speeds up Mysql greatly is the use of an SSD drive on the VPS server. A SSD drive will greatly decrease the amount of time it takes to scan a database that large.

Website Version PHP

I am trying to create a version file that will alert the end user if there is an update.
In theory and practical application:
When the admin logs in and goes to their Management interface, a script reads a remote file and compares the remote version with the local version, if they are the same, report that status, if remote is greater, alert.
The local version is pulled from a MySQL table with MySQLi and confirmed to work, the remote file is read, confirmed to work as well how ever I am running into an issue.
Local version is 1.3, remote version is 1.3 yet it reports:
Your Version - 1.3 is OUT OF DATE, new version 1.3 is available for download
When it should say
Your Version is UP TO DATE!
The code I am using is below
require_once('includes/settings.php');
function check_version() {
global $connection;
$query = "SELECT * from version";
$result = mysqli_query($connection, $query);
// if the query fails, die and report
if(!$result = $connection->query($query)){
die('There was an error running the query [' . $connection->error . ']');
}
else {
$row = mysqli_fetch_assoc($result);
$my_version = $row['minor'];
echo '<p>Local = '.$my_version.'</p>';
$ctx = stream_context_create(array(
'http' => array(
'timeout' => 1
)
));
$rversion = file_get_contents("http://www.twedev.com/projects/microcms/version", 0, $ctx);
echo $rversion;
var_dump($rversion);
if ($my_version === $rversion) {
$dev_msg = '<strong>Your Micro CMS Version - '.$my_version.' is CURRENT</strong>';
}
elseif ($my_version != $rversion) {
$dev_msg = '<strong>Your Micro CMS Version - '.$my_version.' is OUT OF DATE,<br> new version '.$rversion.' is available for download</strong>';
}
else {
$dev_msg = '<strong>Something went wrong with the version server</strong>';
}
}
return $dev_msg;
}
?>
What I need is another set of eyes to look through this is point me in the right direction if possible, this has me overly frustrated today.
You are using the === identical operator to compare values, using that instead of == will compare the value and the type of value.
The value you are getting from file_get_contents() it's a string, and the version number seems to be something different to string, probably a float number.
Update: After using var_dump on both vars we could see the result wasn't the same, and the solution was to use trim to strip whitespaces from the end of the string.

Sphinx search returns different results using command line than php api

Command line search (26019 documents / 26019 hits)
search.exe --config c:\sphinx\sphinx.conf keyword
OR
search.exe --config c:\sphinx\sphinx.conf keyword -e2
PHP API (Total: 1000 / Total found: 51038)
//sphinx command line and php api
mysql_connect("localhost", "username", "password");
mysql_select_db("database");
require_once('sphinxapi.php');
$cl = new SphinxClient;
$cl->setServer("127.0.0.1", 9312); // NOT "localhost" under Windows 7!
$cl->setMatchMode(SPH_MATCH_EXTENDED2);
$cl->SetLimits(0, 20);
$result = $cl->Query("keyword");
if ($result['total'] > 0)
{
echo 'Total: ' . $result['total'] . "<br />\n";
echo 'Total Found: ' . $result['total_found'] . "<br />\n";
echo '<table>';
echo '<tr><td>ID</td><td>Date</td><td>Title</td><td>Content</td></tr>';
foreach ($result['matches'] as $id => $otherStuff)
{
$row = mysql_fetch_array(mysql_query("select * from table where id = $id"));
extract($row);
echo "<tr><td>$id</td><td>$date</td><td>$title</td><td>$content</td></tr>";
}
echo '</table>';
}
else
{
echo 'No results found';
}
Match modes:
SPH_MATCH_ALL / SPH_MATCH_ANY / SPH_MATCH_PHRASE / SPH_MATCH_BOOLEAN / SPH_MATCH_EXTENDED / SPH_MATCH_EXTENDED2
PS: The command line result is the right one because I made the select directly from MySQL and i got it too.
$sql = "SELECT t1.field1, t1.field2, t1.field3, t2.field4
FROM t_table1 AS t1
LEFT JOIN t_table2 AS t2 ON t2.id = t1.t2_id WHERE t2.field4 LIKE 'keyword'";
Forget you found "search.exe". Ignore it. Really. Forget about it. Now. I'll wait.
Right now that over with, try being specific about what index you want to search (on the Query function call)
Also running once query for every row you get, is not good. You should only issue one SQL query for all results. The mysql IN() function makes it easy.
Example http://www.nearby.org.uk/sphinx/search-example5-withcomments.phps
More basic, but still functional example http://www.nearby.org.uk/sphinx/search-example2.phps
Sphinx is limited to returning only 1000 results.
There was some suggestions to raising that limit so check their page.Beware only that you can change that number in older sphinx relesase to whatever you wanted but it would return 1000 as max.
answer here:
http://sphinxsearch.com/docs/manual-2.0.5.html#conf-max-matches
unless you are planing to use the 26019 matches on the same page I suggest you use a pagination (if you're only displaying the results)

How to Get The Count or The Number Of Rows In A Result Set In PHP using ODBC Connection?

While I build a web page in My PHP web application, My Connection works ok but When I want to get the count of rows of the SELECT Statement I used in my query, It gives me -1 !! although my result set has about 10 rows.
I would like to get the actual number of result set rows.
I searched the PHP Manual & documentation but I do not find a direct way like a Count function or something like that.
I wonder if I have to make a Count(*) SQL Statement in another query and attach it to my Connection to get the Count of Rows ?
Does any one knows an easy and direct way to get that ?
the odbc_num_rows function always gives -1 in result so I can not get the actual number of rows.
My Programming langauge is PHP and My Database Engine is Sybase and The Way to connect to Database is ODBC.
Here are you the Code I used:-
<?PHP
//PHP Code to connect to a certain database using ODBC and getting information from it
//Determining The Database Connection Parameters
$database = 'DatabaseName';
$username = 'UserName';
$password = 'Password';
//Opening the Connection
$conn = odbc_connect($database,$username,$password);
//Checking The Connection
if (!$conn)
{
exit("Connection Failed: " . $conn);
}
//Preparing The Query
$sql = "SELECT * FROM Table1 WHERE Field1='$v_Field1'";
//Executing The Query
$rs = odbc_exec($conn,$sql);
//Checking The Result Set
if (!$rs)
{
exit("Error in SQL");
}
echo "<p align='Center'><h1>The Results</h1></p>";
while ( odbc_fetch_row($rs) )
{
$field1 = odbc_result($rs,1);
$field2 = odbc_result($rs,2);
$field3 = odbc_result($rs,3);
echo "field1 : " . $field1 ;
echo "field2 : " . $field2 ;
echo "field3 : " . $field3 ;
}
$RowNumber = odbc_num_rows($rs);
echo "The Number of Selected Rows = " . $RowsNumber ;
//Closing The Connection
odbc_close($conn);
?>
Thanks for your Help :)
odbc_num_rows seems to be reliable for INSERT, UPDATE, and DELETE queries only.
The manual says:
Using odbc_num_rows() to determine the number of rows available after a SELECT will return -1 with many drivers.
one way around this behaviour is to do a COUNT(*) in SQL instead. See here for an example.
in php.net:
The easy way to count the rows in an odbc resultset where the driver returns -1 is to let SQL do the work:
<?php
$conn = odbc_connect("dsn", "", "");
$rs = odbc_exec($conn, "SELECT Count(*) AS counter FROM tablename WHERE fieldname='" . $value . "'");
$arr = odbc_fetch_array($rs);
echo $arr['counter'];
?>
On what basis, do you expect odbc_num_rows to return anything other than -1 ?
We have the fact from the manuals, that OBDC does not support ##ROWCOUNT / odbc_num_rows. So there is no basis for expecting that it "should" return anything other than that which is documented, -1 in all circumstances.
Even if you used Sybase directly (instead of via ODBC), you would have the same "problem".
odbc_num_rows returns ##ROWCOUNT, which is the rows inserted/updated/deleted by the immediately preceding command. -1 is the correct, documented value, if the immediately preceding command is not insert/update/delete.
It has nothing to do with rows in a table.
Use another batch, and either one of the documented methods to obtain rows in a table, and load the value into a variable:
SELECT #Count = COUNT(*) -- slow, requires I/O
or
SELECT #Count = ROW_COUNT(db_id, object_id) -- very fast, no I/O
Then interrogate the result array, to obtain the variable, not odbc_num_rows, which will continue returning -1.

Is mysql_num_rows efficient and/or standard practice?

A while ago I was poking around with SQLite, trying to port some of my sites to use it instead of MySQL. I got hung up on the lack of a function to count results, like PHP's mysql_num_rows(). After searching a little I discovered this mail list, which says (as I understand it) that SQLite doesn't have that functionality because it's inefficient. It states that it is bad form to write code that needs to know how many rows are returned.
I generally use mysql_num_rows to check for empty return results. For example:
$query = "SELECT * FROM table WHERE thing = 'whatever'";
$results = mysql_query($query);
if (mysql_num_rows($results)) {
while ($row = mysql_fetch_array($results)) {
echo "<p>$row[whatever]</p>";
}
} else {
echo "<p>No results found</p>";
}
The vehement distaste for the concept of mysql_num_rows() in the SQLite community makes me wonder if it's that terribly efficient for regular MySQL in PHP.
Is there a better, more accepted way for checking the size of a MySQL result set in PHP besides mysql_num_rows()?
EDIT:
I'm not just using mysql_num_rows to get the count--I would use a COUNT query for that. I'm using it to check if there are any results before outputting everything. This is useful for something like displaying search results - it's not always guaranteed that there will be results. In SQLite world, I have to send one COUNT query, check if there is something, and then send a SELECT query to get everything.
You already have something that is telling you if you've got results in mysql_fetch_array(). It returns false if there are no more rows to fetch (from php.net).
$query = "SELECT * FROM table WHERE thing = 'whatever'";
$results = mysql_query($query);
if($results) {
$row = mysql_fetch_array($results);
if($row) {
do {
echo "<p>{$row[whatever]}</p>";
} while($row = mysql_fetch_array($results));
} else {
echo "<p>No results found</p>";
}
} else {
echo "<p>There was an error executing this query.</p>";
}
Regardless of whether or not you actually use what you SELECTed, all of the rows are still returned. This is terribly inefficient because you're just throwing away the results, but you're still making your database do all of the work for you. If all you're doing is counting, you're doing all that processing for nothing. Your solution is to simply use COUNT(*). Just swap COUNT(*) in where you would have your SELECT statement and you're good to go.
However, this mostly applies to people using it as a complete substitute for COUNT. In your case, the usage isn't really bad at all. You will just have to manually count them in your loop (this is the preferred solution for SQLite users).
The reason being is in the underlying SQLite API. It doesn't return the whole result set at once, so it has no way of knowing how many results there are.
As explained on the mailing list you found. It is inefficient to return the count of rows because you need to allocate a lot of memory to hold the entire (remaining) result set. What you could do, is to simply use a boolean to test if you have output anything.
$query = "SELECT * FROM table WHERE thing = 'whatever'";
$results = mysql_query($query);
$empty_result = true;
while ($row = mysql_fetch_array($results)) {
echo "<p>$row[whatever]</p>";
$empty_result = false;
}
if ($empty_result) {
echo "<p>No results found</p>";
}

Categories