I have an prestashop online, and since 1 month, we have some too long response times.
MySQL: 5.5
All tables(Engine): MyIsam
PHP: 7.0 * / 5.6
I import some csv file with 20 000 lines and when i'am in dev env. it takes 40 secondes and in production [ Tie well ] 10 minutes.
We have removed the material cause, cache , ans charge cause. -> [MySQL has treat 5 billion of requests in 45 seconds -> prod machine].
Actually the export sql file is 1.3 Go -> in prod.
We use Prestashop Db class to do the request.
If you have any suggestion !!
Thanks !!
Since you're using ISAM you may disable indexes before insert.
See MySQL disable & enable keys
Related
I recently upgraded my cloud instance from Ubuntu 14 to 16. It's a Laravel application, there's a page which reads about 200000 entries from a table with 3 columns and writes them to a CSV file. The query involves 2 where clauses- status=0 and CHAR_LENGTH()=7.This is the query-
select code from code where status=0 and CHAR_LENGTH(code)=7 limit 200000;
There are more than 30 million entries in that code table.
The page had been working before upgrading to Ubuntu 16, but now the page gives Error 500 after being stuck on the page for about 1-2 minutes.
Note that the page works if I try to read lower number of entries e.g. 20 entries and the file also gets written.
Can anyone please help me with this it's a critical operation for our website.
I just figured out the problem, after the upgrade the PHP5 folder was deleted and PHP 7 was installed. Previously in php5/fpm/php.ini file the max_execution_time and memory_limit was set to 600 and 2GB respectively. But after the upgrade in php/7/fpm/php.ini file the default values for max_execution_time and memory_limit was set to 60 and 300MB respectively.
I changed the values to 600 and 2GB and now the system is working perfectly.
I have a report that generates an array of data from MySQL server by using looping through PHP code (Laravel framework). However, the maximum that the server can handle is an array with 400 row, and each row contains 61 child value in it.
[
[1, ...,61], // row 1
.
.
.
[1,....,61] // row 400
]
Each value is calculated from running a loop that retrieves data from MySQL server.
There is a no load balancer.
I tried to increase max_execution_time = 600 (10 minutes), but it still show the connection time out problem. Any thoughts? Thanks,
Connection Timed Out
Description: Connection Timed Out
Server version: Apache/2.4.7 (Ubuntu) - PHP 5.6
Would need more info for a definitive answer...
What is the Apache/httpd version (there have been some bugs that relate to this)?
Is there a firewall or load balancer in the mix?
If you are sure it is still a timeout error and not say memory, then it is probably httpd's TimeOut directive. It defaults to 300 seconds.
If still stuck paste the exact error you are seeing.
My PHP version was 5.6. After upgrading to PHP7, my application speed was increased significantly. Everything works fine now.
First of all let me say there are no CLEAR tutorials for how to do this on windows, I'm a programmer and have been using MySQL and PHP for years, and have never seen so many different answers when it comes to linux, and absolutely nothing for windows users.
Anyway I have been working for 2 weeks trying to solve this myself, and I am at the same place no matter what tutorial, method or whatever I try. Basically I have two MySQL Servers. One Master, and One Slave (setup through phpMyAdmin)
I have both servers online and they can see each other and are connected to each other, however when it comes to syncing under the Slave Status table in phpMyAdmin I am constantly getting errors saying tables don't exist. But they do. Now I'm not sure if I am supposed to completely import the databases from my master manually, and then it updates it or whatever (again no clear tutorials are out there for windows). Below is the table i copied from phpMyAdmin.
Slave_IO_State Waiting for master to send event
Master_Host XX.XX.XX.XX
Master_User XXXXXXXX
Master_Port 3306
Connect_Retry 60
Master_Log_File mysql-bin.000012
Read_Master_Log_Pos 1221
Relay_Log_File *servername*-relay-bin.000003
Relay_Log_Pos 253
Relay_Master_Log_File mysql-bin.000001
Slave_IO_Running Yes
Slave_SQL_Running No
Replicate_Do_DB
Replicate_Ignore_DB
Replicate_Do_Table
Replicate_Ignore_Table
Replicate_Wild_Do_Table
Replicate_Wild_Ignore_Table
Last_Errno 1146
Last_Error Error 'Table '*databasename*.*tablename*' doesn't exist' on query. Default database: '*databasename*'. Query: 'DELETE FROM *tablename* WHERE expires
Skip_Counter 0
Exec_Master_Log_Pos 107
Relay_Log_Space 211493
Until_Condition None
Until_Log_File
Until_Log_Pos 0
Master_SSL_Allowed No
Master_SSL_CA_File
Master_SSL_CA_Path
Master_SSL_Cert
Master_SSL_Cipher
Master_SSL_Key
Seconds_Behind_Master (blank but would like this setup)
I should also add that this database that supposedly has missing tables is working perfectly on the master for an active website.
Any help to point me in the right direction would be appreciated.
You want to take a full backup of the master database and load it onto your slave or manually copy them over while the master isn't doing work. When you run your CHANGE MASTER command on the slave before starting it you want to specify the first binary log file that was created after your backup was taken like this:
CHANGE MASTER TO MASTER_HOST='xx.xx.xx.xx', MASTER_PORT=3306, MASTER_USER='user', MASTER_PASSWORD='*secret*', MASTER_LOG_FILE='mysql-bin.004388', MASTER_LOG_POS = 0
The Seconds_Behind_Master will automatically show once the slave doesn't have an error.
That is all there is to it.
I need some help with oracle settings to reproduce some issues we're having and to clarify, I'm not a oracle expert at all - no experience with it.
I've managed to installed oracle-xe(because it's easiest & smallest) and got our software running on it.
Now, reports say, that the connection in some some production setups time out(long running scripts/programs) and are not reconnected (not even throwing exceptions).
So that's what I'm trying to reproduce.
After some browsing around on the internet I found that running these queries should limit my connection & idle time to 1 minute:
ALTER PROFILE DEFAULT LIMIT IDLE_TIME 1;
ALTER PROFILE DEFAULT LIMIT CONNECT_TIME 1;
ALTER SYSTEM SET RESOURCE_LIMIT = TRUE;
Results in:
SQL> select * from user_resource_limits where resource_name in ('IDLE_TIME','CONNECT_TIME');
RESOURCE_NAME LIMIT
-------------------------------- ----------------------------------------
IDLE_TIME 1
CONNECT_TIME 1
After that I made a simple php script 'test.php', it runs a query - sleeps, and runs a new query.
require_once('our software');
$account1 = findAccount('email1#example.com');
sleep(100);
$account2 = findAccount('email2#example.com');
Isn't this supposed to time out?
Some extra details about what software I'm running:
Centos 5
Oracle XE
php 5.3.5
using oci8(not pdo)
I've been using the same DB abstraction library for years. But today it started writing these Notice (8) messages in my log.
The application is working correctly but every time a script connects to the DB the same notice is logged.
I cannot think what might have changed. This is happening on my local dev machine.
OS X 10.6.2
PHP 5.3.0 (cli)
mysql Ver 14.12 Distrib 5.0.87
mysqlnd 5.0.5-dev - 081106 - $Revision: 1.3.2.27 $
If someone is struggling with this issue, here is the fix:
Try changing/setting up wait_timeout in your mysql my.cnf config file:
wait_timeout=3600
This config file is located in the /etc/mysql/my.cnf (Ubuntu/Debian) and /usr/local/mysql/my.cnf (OSX).
Restart mysql server and it should work.
Only solution I've found so far is, changing
// From
PDO::ATTR_PERSISTENT => true
// To
PDO::ATTR_PERSISTENT => false
Not so happy with this, but works in the meantime.
I'm using a very old PC for personal project, so I'm guessing the problem could have to do with lack of resources.
I am using PHP 5.6.20, PDO (throwing exceptions only), and MySQL 5.6.28 with persistent connections and EVERYTHING is utf8mb4. My entire stack is set up for utf-8 (dsn string settings, connections, database server databases, tables, columns, Apache 2.4.12, PHP, all webpages, CSS ... you name it).
I get the following error message intermittently and it is mystifying and annoying.
Notice: PDO::__construct(): send of 5 bytes failed with errno=32 Broken pipe in file /foo/bar/baz
Assuming a persistent connection is a noninteractive one, the MySQL 5.6 manual (5.1.4 Server System Variables) says the following about the server system variable wait_timeout.
The number of seconds the server waits for activity on a
noninteractive connection before closing it.
Default: 28800 sec
(28000 sec / 1) * (1 hour / 3600 sec) = 8 hours
Max: 31536000 sec
((31536000 sec / 1) * (1 hr / 3600 sec) * (1 day / 24 hrs) = 365 days
Therefore, check wait_timeout in your my.cnf and decide if persistent connections are what you need. Also, you'll have to invest in making your application more robust to account for a persistent connection that has been torn down. Clearly, you do not want your client to come back the next day (having gone home for the night) and say "What the heck?!"
It may be because your data contains 'utf-8' characters. I had the similar issue is caused by it.
Exception: mysql_query(): send of 1462592 bytes failed with errno=32 Broken pipe
I used
mysql -u username -p database < dump_file # this is bad
to import the sql file contains lot of UTF8 characters (Thai language), but I didn't set default-character-set=utf8 for [mysql]. So the wrong coded data in the database caused that issue.
just remove mysqlnd driver and use mysqli
Yes mysqlnd more modern, but what about stability?
next commands fix your problem
apt-get remove php5-mysqlnd
apt-get install php5-pdo-mysql