Tuning mysql, mysqltuner warnings - php

I'm running a Magento website with 100 visitor/hour and have 64GB RAM with an Intel Core i7 CPU. When I run mysqltuner, I receive the message "MySQL's maximum memory usage is dangerously high".
I need help with tuning MySQL to fit my needs, and to clear the warning.
Here is my.cnf file :
key_buffer = 32M
max_allowed_packet = 16M
thread_stack = 256K
thread_cache_size = 100
table_open_cache = 4000
table_definition_cache = 8192
tmp_table_size = 512M
max_heap_table_size = 512M
innodb_buffer_pool_instances = 24
innodb_flush_log_at_trx_commit = 0
innodb_read_io_threads = 64
innodb_write_io_threads = 64
innodb_io_capacity = 50000
innodb_flush_method = O_DIRECT
innodb_file_per_table
innodb_additional_mem_pool_size = 16M
transaction-isolation = READ-COMMITTED
innodb_support_xa = 0
innodb_commit_concurrency = 8
innodb_old_blocks_time = 1000
innodb_lock_wait_timeout = 120
innodb_buffer_pool_size = 40G
thread_concurrency = 10
max_connections = 64
innodb_thread_concurrency = 18
thread_cache_size = 64
table_cache = 2000
query_cache_type = 1
query_cache_size = 128M
query_cache_limit = 3M
key_buffer_size = 8G
sort_buffer_size = 64M
read_buffer_size = 64M
read_rnd_buffer_size = 64M
join_buffer_size = 128M
What can I do to make the mysqltuner warning disappear?

Suggestions to consider for your my.cnf-ini [mysqld] section
Lead with # the following or REMOVE them will allow MySQL defaults
to be used and go a long way toward eliminating the message.
sort_buffer_size = 64M
read_buffer_size = 64M
read_rnd_buffer_size = 64M
join_buffer_size = 128M
You will find response time is much better.
Please post complete TEXT of MySQL Tuner report.
Additional suggestions Jul 14, 2018
innodb_buffer_pool_instances=8 # from 24 minimize overhead
innodb_lru_scan_depth=128 # from 1024 to conserve CPU every second
For additional assistance, check my profile, clk Network Profile for contact info. Thanks

Related

How to optimize SQL settings for fully cached wordpress site

I have a site with mysql 5.5, I have installed wordpress and a cache plugin that generates html pages. The site has about 50k visitors per day, and sometimes my site "crashes" (backend only or sometimes also frontend) so I need to stop sql server, and reboot it. (this crashes can be 1 in months or more than 1 in a week, they happen randomly)
(when I reboot the sql server, the site still fully working because of the html cache + "cloudflare always on") but I want to avoid this crashes. This is my sql config (ovh)
ps: my site dont have WP users, only admins, so cache is always on for every visitor
[mysqld]
tmp_table_size=400M
query_cache_size=1M
skip-external-locking
key_buffer_size = 12M
max_allowed_packet = 1M
table_cache = 4
table_open_cache = 96
sort_buffer_size = 64K
read_buffer_size = 256K
read_rnd_buffer_size = 256K
net_buffer_length = 2K
thread_stack = 128K
thread_cache_size = 4
max_heap_table_size = 600M
max_binlog_cache_size = 1M
max_join_size = 1M
max_seeks_for_key = 2M
max_write_lock_count = 512K
myisam_max_sort_file_size = 1M
########################
##Configuration Innodb##
##Uncomment the next line to disable Innodb
skip-innodb
default-storage-engine=myisam
innodb_buffer_pool_size = 16M
innodb_additional_mem_pool_size = 2M
innodb_log_file_size = 10M
innodb_log_buffer_size = 4M
innodb_flush_log_at_trx_commit=1
How I can optimize this configuration?
Or maybe I need to upgrade my offer?
actually I have this: https://www.ovh.co.uk/web-hosting/performance-web-hosting.xml web performance 1 + private sql with 128mb ram + cloudflare cdn/cache
also, the plugin used for cache is this: https://wordpress.org/plugins/wp-fastest-cache/faq/ and I run "optimize/repair" query very often
I think you should increase the log file size to > 2BG
I config for a Magento site with > 20k products and 3k users/days with values my.cnf bellow:
innodb_buffer_pool_size = 8G
innodb_change_buffering=all
innodb_log_buffer_size=16M
innodb_additional_mem_pool_size=20M
innodb_log_file_size = 1536M
innodb_autoextend_increment=512
thread_concurrency = 3
thread_cache_size = 32
table_cache = 1024
query_cache_size = 512M
query_cache_limit = 512M
join_buffer_size = 256M
tmp_table_size = 512M
key_buffer = 256M
max_heap_table_size = 512M
read_buffer_size = 512M
read_rnd_buffer_size = 512M
bulk_insert_buffer_size = 128M
The important values are
innodb_log_file_size
innodb_buffer_pool_size
Hope that make sense.
At 50k visitors a day, you are receiving 0.5787037037 visitors a second. If that's the case, I suggest either upgrading your server, to a SQL database that has more than just 128MB of RAM. If that's not an option, caching is your last resort.
Note though, no matter how hard you push caching, if your server is weak or poor to begin with, caching is still very limited.

Mysql do not release memory

I have a project that uses 5 different data bases(oracle, mssql, ibm Informix , etc, ... and ...mysql)
Every timeout cron imports data into MySQL. Unfortunatelly we have one very huge cron script, that imports nearly 400 k rows. I made it to import data by small packs, but still, when cron runs this script, MySQL takes nearly 2 GB og memory, and wait for next call. Than takes 2 GB more, and this repeats before MySQL crashes.
We have already faced to the problem that MySQL innodb was corrupted and ran recovery mode from 1 to 4 to restart data base several times.
Total server virt memory is 12 GB.
Our configuration of /etc/my.cnf.d/ server.cnf looks like this:
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql/
#
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
default_storage_engine=innodb
character-set-server=utf8
back_log = 50
max_connections = 100
max_connect_errors = 10
table_open_cache = 2048
max_allowed_packet = 64M
binlog_cache_size = 16M
max_heap_table_size = 1G
read_buffer_size = 8M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
thread_cache_size = 32
# You should try [number of CPUs]*(2..4) for thread_concurrency
thread_concurrency = 8
query_cache_size = 64M
query_cache_limit = 1M
tmp_table_size = 1G
thread_stack = 240K
transaction_isolation = REPEATABLE-READ
ft_min_word_len = 4
log-bin=mysql-bin
binlog_format=mixed
#slow_query_log
#long_query_time = 10
group_concat_max_len = 10000
#*** MyISAM Specific options
key_buffer_size = 16M
bulk_insert_buffer_size = 64M
myisam_sort_buffer_size = 64M
myisam_max_sort_file_size = 10G
myisam_repair_threads = 1
myisam_recover
# *** INNODB Specific options ***
innodb_file_per_table
innodb_buffer_pool_size=8G
#innodb_data_file_path = ibdata1:10M:autoextend
innodb_write_io_threads = 8
innodb_read_io_threads = 8
innodb_thread_concurrency = 16
innodb_flush_log_at_trx_commit = 1
innodb_log_buffer_size = 8M
innodb_log_file_size = 512M
innodb_log_files_in_group = 3
innodb_max_dirty_pages_pct = 90
innodb_lock_wait_timeout = 120
expire_logs_days=5
[mysqldump]
# Do not buffer the whole result set in memory before writing it to
# file. Required for dumping very large tables
quick
max_allowed_packet = 256M
[mysql]
no-auto-rehash
# Only allow UPDATEs and DELETEs that use keys.
safe-updates
[myisamchk]
key_buffer_size = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M
[mysqlhotcopy]
interactive-timeout
[mysqld_safe]
# Increase the amount of open files allowed per process. Warning: Make
# sure you have set the global system limit high enough! The high value
# is required for a large number of opened tables
open-files-limit = 8192
If any info is necessary to find out why this happens I will provide it.
Please! help me to find out the solution ! Should I reinstall MySQL, or it is still possible to resolve by config modifications???

MySQL Timeout in XAMPP

I have XAMPP installed on Microsoft Windows Server 2003. I have a PHP application in xampp/htdocs directory. I have started the Apache and MySQL instances.
I have noticed that every morning, my application gives a MySQL Error when I try to log in to my application. This is fixed after restarting MySQL instance in XAMPP control panel on the server. I think this may be due to timeout. How do I prevent timeout?
This is the my.ini:
[client]
port = 3306
socket = "D:/xampp/mysql/mysql.sock"
[mysqld]
port= 3306
socket = "D:/xampp/mysql/mysql.sock"
basedir = "D:/xampp/mysql"
tmpdir = "D:/xampp/tmp"
datadir = "D:/xampp/mysql/data"
pid_file = "mysql.pid"
key_buffer = 16M
max_allowed_packet = 1M
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
log_error = "mysql_error.log"
plugin_dir = "D:/xampp/mysql/lib/plugin/"
skip-federated
server-id = 1
innodb_data_home_dir = "D:/xampp/mysql/data"
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = "D:/xampp/mysql/data"
innodb_buffer_pool_size = 16M
innodb_additional_mem_pool_size = 2M
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
This is the error I come across every morning before restarting MySQL on XAMPP:
Warning: mysql_pconnect(): MySQL server has gone away in D:\MyApplication\mysql.inc on line 432
Warning: mysql_pconnect(): Link to server lost, unable to reconnect in D:\MyApplication\mysql.inc on line 432
Fatal Error: Failed to connect to database

Remote connect to mysql server [PHP]

I manage a file sharing website (like mediafire,hotFile, etc.) and there is a problem when people try to download from one of the servers.
It goes like this: I have one main server - there I have the mysql database and the website himself, and I have more servers to host the files.
The download process goes like this: user get a link to the outside server. This page connect remotely to the main server and make querys with the database. After the querys, the download get started.
Now, all of the downloads works fine, except from one of the servers. The weirdest thing is that sometimes the download from this server is works, and sometimes not!
It's about 70% of times the download from the problematic server is works, and 30% of times the download isn't works.
When the download isn't working the error message is:
Connect failed: Lost connection to MySQL server at 'reading authorization packet', system error: 0
The my.cnf looks like this:
[mysqld]
skip-name-resolve
bulk_insert_buffer_size = 8M
concurrent_insert = 2
connect_timeout = 10
default-storage-engine = MyISAM
innodb_buffer_pool_size=16M
interactive_timeout = 35
join_buffer_size = 2M
key_buffer_size = 192M
local-infile=0
log-error=/var/log/mysql/error.log
log-slow-queries
log-slow-queries=/var/log/mysql/mysql-slow.log
long_query_time=1
max_allowed_packet = 32M
max_connections = 3000
max_heap_table_size = 256M
max_user_connections= 400
max_write_lock_count = 8
myisam_max_sort_file_size = 256M
myisam_sort_buffer_size = 64M
open_files_limit=128K
query_alloc_block_size = 65536
query_cache_limit = 16M
query_cache_size = 128M
query_cache_type = 1
query_prealloc_size = 262144
range_alloc_block_size = 4096
read_buffer_size = 2M
read_rnd_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 48K
thread_cache_size = 512
tmp_table_size = 256M
transaction_alloc_block_size = 4096
transaction_prealloc_size = 4096
wait_timeout = 100
max_connect_errors = 5000
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
How can I fix the problem?
Thank you very much, and sorry for my English.

How to set memory limit in my.cnf file [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
How to set memory limit in my.cnf file. I tried with memory_limit = 64M . But its showing error while restarting MYSQL server. Any one please help...
my.cnf
[mysqld]
datadir=/home/mysql/
tmpdir=/home/mysqltmp
#max_connections = 175 #was 175
max_connections = 80
#max_connect_errors = 350 #was 250
max_connect_errors = 250
safe-show-database
skip-locking
key_buffer = 1024M # was 128M
max_allowed_packet = 6M
myisam_sort_buffer_size = 64M
#old settings, for 900 ish max maxconn
#sort_buffer_size = 32M
#read_buffer_size = 32M
#read_rnd_buffer_size = 32M
sort_buffer_size = 5M
read_buffer_size = 5M
read_rnd_buffer_size = 5M
query_cache_size= 1024M
query_cache_limit= 16M
max_heap_table_size = 128M
tmp_table_size = 128M
thread_concurrency = 16
wait_timeout = 10
innodb_file_per_table
innodb_log_file_size = 10485760
open_files_limit = 8192
low_priority_updates = 1
#log_slow_queries = /var/log/mysql_slow.log
#log_queries_not_using_indexes = 1
#slow_queries_log_file = /var/log/mysql_slow.log
memory_limit = 64M
# who set these? these are NOT memory settings, but rather integer settings.
#table_cache = 1024M
#thread_cache_size = 8M
table_cache = 512
thread_cache_size = 8
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
There is no such variables like memory_limit in MySQL my.cnf file. You can add variables only from MySQL server system variables. Read this How Mysql uses memory
It depends on RAM size of your MySQL server. You can configure your my.cnf file accordingly based on following basic formula for MySQL memory requirement calculation:
key_buffer_size + (read_buffer_size + sort_buffer_size) * max_connections = K bytes of memory
you may need to configure these basic parameters.
Sample variables from my.cnf file:
#MyISAM
key_buffer_size = 8G
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 2M
myisam_sort_buffer_size = 2M
join_buffer_size = 2M
#Innodb
innodb_buffer_pool_size = 16G
innodb_additional_mem_pool_size = 2G
innodb_log_file_size = 1G
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 30
innodb_file_format=barracuda
There is no memory_limit limit setting in MySQL. The only way you can manage or limit the memory usage of MySQL is to lower the settings for cache, buffer and pool sizes(the exact settingnames depend on what storage engine you are using.
Some settings that apply for the MyISAM engine (the default engine) is:
table_cache=1024
record_buffer=1M
sort_buffer_size=2M
read_buffer_size=2M
read_rnd_buffer_size=2M
myisam_sort_buffer_size=64M
thread_cache_size=128
query_cache_limit=1M
query_cache_size=64M
query_cache_type=1
For InnoDB you have:
innodb_buffer_pool_size = 256M
innodb_additional_mem_pool_size = 20M
innodb_log_file_size = 64M
innodb_log_buffer_size = 8M
If you are looking to change the memory limit in PHP that line(memory_limit = 64M) should be in the php.ini file, not my.cnf which is the configuration file for MySQL.

Categories