I have a machine that is only able to connect to a 2008 SQL server sporadically. I created a monitoring script that I have automated to attempt a connection (via sqlsrv driver and php) every 15 mins. This script has ran fine for the last two weeks but recently it has been reporting a SQL server going down. On extensive investigation I am able to connect to it from every other machine consistently except for the machine automating the monitoring script. This script also connects to two other SQL servers (both are sql server 2005) without any problems. Here is what I have tried to resolve the failures:
1. Reset both machines (serveral times)
2. Attempted connections via several different logins countless times (including sa).
3. Deleted and recreated logins and attempted to connect.
4. Turned off windows firewall (SQL server is running on win2k8)
This is the error I get when I cant connect: http://pastebin.com/9d80mAyV
Typically it wont work for about 5 attempts (spaced 15 mins apart) then it will work for 1 attempt. Then it fails for another 5 attempts. There is no absolute consistency.
My thoughts is that it has blacklisted my client from connecting too often. Is this possible? How could I resolve this problem? Any other thoughts or suggestions?
A few more details needed:
What are you using to connect? is it an application or management studio (what version?)
Where are the servers located? i.e. are you on the same network?
You can try to increase your connection timeout. If you are connecting from management studio, then you can do this in the server registration properties.
Alternatively, you can setup sql monitoring on the server itself, and it will email you when it goes down.
Related
I'm using mysqli connection over ssl,
so using in code following:
$mdbconnection = mysqli_init();
$mdbconnection->ssl_set('/ssl/xyz.pem', '/ssl/abc-cert.pem', NULL, NULL, 'AES256-SHA');
$mdbconnection->real_connect(SV_SERVER, SV_MYSQL_LOGIN, SV_MYSQL_PASSWORD, SV_MYSQL_DATABASE);
I noted down that I am not able to connect over mysql server all the time.
if trying 100 times failing 3 time.
and getting following error
echo($mdbconnection->connect_errno);
printing:
2003
and if I am trying second time after failing it{echo($mdbconnection->connect_errno);} will give always 1045.
I am trying to solve this bug.
Any ideas would be appreciated.
The 2003 error code means the connection couldn't be established:
https://dev.mysql.com/doc/refman/5.5/en/error-messages-client.html
Error code 1045 means the client connected but the server rejected the credentials (access denied):
https://dev.mysql.com/doc/refman/5.5/en/error-messages-server.html
I'd say that if you're getting the 2003 error code a few times out of every 100 tries, it sounds like some of the possibilities could be:
Some intermediate application or piece of hardware might be seeing your attempts as a possible flood attack (DDoS) and could be intercepting and blocking the connection (if you're connecting 100 times really quickly)
You might have some network stability issues on the client side. It might be worth the time to run Wireshark and reproduce the problem with it running in order to capture the problem and see if it shows anything significant (e.g. DNS failure).
The MySQL server might be crashing and then restarting. Check your server-side MySQL error log (https://dev.mysql.com/doc/refman/5.7/en/error-log.html). Actually, you should really do this as a first step any time you have bizarre behavior in MySQL.
If the MySQL server log doesn't show anything, check your system logs, especially if you're logging anything like blocked firewall (iptables or whatever f/w you're using) packets and search them for your client IP around the time of the problem.
Finally, if you can reproduce this easily/quickly (e.g. make 100 connections via a script and see it fail), try eliminating SSL out of the equation and connecting without SSL and see if it gives you the same problem. If it works flawlessly without SSL, perhaps there's a problem with the SSL library that was used during the compilation of your MySQL package.
You might also want to check to see if you have any load-balancing in place that might make your script connect to different instances of the MySQL server (this is less likely, but could explain the symptoms you're seeing if one server is down and the next doesn't have the credentials you're providing)
I have setup a local server on a regular desktop (not a server desktop) and have 3-4 client machines accessing the local web application I developed from the server via a WIFI router (server is connected to router via cable. All clients via WIFI).
When two of the clients are connected to the application all is well, but when a third (or more) machine joins in there are periods where each machine does not get any service from the server (the application webpage remains loading until I manually reset Apache on the server via services). At times the server responds when one of the clients refresh their page but most of the time I have to perform a reset of the Apache server.
This occurs roughly once an hour on average (6 hours of continuous usage) as the clients are using the application.
Server is running Windows 7 and Apache v2.4 with PHP v5.4
Server and all client machines are running AVG internet security
Firewall is handled by AVG Internet Security
Is this issue due to the code in my application, desktop not being able to manage requests like a server machine, antivirus or a mix of the three?
If so, how can I set-up the server to reset automatically?
Thanks
UPDATE
It is a application where users write reports on files after reviewing information
-Frequent sql requests for file data
-No images
-Some pages contain multiple sql queries that represent the page content
-Network has no internet connection
-Code does not make requests for external information from the internet
-All client machines run the application on Google Chrome web browser
But it rarely happens but sometimes the amount of connection is restricted by the third-party interface being used by the application. We are unable to figure out the reason unless having more details like what content of your app, and the error code apache or HTTP returning.
This kind of situations is difficult to track, especially on Windows where diagnostic tools aren't as readily available as on other platforms.
I suppose you can try and check the antivirus by either running server and clients with no antivirus at all for some hours, or disabling and re-enabling the antivirus when the hangup occurs.
Apart from that, you would need to pinpoint where the error occurs:
in the connection stage (Windows OS is the problem)
in the response stage (Apache is the problem - try fiddling with the child spawning parameters)
in the management stage (PHP is the problem - you can probably check this by changing the setup from PHP-as-a-module, and PHP-as-CGI-application)
in the response stage (that is, connection to the SQL server). You can check this by setting up some pages that use different combinations of session, database, and output buffering and see whether those pages remain reachable even when the application is hung up.
For an example of the last, if a page such as
<?php print date("H:i:s"); phpinfo(); ?>
remains reachable and correctly refreshes (that's why the date() command) even when the app does not respond, this demonstrates that Windows, Apache and PHP are "innocent", and either the SQL server has issues, or you do not interface with it correctly. It might be for example be the case (though unlikely in this instance) that the resident PHP module is accumulating connections to the SQL server and not releasing them, so that after a while you need to stop Apache (thereby freeing the module) and restart.
If this were the case, even if it's not a "real" fix, you can set up Apache so that all children die and are replaced after a small number of requests (once it was 150, but when leaks all but disappeared, I believe that the default became 0 -- Apache children no longer die. Check it out, I might well misremember).
UPDATE
So I completely disabled my server's firewall and it appears to be the culprit. I had tried to disable certain rules before but disabling the whole thing worked. Very frustrating, but problem solved I guess... I think the key indicator that it was the firewall was because it happened at exactly midnight when my server likes to apply updates and such.
This is pretty strange to me, I have a server downstairs that hosts my websites and MySQL servers and it has been running for years without many issues. I have 2 routers bridged together behind my modem and my server is behind one of them. All other devices connect via WiFi. All of the proper ports are open on the router and I have users configured in MySQL that haven't changed and have been working fine this whole time.
So last night I was working on a project and I decided to sync everything with a backup on my SkyDrive. I have a scheduled backup for MySQL that runs at midnight (daily) and it just turned over to midnight so I decided to open my network and watch the file get populated before I sent a copy to my SkyDrive. After the backup was complete (which it did successfully), I was going to continue to work on my program but all of a sudden I can no longer connect via my local network to the MySQL server. I'm using PHP and my connection string never changed and all other MySQL admin tools don't connect. The live site works fine, so MySQL was definitely running and working but no remote connections were being accepted. Why is this happening all of a sudden?
Things I've tried :
I did notice that my logs were packed full of BINLOG errors so I turned off the binlog since I recently turned it on (a couple weeks ago).
Restarting MySQL
Turning off Windows Server 2008 firewall (temporarily)
Connecting from a different device (mobile phone, tablet), no luck
Temporarily allowing port 3306 on my router
Checked server logs for intrusion attempts, none present...
Setup :
PHP 5.4 on local machine and server
Windows Server 2008 Enterprise (on server...)
MySQL Version 5.5.25a
Does anyone have any clue as to what's going on here? I'm going to reboot my server when the load is low and see if this helps any, I will update this once it comes back online.
So I completely disabled my server's firewall and it appears to be the culprit. I had tried to disable certain rules before but disabling the whole thing worked. Very frustrating, but problem solved I guess... I think the key indicator that it was the firewall was because it happened at exactly midnight when my server likes to apply updates and such.
(Not a Linux guru!) At work we have for the past month had a serious issue that we haven't been able to solve. From a server running Ubuntu-Linux we are using PHP to connect to a server running Microsoft SQL Server 2005. This has been working without problems for a long time. At the start of February 2013 we moved the SQL Server machine into the same core network as the Linux-machine, which involved changing IP-address on the Windows-server running SQL Server 2005.
After this operation we've had issues connecting to the SQL Server from this particular Linux-machine. Maybe one out of 1000 connections fails with the following error message:
PHP Warning: mssql_connect(): Unable to connect to server
I'm not aware of any other changes to either of these two servers. We've tried to look at 'everything'.
FreeTDS
php.ini adjustments
IP-address in SQL Server Configuration Manager
Unlimited number of concurrent connections in SQL Server
Network and firewall issues (No packet loss during 24 hours)
Any suggestions to how we can pursue error searching for this problem? Any more info anyone would like to know about this setup? It's kinda serious for us since many of our cronjobs fail to execute normally due to this random connection problem.
Rgds,
Sven David, Norway
Very difficult to diagnose from an external perspective, however, it feels like a networking issue rather than php/sql config. Presumably, because it's intermittent, it's not mssql functions/extension. Very little is truly random!
Perhaps approach this by writing a script that monitors the servers (both hosts) vital stats - like a heartbeat: IPs, DNS, DHCP?, requests etc and setting it to record anomalies/alert you when any element changes on either host or connect requests fail...
Probably worth posting on other SE sites too - not just Stackoverflow.
If MySQL is dropping connections in a PHP application, and MySQL connection limit is set above the number of concurrent users in the application, which other factors can contribute to this behavior? Also, analyzing moodle logs (the only app running at these servers), yesterday I had 4 times more activity and it didn't drop anything, but today there were times where it was frustrating the number of dropped connections.
My main question here is why the database is rejecting connections when it didn't before with 4 times more activity (and everything it's the same, I didn't change anything in between).
Some background: I've got 2 servers contracted at my hosting:
shared server running Debian Linux/PHP 5.3 (FastCGI)
VPS running Debian Linux/MySQL 5.1.39
On this environment I'm running only moodle 1.9.12 (for the database connection using adoDB and persistent connections), php part on the shared server, database on the VPS. I suspect that, by PHP running on a shared server, other hosting accounts are affecting me (the database rejecting connections I mean, I really don't care about RAM/CPU).
Reading about the issue I've seen in places that persistent connections don't work well with PHP as CGI/FastCGI and that if both servers are in the local lan it really doesn't matter using persistent or not persistent connections because of the connection is going to be quick anyway. So now I'm using not persistent connections. I guess that may be part of the problem, but I fail to understand why it worked with more load. Which PHP/Apache settings are involved here?
Since your database and web server are on two different machines, there are two other possible causes: the network in between, and the network layer of the operating system.
If you did not change anything in your configuration and it worked with higher loads before, it is more likely to be an issue with the network connectivity between the two machines. Since the database machine is a VPS you also don't know how much real network load it is handling. If your ISP has competent support personnel (which unfortunately isn't always the case) it can't hurt to ask them if they have an explanation.
The same goes for your "shared" web-server. While it is unlikely, it is not impossible that it is a an issue of too many connections on that machine.
It would also help to know how exactly you are measuring dropped connections. If you are looking at the aborted-connections counter of mySQL, it is not necessarily a measure of an actual problem: http://dev.mysql.com/doc/refman/5.1/en/communication-errors.html. A user aborting a page load already may increase this counter.
If PHP throws an error, because it could not connect to the server or lost connection to the server during a query, then it is an issue.