I have a PHP API (using SLIM) that users different external microservices.
My apache is configured to handle a huge amount of simultaneous connections (without my CURLS I can handle 50k simultaneous connections without any effort). So is my microservice (in Python with GUnicorn - 25 Workers and 25 threads each).
However, I am having a issue when CURLing my Python API with my Apache API.
Indeed when I am having high traffic spikes (and therefor a high amount of CURL connections from my PHP to my Python) - my apache starts hanging.
It seems as the CURLs are queued.
My first thought was that my Python script executes too slowly (500ms). But I then noticed that i could manually launch the Python script without any delay even when it was spiking.
This is why I believe the problem comes from the actual CURLs. Is it possible that they are getting queued when I have too many users Curling at the same time or when the CURL takes too long to respond ?
The consequence of it all is that APACHE takes time to respond and slows down. It even happens that APACHE goes into 503.
FYI : The two APIs are on different DOCKER CONTAINERS but on the same server. Both containers have their SOMAXCON at a high number.
If anyone has any idea, please help.
By CURLs or Curling i mean sending CURL requests to my python api.
I will check mod_wsgi out m, thanks!
Edit : sorry, I meant to send it as a comment
Related
We have made the backend of a mobile AP in laravel and mysql. The application is hosted on AWS Ec2 and using RDS mysql database.
We are stress testing the app using jmeter. When we send upto 1000 API requests from jmeter, it seems to work fine. However, when we send more than 1000 (roughly) requests in parallel, The jmeter starts getting internal server error (500) as a response for many requests. the internal 500 error percentage increases as we increase the number of APIs
Normally, we would expect that if we increase the APIs, they should be queued and the response should slow if the server is out of resources. We also monitored the resources on the server and they never reached even 50% of the available resources
Is there any timeout setting or any other possible setting that I could tweak so that the we dont get the internal server error before reaching 80% of the resource usage
Regards
Syed
500 is the externally visible symptom of some sort of failure in the server delivering your API. You should look at the error log of that server to see details of the failure.
If you are using php scripts to deliver the API, your mysql (rds) server may be running out of connections. Here's how that might work.
A php-driven web server under heavy load runs a lot of php instances. Each php instance opens up one or more connections to the mysql server. When there are too many php instances x connections per instance the mysql server starts refusing more of them.
Here's what you need to do: restrict the number of php instances your web server is allowed to use at a time. When you restrict that number, incoming requests will queue up (in the TCP connect queue of your OS's communication stack). Then, when an instance is available to serve each item in the queue it will do so.
Apache has a MaxRequestWorkers parameter, with a default extremely large value of 256. Try setting it much lower, for example to 32, and see whether your problem changes.
If you can shrink the number of request workers, you paradoxically may improve high-load performance. Serializing many requests often generates better throughput than trying to do many of them in parallel.
The same goes for the number of active connections to your MySQL server. It obviously depends on the nature of the queries you use, but generally speaking fewer concurrent queries improves performance. So, you won't solve a real-world problem by adding MySQL connections.
You should be aware that the kind of load imposed by server-hammering tools like jmeter is not representative of real world load. 1000 simultaneous jmeter operations without failure is a very good result. If your load-testing setup is robust and powerful, you will always be able to bring your server system to its knees. So, deciding when to stop is an important part of a load testing plan. If this were my system I would stop at 1000 for now.
For your app to be robust in the field, you probably should program it to respond to 500 status by waiting a random amount of time and trying again.
I am working on a PHP script that:
receives a client request;
processes the request via a CPU-and-Time-Intensive-binary-computation
store the result of the computation into a MySQL database
then respond to the client with a Status 200 OK
Problem: when there are 10s of 1000s of requests coming in per second during peak hours: the clients have to wait for a long time to receive Status 200 OK.
Flexibilities: The script does not need to respond to the client with the result of the computation. The script does not even need to respond Status 200 OK based on the success/failure of the computation - the computation may eventually fail and that's completely okay. So the actual computation could really happen in parallel behind the scene.
What tools / packages / libraries / strategies should be used to achieve this kind of intensive request handling design on PHP? Is it even something on the PHP side or is it solvable from the Apache side?
Notes:
Running Apache, MySQL, PHP, Redis on Ubuntu [AMPRU]
Clients will just send a request and receive a Status 200 OK right away.
Clients will not wait for the computation of the request to complete.
There is no auto-scaling or load-balancing concept in place: it's a single AMPRU server.
Better if multiple computations can happen in parallel behind the scenes
This is classic use case for a queue. Of the tech-stack you have listed, Redis has support for queues (check out PHP-Resque for the library), or there are other tools that can be used, such as Beanstalkd (a favourite of mine, with the Pheanstalk PHP library), or Amazon SQS. There are a number of other options, both self-hosted., or available as services.
The website, or other mechanism receives the data, and queue it - returning the 200 OK. The back-end workers, as simple as a cron-based system, or (better) multiple long-running (occasionally restarting to clean-up) scripts, pull items from the queue and performs the work, saving the results.
I've literally run hundreds of millions of jobs like this, through such systems. The workers, if they can reach the queue and database servers, don't even have to run on the same machines (I was running dozens of workers across as many servers).
I'm currently working on Web Service which typically should handle 100 request at 1 minute and process all requests parallel. As per I know, the $_GET method only accept one request at a time and process it even if the client send multiple request at same instance of time. Until first request does not complete other request can not be executed.
For Example.. If suppose Client send the 10 request to the Web Service in one instance of time and consider that each request will take 10 secs to execute that means 10 requests will take 100 seconds to execute.
My question is; Can't we reduce the response time. I mean, If i execute all 10 request in parallel that means all request will execute within 10 Secs. I know this type of thing can be achieved in Java. Since I never created the web service in PHP. So please can anyone tell me how to achieve this in PHP.
Is there way to handle the requests concurrently or parallel in PHP. I searched many things regarding this on internet but unfortunately I didn't find appropriate results.
Thanks for replying on my post.. The number of concurrent will be changed once the web service is successfully serves the 100 request per minute.. My first target is to handle 100 request.. If this works fine then my next target will be 1000 per minute..
Although I tried to install pthread on my hosting space(On Godaddy) using pcntl. But unfortunately that installation failed..Also, I did not find proper documentation of PThread. Is it possible to install PThread on my local wamp?? If yes could share the the steps with me..If I successfully install PThread on local wamp then i can set my local ip over the internet so that web service can be accessed over the internet..
I've written some JS scripts on my school's VLE.
It uses the UWA Widget Format and to communicate with a locally-hosted PHP script, it uses a proxy and AJAX requests.
Recently we've moved the aforementioned locally-hosted server from a horrible XP-based WAMP server to a virtual Server 2008 distribution running IIS and FastCGI PHP.
Since then - or maybe it was before and I just didn't notice - my AJAX calls are starting to take in excess of 1 second to run.
I've run the associated PHP script's queries on PHPMyAdmin and, for example, the associated getCategories SQL takes 0.00023s to run so I don't think the problem lies there.
I've pinged the server and it consistently returns <1ms as it should for a local network server on a relatively small scale network. The VLE is on this same network.
My question is this: what steps can I take to determine where the "bottleneck" might be?
First of all, test how long your script is actually running:
Simplest way to profile a PHP script
Secondly, you should check the disk activity on the server. If it is running too many FastCGI processes for the amount of available RAM, it will swap and it will be very slow. If the disk activity is very high, then you know you've found your culprit. Solve it by reducing the maximum number of fastcgi processes or by increasing the amount of server RAM.
I have a database in the cloud, i need to know, at what time and the number of requests will the server crashes down, so I have thought of sending asynchronous requests using php and then find the time needed for serving each of it. I am bit confused as in how to proceed, I am not sure, if cURL will be useful here. Just a layout of how to proceed will be helpful.
ab -n 1000 -c 10 http://yourserver.com/
-n number of requests
-c concurrency
There other tools to benchmark server
ab is a part of apache tools
Use siege or Apache benchmark tool to load test your server by calling single or multiple urls, you can increase concurrency, volume of the requests to the server. siege will give you detail report of the requests and concurrency and how is your server performing, you even call your single server from multiple other servers.
It means that server is heavly loaded with the request i.e, all the threads are busy serving the request.
Solution : either increase the maxThread attribute count for connector in server.xml file or increase acceptCount attribute value.
acceptcount : The maximum queue length for incoming connection requests when all possible request processing threads are in use. Any requests received when the queue is full will be refused.