Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I've been learning to program in PHP and made an application which makes several independent things, the problem is it takes about 20-30 seconds to finish the task, because the code is executed sequentially.
I was reading and found out that there are no threads in php, is there any way to get around?
Edit: added information:
Basically, my application will seek information from news, weather, etc. (with file_get_contents($url)), but performs the functions sequentially, in other words, first fetches the news, then information about weather, and successively, instead of running it all at the same time .
Use some kind of job-queuing software like Gearman or RabbitMQ, then - put those ops in the consumer.
use CURL_MULTI instead, much faster. http://php.net/manual/en/function.curl-multi-init.php
It will reduce the loading \ processing time noticeably if you are reading numerous pages.
You could also try to hack in some threading behaviour by launching different requests to your webserver at the same time. For instance, your index.php would serve a simple page, which contains a number of AJAX calls to, say, fetchNews.php and fetchWeather.php. These requests would then be run asynchronously, in parallel, by the browser, and you'd circumvent phps limit on threading by just launching different webserver requests.
You mention that you're doing a bunch of file_get_contents($url)-calls. These are pretty slow. It would be a huge timesaver if instead of pulling these files in every time you load the page, you cache them to local storage and read them from there: that would be almost instant. Of course, you'd need to keep in mind how fresh you need your information.
For instance you could run a cron job that fetches these files every minute or so. Then you could have your website render this fetched information: the information would only be max 1 minute + the time it takes to run that script out of date.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I have a table, that consists of all the info to run a campaign. The info includes, the time interval to check the campaign (10 mins, 15 mins etc.) and other information to check whether it meets the specific requirement or not, to run the campaign.
At the moment, what I am planning to do is:
Add my code in one php file
In the code, go through all the rows of the table
Check if it's the time to check the campaign or not (via the interval)
If it's the time to check the campaign, then go through other details of the table and based on the set conditions, send an email or SMS.
I am planning to run a cron job which goes through this php file, after every 10 minutes (As it's the shortest check interval)
I need suggestions, whether it's the proper solution or not OR if someone has any better and efficient solution?
That's a decent starting point...
If it's the time to check the campaign...
Keep in mind that sometimes a cron process takes longer than expected, gets stuck or your system crashes in the middle. Ideally your process will keep track of a.) what it's doing, and b.) when it did it. And be able to fix problems like skipped or stuck processing.
It could be that you never want to send a message that late. Then again you may want to make sure all of the missed messages get sent. Your code should be able to handle this case automatically to some degree. Maybe automatically do anything that's should've been done in the last hour but wasn't and ignore anything older than that. For older stuff you'd have to manually run the script. Make sure your script has command line arguments that simplify you forcing it to run for prior time intervals and specific campaign IDs. This will make your life way easier after a disaster.
I suggest that you have some kind of reporting so you can keep track of your processing in real time. Pretty simple if you're writing state info to your database. Add on an end of processing timestamp and you can even see how long your cron jobs are running. If you don't want to use this state info in your cron job you can just write it to a log file instead of a database. And in that case (if needed) you would use a lock file to indicate when a cron job is running and prevent other cron jobs from starting at the same time. Regardless, it's good practice to write a log file so you have a record of what happened. Imagine if your cron job sent an email but crashed while attempting to write the state to the database. You'd at least have a log line to help you investigate later.
I am planning to run a cron job which goes through this php file,
after every 10 minutes (As it's the shortest check interval)
So the speed of your script will vary with the amount of data, latency of external services (assuming your script talks directly to such services). I would start with a much longer cron job start interval - assuming that your client/use case allows for that. If you follow the suggestion above to have your script automatically handle skipped times this isn't a problem. The more stuff you're processing the more time your script will eventually need. So on day 1 it might only need 1 second. But on day 300 it might need 15 minutes? (At that point you could decide that you want to have multiple processes/threads running at the same time with each one focused on a single campaign or range of campaigns. Who knows...) But you'll know because you have reports/alerts/logs on the start/end processing times.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I want to run more than 800 PHP scripts in the background simultaneously on Linux. Each PHP script will execute forever, meaning it will not stop once it has started. Each script will send request and get response from the server. How much RAM do I need for that? Will it possible to run more than 800 scripts? What kind of hardware do I need?
You're probably doing it wrong. Since your scripts are I/O bound instead of CPU bound, an event loop will help you. That way you just need as many workers as CPU cores.
This approach does not only lower your required resources in terms of memory and CPU cycles, but also reduce the number of scripts you have to monitor.
There are various PHP implementations, here are the three most popular ones:
Amp
Icicle
React
Well, I'm sure the hardware you seek exists, but you will need a time machine to access it ... do you have a time machine ??
I'm going to assume you do not have access to, or plans to build a time machine, and say that this is not sensible.
In case humour didn't do it for you; There is no hardware that is capable of executing that many processes concurrently, setting out to create an architecture that requires more threads than any commonly available hardware can execute is clearly a bad idea.
If all you are doing is I/O, then you should use non-blocking, asynchronous I/O.
To figure out how much ram you will need is simple, how much data will be stored in memory during execution x 800.
You can improve memory usage by setting variables to null as soon as you are done with the data, even if you are re-using the variables again I would highly recommend this. That way the execution will not turn into a memory leak filling up RAM and crashing your server.
$myVariable = null; //clears memory
The second part of your question "execute forever" is easy too, you simply need to tell PHP to allow the script to run for a long time... Personally though I would do the following:
Setup 800 crons of your script all running every 1 hour.
I would assume your script is in an infinite loop... note the time into a variable before the infinite loop and in the loop check if 1 hour has passed, if 1 hour has passed end the loop and the process (a new one will replace it).
Doing the above will ensure the process will be cleaned every hour, also if for some reason a process gets killed by the server due to resource or security checks the process will spring back up within the hour.
Of course you could lower this to 30 mins, 15 mins, 5 mins depending on how heavy each loop is and how often you want to re-establish the processes.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
The title might be a bit unclear. But I have this problem with my website. I have some data stored in a sql database, including a timestamp. But now I need to make some calculations with that data and send an email, after a certain duration after that timestamp. But how do I do this? Since my website and my database only contain static data...
So to be clear: I have a site on shared hosting with a database. In that database are items that all contain a timestamp value. Now I need the server to undertake an action for every item in the database on a certain duration after that item's timestamp, for example send a mail.
This is probably an easy question, but I have no clue what to look for...
EDIT: I think I have found a solution by using setcronjob.com , thanks!
You could run a job every so often to figure out how old that timestamp is and then send the emails you need to send. But how to run the job?
If you can run scheduled jobs on your hosting (ie, you own the machine/VM and are not on shared hosting) you can run these jobs every x days/hours/minutes/seconds. In Linux, look up cron jobs, and Windows look up Task Scheduler (I think)
If you're on shared hosting it's more difficult. Maybe you can convert some of your static pages into dynamic pages that run these jobs? Depending on how long they take to run you might want to look at running these jobs in a new thread (pthreads). You'll need to have pthreads installed for this on your hosting, so #1 - scheduled jobs - is definitely more preferable.
You probably want to use a Cron task to search your database for every row with the timestamp near the current time. From that point, you can simply do your calculations and send the email.
Could you tell me more about that "certain duration after that timestamp"?
Also, in the database you could setup a job that runs on a schedule to make the calculations and email the results.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I’m looking at ways to send out 1 email every 1 min. I’ve looked at the example below where the top answer is to use PHP sleep() function.
However, I’ve also found suggestions that sleep() might slow down the server.
I’m not looking for exact answers but general approaches would be great.
However, I've also found suggestions that sleep might slow down the
server.
Yes, and hitting the pause button on a movie playing on your computer will slow down the duration of the film based on the amount of time you pause the movie.
The purpose of sleep is to put a pause in your script. As described in the official PHP documentation:
Delays the program execution for the given number of seconds.
So yes, it slows down your server. But only on content or pages where sleep is active.
So if this is a fronted script with sleep in it, it slows down the ability for anyone to view content via the PHP script that uses sleep. Place it in the middle of a page where HTML is rendering with a 1 second delay & your page now takes 1 second longer to render.
If this is a backend process only you really know about or trigger, no big deal. It’s a background process anyway so it will just expectedly slow things down in that realm.
That said, let’s look at your core question which is the first sentence of your post:
I’m looking at ways to send out 1 email every 1 min.
Then what you are looking for is a cron job which is a timed job on a Unix/Linux system. An entry for a cron job for something sending mails out every minute might be something like this:
* * * * * /full/path/to/php /full/path/to/your/php/script.php
But that is superficial. It basically just triggers the script.php every minute. Then within your script.php you would have to create some core logic that would control what happens each time it’s triggered. If you are using a database, then maybe you could create a last_sent field where you sent a time stamp of the last time a mail was sent to a recipient and then you act on that. But again, the logic is based on your core needs.
But at the end of the day, I am not too clear how sleep would factor into any of this. Might be worth it to take a step back and better architect your script to fit your needs knowing what cron is, what sleep is & what they are as well are not.
It is generally done with a separated worker and a queue manager.
That's it: you have a queue manager (i.e. RabbitMQ) that a sending email worker is bound to,
Then when you need to send 10 emails you put all of them to the corresponding queue at once in the script that serves HTTP response. This step is immediate.
Then a worker reads emails one by one and sends them with required delay. This step takes some time but we don't care.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I'm trying to write a Wordpress plug-in that automatically posts a blog post at a certain time of day. For example, read a bunch of RSS feeds and post a daily digest of all new entries.
There are plug-ins that do something similar to what I want, but many of them rely on a cron job for the automated scheduling. I'll do that if I have to, but I was hoping there was a better way. Getting a typical Wordpress user to add a cron job isn't exactly friendly.
Is there a good way to schedule a task that runs from a Wordpress plug-in? It doesn't have to run at exactly the right time.
http://codex.wordpress.org/Function_Reference/wp_schedule_event
pseudo-cron is good but the two issues it has is
1, It requires someone to "hit" the blog to execute. Low volume sites will potentially have wide ranging execution times so don't be two specific about the time.
2, The processing happens before the page loads. So if teh execution time happens and you have lots of "cron" entries you potentially upset visitors by giving them a sub standard experience.
Just my 2 cents :-)
vBulletin uses a sort of Pseudo-Cron system, that basically checks a schedule on every page access, and fires any processes that are due then.
It's been a while since I worked with Wordpress, but I think something like this should work if it runs on each page view.
I think the best way to do this is with a pseudo-cron. I have seen it on several occasions, and although not exact in the timing, it should do what you need it to do. Since in Wordpress the index.php is the first thing always hit based upon the settings in the .htaccess, create a file called pseudo-cron.php, dump it into the root directory, and then require it once from the index. Whenever someone hits the site, it will run, and you can use it to initiate a script, and check if another daily digest needs to be generated depending upon the time of the day, and when the previous digest ran.