I have tried reading and have understood PHP console to be a command-line interface (CLI) like one used in composer. I do not understand the difference between a web script and a console script. I do not see the use of having the two.
I want to crawl data from a certain link. Should I use a console script or web script and why?
Please explain in the simplest manner possible.
There is no difference between the two. In most instances, the same PHP script will run whether you execute it from the command line or via the web.
There is, however, a difference between the environment the script will execute within. A CLI script is initiated from and executed within your shell on your computer. It is very self-contained. A web script, on the other hand, is (typically) initiated via a HTTP request from a browser, passes over the web to a web server, is executed on that remote server and a result (typically a web page) is passed back to your browser. In the latter case, there are special environment variables related to the web request made available to the script.
It's a bit hard to know which is the best case for your web crawler script without knowing more detail. But I'd say a command line script is what you're after.
One difference between a web page and a CLI instance is the way the script is executed: webpages will be loaded via a web container, while CLI's will be usually executed by the shell used to launch the PHP. Due to this, a CLI might not have access to all $_SERVER variables as the webpage as practically there is no HTTP request involved.
CLI scripts are useful for doing background tasks that are not initiated by the web server, for example a cron job that periodically cleans your database, on one that executes queued jobs. Think of CLI as shell scripts, you can write a PHP script instead of a bash one.
The PHP interpreter is the same in both cases, and it's up to you to decide which one suits best your needs: webpages are more common, however if you need to have you server do some work without waiting for a web request, the you can go with CLI.
Well, basically a console script is the way for your task.
The difference resides in the fact a Webscript will block your browser, will not show your progress real-time, etc.
I was able to crawl and download about 6000 images from my beloved anime with a console script, showing the progress status, something harder with a Web script as the browser will cache the output. Also you can chain your script and also make some cron magic(assuming you are on nix box)
Related
I have an ubuntu server that I run and I was wondering if it is possible to start up game servers from my website.
As in like running the shell script in a screen and having the game server run continuously?
What you're asking for is definitely possible.
You don't really need to know much PHP, in fact I would recommend something simple like a Perl or Bash script for the server side and just raw HTML (or PHP) to take care of the job.
Take a look at how to run a .sh file from php?
You'll need to:
Create a server side script that runs whatever other script you run to start your game server
Make sure the script is accessible/responds to http requests
Depending on what you choose to use, it should be fairly simple to display the progress of the script and get it done. Keep in mind that there are many different ways to accomplish what you want to do so I'm not going into detail since I'm not even sure what kind of web server you're running.
First of all sorry to post a question that seems to have been flogged to death on SO before. However, none of the questions I have reviewed helped me to solve my specific problem.
I have built a web application that runs an extensive data processing routine in PHP (i.e. MySQL queries, calculations, etc.).
Depending on the amount of data fed to the app this processing can take quite a long time so the script needs to run server-side and independently from the web front-end.
There is a problem, however. It seems I cannot control the script execution time limit as long as the script is invoked via cgi.
When I run the script via SSH and the command line it works fine for however long it takes to process the data.
But if I use the exec() command in a php script called via the webserver I always ends up with the error End of script output before headers after approximately 45 seconds.
Rather than having to fiddle with server settings (a nightmare in terms of portability) I would like to find a solution that kicks off the script independently from cgi.
Any suggestions?
Don't execute the long script directly from the website (AKA, directly from Apache) because, as you've mentioned, it will block until it finishes and potentially time out. Instead, use the website to schedule a job (an execution of the long script) to be run immediately.
Here is a basic outline of how you can potentially do this:
Create a new, small database to store job requests, including fields job_id, processing_status, run_start_time, and more relevant fields
Create some Ajax that hits your server and writes a "job request" to this jobs database, set to execute immediately.
Add a crontab script or bot that periodically watches for new jobs. If it finds a job that is yet to be processed but has passed the run_start_time, run it using exec() or some other command executor. This way the command won't timeout because it is not being run by Apache, but by the cron daemon.
When the command finishes, update the jobs database saying that processing is finished.
From your website, write a frontend that allows the user to see if the requested job is finished yet. Once it finishes, it displays some kind of "Done" indicator or something similar.
I wrote an HTML page with JavaScript, a PHP file and a shell script
they are all on the same machine
I run the shell script, it will open the html page with Firefox,
when the JavaScript finishes, it will POST to a getdata.php
<form id="hidden_form" method="POST" action="getdata.php" ></form>
The getdata.php will do something and then it will send a signal to the shell script
the above is the normal behaviour, I'm afraid at some time, the PHP or JavaScript run into error and don't send signal to the shell script
are there any good and simple way for shell script to detect whether JavaScript is running?
a guy below mentioned that I can let javascript send request to the server periodically, like once every 2 minutes, but how to let shell script notice this/get the signal?
As someone suggested, you can poll the server with setInterval to see if the javascript engine is still running, but that might as well mean that only the javascript thread doing setInterval is the one actually running (depending on what kind of error could arise). You could do the post to "status.php" which would touch/write to a file on the disk, which the bash script could poll to see if it's updated in regular intervals.
I'd however suggest that you look into something like phantomjs which allows you to solve these kind of problems.
No.
Remember: Javascript is running remotely over on the client browser; PHP (and your shell script) are on the server.
What you can do is set up an alarm (on the server) when you send the web page, and invoke an alarm handler if it triggers:
http://www.woodwose.net/thatremindsme/2011/05/forking-parallel-processes-in-bash/
If your JavaScript is running a long-to-process script, you may intermittently submit a request to the server over a set interval. If you don't see the request after a set period... ( interval x2 or w/e ) and the signal has not been received upon completion, you can assume that the JavaScript itself has stopped running / broke.
setInterval("methodofcommunicatingtotheserver", 1000);
etc.
..as stated before: No.
Your best options in this case are, either:
Re-design your script. Is it absolutely necessary for it to send the request to the php page via the javascript in-browser?
If the answer to the above is yes, have a look at Selenium, especially Selenium WebDriver.
It does not provide bindings for bash scripts out-of-the-box, but it's pretty easy to use from other languages such as Python (or PHP, they say).
i start a linux console app from my php5 script, it starts ok but then termintates. I've tried using system(), shell_exec and tried starting as background process but to no avail it starts and then quits.
What i am trying to achieve is from a remote browser start a console app using a php5 script and then it should remain running (just as it would if i started it from a bash shell) , i then want to send commands (from a bash shell it would be keyboard strokes) to the console app from another set of php5 scripts. Hope its clear what i am trying to do.
If anyone could give some info on the best way about doing this, as i think i may have something fundamentally wrong.
I have a Debian Lenny box running apache.The console app is just a simple program that prints to stdout and reads from stdin.
How do you expect to send input to this app? Where is it listening for input?
It simply may only support interactive use, and exit as a result of that. Or, even simpler, it may terminate because it sees that is has no input (nothing piped in or nothing from some file) and since it's not connected to an interactive shell, it has nothing to do. There's no point in waiting for input from a user that doesn't have a way to interact w/ the application.
On every request, PHP starts up, compiles your script and executes it. After execution, the script exists. When the script exits, all of the resources it was using, including file handles, database handles, and pipes to other programs are terminated.
You're going to need to find another way to keep your program open and have PHP communicate with it. Otherwise, every request to your script is going to open a new copy of the program, and then both will exit when the PHP script is complete.
Unfortunately without knowing what the program is, it will be hard to offer suggestions on how to go about doing this.
I am working on a site that require a php script running on a server without any request,
it is a bot script that keeps (not full time but at least once a day) checking client accounts and send alert messages to clients when something happens.
any ideas are appreciated.
Assuming you need to do this on linux, you may run any php script from the browser and from the CLI as well.
You may run a simple php script:
<? echo "Ana are mere"; ?>
like this:
php -f ./index.php
Be careful about file-permissions, and any bug that may creep inside your code, memory leaks or unallocated variables will become VERY visible now, as the process will run continuously.
If you dont want it running in the background all the time, take a look at crontab (http://unixgeeks.org/security/newbie/unix/cron-1.html) to be able to start jobs regularly.
-- edit--
take a look at php execute a background process and PHP: How to return information to a waiting script and continue processing
Basically you want to start a background process, and you may do this by either using exec() or fsockopen() or a file_get_contents() on your own script probably in this order, if don't have access to exec, or socket functions.
Also take a look at http://us2.php.net/manual/en/function.session-write-close.php so the "background script" won't "block" the request and http://us2.php.net/manual/en/function.ignore-user-abort.php
Use a cron job to do it http://www.cronjobs.org/
You can automatically call a script at any interval you like indefinitely. Your hosting provider should support them if they are good.
You should also consider putting a unique key on the end of the page
ie. www.yoursite.com/cronjob.php?key=randomstring
and then only run the script if the key is correct, to prevent bots and other users from running the script when you don't want it run.
If you can't create a cron job, then create a page that does what you want and create a scheduled task on another machine (maybe your PC?) that just goes out and hits that page at a certain time every day.
It's really a hack, but if you absolutely can't set up a cron job, it would be an option.
As Evernoob and Quamis said, you want to have a cron job (UNIX/Linux/Mac OS) or a scheduled task (MS Windows). Furthermore, you can either have the PHP script run using the PHP command line interface (CLI), in which case you can invoke the PHP executable and then your script name. As an alternate, you can use a tool like wget (availble on all platforms) to invoke the PHP script as if someone had typed the URL in the location bar of a web browser.
A php script could not be used like you imagine here. Because it's executed through apache after a request from somewhere.
Even if you do while(1) in your script, apache/php will automaticly stop your script.
Responding to your comment, yes you'll need ssh access to do this, except if your web interface allow you to add cronjob.
Maybe you can write a service which can be executed with a program on another server and do the job.
If you have no access to the server the easiest way would probably be to hit it through the browser, but that would require you or an external script hitting the URL at the same interval each day when you wanted it to one. You may also be able to setup a Selenium test suite that runs locally on a schedule and hits the page. I'm not 100% if that's possible with Selenium though, you may need some 3rd-party apps to make it happen.
Something else you could try would be to see about using PHP's Process Control Functions (link). These will let you create a script that is a deamon and runs in the background. You may be able to do this to keep the script running on the server and firing off commands at programmed intervals. You will still need some way to get it running the first time (browser request or via command line) though.