My web application needs to parse remote resources from multiple remote servers. The problem is that the output of those remote servers is long / staged. I hence need a piece of code implementing the following logic:
Populate array $links_array with a set of links.
Some code here....
For $i in count($links_array)
`$results_array[$i] = {what has been output until now without waiting for full response}
Some code here....
The answer must not use extensions (except cURL) and work on PHP 5.3.
Thanks a lot for your help.
It seems like you need an asynchronous method of download links. You can look into the Guzzle Library (https://blog.madewithlove.be/post/concurrent-http-requests/). Another option is to use multi curl: http://php.net/manual/en/function.curl-multi-init.php
Related
I'm building a Joomla 3 web site but I have the need to customize quite a few pages. I know I can use PHP with Joomla, but is it also possible to use Python with it? Specifically, I'm looking to use CherryPy to write some custom pieces of code but I want them to be displayed in native Joomla pages (not just iFrames). Is this possible?
PHP execution of Python "scripts"
This will work for scripts, which do stuff and return the output, not for CherryPy.
<?php
// execute your Python script from PHP
$command = escapeshellcmd('myPythonScript.py');
$output = shell_exec($command);
echo $output;
// take response content to embed it into the page
?>
PHP to access a Python/CherryPy served website
import cherrypy
class HelloWorld(object):
def index(self):
return "Hello World!"
index.exposed = True
cherrypy.quickstart(HelloWorld())
This starts a http://localhost:8080 and you should see Hello world!.
Now you could access CherryPy's output by accessing it at it's localhost:port.
Not good performance-wise, but works.
<?php
$output = file_get_contents('http://localhost:8080/');
echo $output;
?>
Joomla + Ajax to access a Pyhton/CherryPy served website
An alternative solution would be, to not use PHP to fetch the content, but to do the fetch from client-side. Basically, you would use an Ajax-Request to the CherryPy served website, to fetch it's content and embed it into the dom of the Joomla served page.
// add jQuery Ajax reqeust from your Joomla page to CherryPy
$.ajax({
url: "https://localhost:8080/", // <-- access the 2nd served website
type: 'GET',
success: function(res) {
//console.log(res);
alert(res);
$("#someElement").html(res);
}
});
Probably the most efficient way of constructing pages from different sources (Joomla and CherryPy) is probably using Edge Side Includes esi.
Basically you run a Joomla instance that serves the documents that contain ESI tags. You run a CherryPy instance that serves the pieces you want to put on the location of the esi tags. And you run a reverse proxy that supports ESI like varnish to stitch it all together.
That may look like a lot of moving parts, but it will be responsive like... something that is really responsive. Tune the caching parts and you relieve your Joomla and database from a lot of work.
The exec way from the other answers will work for small python scripts that print something, but will not work for CherryPy.
You can execute it through a php file. Try this:
exec("python/path/your-script.py");
It's possible, but not very efficient.
You can execute a Python script from PHP using the exec() function:
PHP Code:
exec("python /path/to/python-script.py");
There are also a variety of similar PHP functions that can be used to
accomplish the same thing with minor differences to the way input and
output are handled (passthru, system, proc_open, backticks).
The Python script will be executed using its command line interface -
not using CGI (or similar) interface as you would have if the web
server were directly executing the Python script. This means that the
Python script will not have access to information about the HTTP
request - GET/POST values, the client's IP address, the page URL, etc.
You could pass this information from PHP to Python using command line
parameters, a pipe, a temporary file or some other form of
inter-process communication, but you need to pass each piece of
required information explicitly.
The reason this is inefficient is because every call to exec will
spawn a whole new process for the Python script. That's a fairly
expensive operation to do on every HTTP request (this is why servers
like Apache and interfaces like Fast-CGI re-use child processes and
threads instead of creating new ones). Additionally, if you have more
than one call to exec, every single one is going to spawn a new
process.
Extracted from here.
More info : Unable to put Python code to Joomla
Before considering Python
What are you wanting to customize? (perhaps some clever Javascript or a Joomla extension already exists)
Is the Joomla-way not a better solution for your problem, given the fact that you're using Joomla? (change the template, or the view-templates of the modules and component in particular)
i.o.w.: do you understand Joomla enough to know that you need something else? See below:
If Python is still the way to go:
Does your hosting support Python?
Should you reconsider your choice of CMS?
I like your choice of CherryPy.
I'd like to create an RPG in JavaScript. The RPG will have a lot of dialog in it. I have heard XML is the way to go..., but I have no experience using XML, and a good amount of experience with PHP and MySQL.
My questions are:
Would it be better to store dialog in a MySQL database and access it with PHP using IDs? Or should I just keep it all on the front end by accessing XML files with Javascript?
If I use XML, do I need to run the javascript on a webserver (Like WAMP)? Or can I parse it locally with:
function parseXML() {
xmlhttp=new window.XMLHttpRequest();
xmlhttp.open("GET","dialog.xml",false);
xmlhttp.send();
xmlDoc=xmlhttp.responseXML;
}
(Which is currently giving the error XMLHttpRequest cannot load file:///C:/Users... Cross origin requests are only supported for HTTP.
Even after reading up on XML, I'm still not sure how I can integrate it with JavaScript. Can someone help give me an example of how I could take a sample rpg XML dialog and meaningfully parse it in JavaScript?
Thank you
Why don't you use JSON instead? It's much easier to use from JavaScript code. While you're at it, avoid usind XMLHttpRequest by hand and use some library that handles the bad stuff for you (e.g., jQuery).
For the "dialog" part, I'm not sure what you're gonna use MySQL for. Anyway, you can use any form of storage on the server side, provided that your PHP scripts give well-formed replies to the client side code.
Would it be better to store dialog in a MySQL database and access it
with PHP using IDs? Or should I just keep it all on the front end by
accessing XML files with Javascript?
The main advantage of having a database here is that you gain a lot of flexibility. You can create/edit/delete a piece of dialogue and its relations quite easily and without having to maintain/regenerate a huge data file every time (As you would do with a XML or JSON or w/e formalism you choose).
On the other hand, this solution might be overkill if you just have simple dialogues with no relations and limited content.
Since you are about to do a game in Javascript, and if your content is not changing that much, you can consider a JSON encoding for your dialogues. You could parse the data instantly via Javascript and still have the possibility to make all kind of complex relations between dialogues as you would in XML.
If I use XML, do I need to run the javascript on a webserver (Like WAMP)? Or can I parse it locally with: ...
You have this error because you indeed need to run a server here. If you just want to do some tests, no need to use WAMP or whatever big webserver, just run a python server for now and that will do the trick.
cd /yourProject
python -m SimpleHTTPServer
Let the server run and open your browser at:
http://127.0.0.1:8000
Give the input file a base url like http://localhost/rpgame/dialog.xml.
I am currently working on developing an API for a company. Specifically, here's my issue. They have JavaScript arrays on a webpage that their webmaster updates. I have to pull these arrays into either a simple JS script or PHP file and get the contents of these arrays, which I can then arrange according to the API's specifications and output it as JSON.
How do I pull a JavaScript variable in from a remote page in either PHP or jQuery/JS and make it usable for other applications?
No, I don't have access to the company's website. I have to work off of page scraping for this one.
Thank you!
You can't access private javascript variables remotely, due to Same origin policy. They would have to output the arrays in some kind of readable format that you could access using AJAX, probably as JSON.
Edit: As mentioned below, if the array is explicitly defined as text in a javascript file, you could grab the contents of that file using cURL in PHP
If I where you, I’d use PHP to file_get_contents (or CURL depending on the server config) the page and then parse it based on whatever the markers are to find the value of the variable, assuming it’s written out to the page in the first place.
I understand that within same folder, I can use include() function for external PHP file, but now I would like to call the function in another PHP file which located in another URL.
For example, my live website (liveexample.com/table.php) has drop-down list and table, but without data.
My another PHP file (dataexample.com/data.php) is connected to database and process to extracting data out. But, it is in another server.
I need to make my data on [dataexample.com/data.php] delivers to [liveexample.com/table.php] and let the looping to draw table with data out on [liveexample.com/table.php] page.
Anyone has idea to design this method of delivering data from another server to another by using function call in PHP?
Or any other better solution to deliver my data between two different servers such as make the data record set into array and send to [liveexample.com/table.php]?
Please give me advise or consultation. Appreciate much!
I think SOAP webservice would be perfect for you to attain what you want but if possible just copy the same codes you have from the separate server.
If you make [dataexample.com/data.php] output your data as XML, then you can use it as a web service. What that means is, you can take that XML output (by sending a request the the data URL), and then parse it to load the data. This way, you can use that service any way you want. One way would be like you wanted, other examples would be via AJAX, or Flash etc.
So here are a few topics worth looking into:
using PHP for web services: http://wso2.org/library/3032
parsing XML data: http://www.w3schools.com/php/php_xml_simplexml.asp
I hope this will give a pretty good idea of how to achieve what you want to accomplish, because there a few options you can go by. Like Cristopher said, SOAP is one of them.
Have a great day.
I would write a WordPress plugin to parse all image source and check it's broken link or not.
My idea is :
Select all post&page's images by regex from MySQL
Navigate the image url and get the response header (404 ,403 error etc)
print a report
Since i don't need actual to download the binary file, so in performance ,compare in CURL , fopen , fsocketopen
Which one is worst to use?
And one more question, which method can execute in multi-thread?
The cost of opening a connection to the remote server makes the performance of the library a fairly moot point. In other words it isn't worth worrying about the performance of the functions.
A better option would be to use wse whatever function allows you to make HEAD requests (Which only return the HTTP headers). While you can do it with fsockopen (I don't know about fopen), it is a lot of work when cURL has code already written to send the request and parse the response.
For an example of how to do a head request using cURL see this answer.
And one more question, which method can execute in multi-thread?
PHP doesn't have threads