I am using PHP Simple HTML DOM Parser to get a data from a website. I call this following function in a while loop say for example 15 actors data to be scraped. what happens is after scraping data for 5 or 6 or somewhere in loop, it fails to open the stream and breaks the script. I used try and catch function. but still i am not to fix it out. Actually my need is even if it fails in the middle of a loop, it has to go on till the end. Can anyone assit me pls.
function get_data($actor) {
try {
$html = file_get_html('http://someurl.com', false, $context);
if (isset($html)) {
$table1 =$html->find('table.table_border',0) ;
// doing some process here
}
} catch(Exception $e) {
echo $tt_id."-Failed once.<br>";
}
}
Sometimes i am getting the following error and the script stops.
Warning: file_get_contents(http://someurl.com) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 503 Service Temporarily Unavailable
Fatal error: Call to a member function find() on a non-object in somefile.php on line 37
Related
I am trying to create a simple script with file_get_contents and Facebook graph. However, there seems to be some issue with the error handling.
Here is the code I have:
$json = file_get_contents('https://graph.facebook.com/', true);
if($json !== FALSE) {
echo "test";
}
It returns the following error message:
Warning: file_get_contents(https://graph.facebook.com/): failed to open stream: HTTP request failed! HTTP/1.1 400 Bad Request in /home/test/tst/mellemmand/test.php on line 4
I have a php script that will return a google search but it gives me the HTTP error.
if (strstr(file_get_contents("http://www.google.com/search?q=site:http://". $url ."&gws_rd=ssl"), 'did not match any documents.'))
{
return "No";
}
else {
return "Yes";
}
A PHP Error was encountered
Severity: Warning
Message: file_get_contents(http://www.google.com/search?q=site:http://google.com&gws_rd=ssl): failed to open stream: HTTP request failed! HTTP/1.0 503 Service Unavailable
Filename: libraries/google_index.php
Line Number: 8
error : HTTP request failed! HTTP/1.0 503 Service Unavailable.
how can i force this or something so it will get the contents.
If you put an URL as GET parameter of another URL, you must urlencode() it. Otherwise, some characters like ? or & will be considered as parts of the main URL.
Try encoding the whole URL, like:
$string = "google.com/search?q=site:". urlencode($url ."&gws_rd=ssl")
My script try to open every site from a file read line by line but when try to parse a site which doesn't exist the script stop and give me these error:
Warning: file_get_contents(): php_network_getaddresses: getaddrinfo failed: No such host is known. in C:\xampp\htdocs\simple_html_dom.php on line 76
Warning: file_get_contents(http://www.thissitedoesntexist.com): failed to open stream: php_network_getaddresses: getaddrinfo failed: No such host is known. in C:\xampp\htdocs\simple_html_dom.php on line 76
Fatal error: Call to a member function find() on a non-object in C:\xampp\htdocs\test2.php on line 11
How can I fix it ? How can I make it to run even if a site doesn't exist... to read a new line from my file(which means to read another site and then to read it) Also the scrip stop when receive errors like 404, 403, etc.
I would do this, to check for the error codes ---
foreach ($sites as $site) {
$ch = curl_init('http://'.$site);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// Execute
curl_exec($ch);
// Check if any error occurred
$info = curl_getinfo($ch);
if($info['http_code'] != 200) {
continue;
}
//rest of loop, here --
}
You could even do something different depending on the error code you get with a case-switch --
According to the documentation, file_get_contents() returns FALSE on failure.
So, check what it returns to be sure that the site exists before trying to parse the returned content. If it doesn't exist, iterate to the next line in the file and keep continue the process:
if($file = file_get_contents("http://www.thissitedoesntexist.com")) {
// Parse file here
// Then continue reading the file by
// starting at the next line.
continue;
}
Reference:
file_get_contents()
I'm getting an error from this Twitter script that is causing the rest of the page to not load. Not sure why suddenly this is happening, where it was functioning properly for quite some time.
The script looks like this, and it pulls the users current status:
<?php
$response = new SimpleXMLElement('http://twitter.com/users/show/tuscaroratackle.xml',NULL,TRUE);
echo $response->status->text.'';
?>
Here's another post that I was trying to figure out the answer to another bug which pointed me to this Twitter error.
You can see it here in the footer, or a screengrab of the output: http://cl.ly/33IZ.
The relevant error (which is displayed in the footer of the page you linked to) is:
Warning: SimpleXMLElement::__construct(http://twitter.com/users/show/tuscaroratackle.xml) [simplexmlelement.--construct]: failed to open stream: HTTP request failed! HTTP/1.1 400 Bad Request in /home5/tuscaror/public_html/footer.php on line 47
Warning: SimpleXMLElement::__construct() [simplexmlelement.--construct]: I/O warning : failed to load external entity "http://twitter.com/users/show/tuscaroratackle.xml" in /home5/tuscaror/public_html/footer.php on line 47
Fatal error: Uncaught exception 'Exception' with message 'String could not be parsed as XML' in /home5/tuscaror/public_html/footer.php:47 Stack trace: #0 /home5/tuscaror/public_html/footer.php(47): SimpleXMLElement->__construct('http://twitter....', 0, true) #1 /home5/tuscaror/public_html/index.php(119): include('/home5/tuscaror...') #2 {main} thrown in /home5/tuscaror/public_html/footer.php on line 47
The first warning tells you what happened: "HTTP request failed! HTTP/1.1 400 Bad Request".
So, for some reason, your server is failing when making the HTTP request to twitter to retrieve the document "http://twitter.com/users/show/tuscaroratackle.xml". The return code is 400 Bad Request.
I just tried that same request from my web browser, and it worked fine, so either twitter was temporarily "out to lunch" (which does happen from time to time), or there is something unique about your server's network configuration. My first guess would be that somewhere up-stream from your server, someone has installed an HTTP proxy which is (for some unknown reason) blocking your request.
Here's what twitter has to say about it:
400 Bad Request: The request was invalid. An accompanying error message
will explain why. This is the status code will be returned during rate limiting.
Here is twitter's page on Rate Limiting. I suspect that this is your culprit. If you think otherwise, then you might try retrieving the document as a string and examining it before you try to parse it, so you can see what the message is.
This is quick and dirty, but it'll get the message so you can see what's going on:
$str = file_get_contents('http://twitter.com/users/show/tuscaroratackle.xml');
echo $str;
that may fail due to the 400 response code. if so, you'll need to use php curl to get the un-parsed response body.
good luck!
I have a PHP script that connects to an api and posts information to their systems, but when its trying to connect it throws a fatal error:
Fatal error: Uncaught exception 'Exception' with message 'Problem with
'http://apitestserver.co.uk:9000/Service.svc/Items' in
/var/www/html/e/connect_test.php:17 Stack trace: #0
/var/www/html/e/connect_test.php(39):
do_http_request('http://apitest....', 'hello') #1 {main} thrown in
/var/www/html/e/connect_test.php on line 17
If I send it to a PHP script which just grabs the IP then it works, but if I send it to the API it doesn't. My PHP script creates XML and then forwards to the server. I was getting errors so I just created the following smaller script purely to test the connection:
function do_http_request($url, $data, $method = 'POST',
$optional_headers = 'Content-Type: application/atom+xml') {
$params = array(
'http' => array(
'method' => $method,
'content' => $data
)
);
if ($optional_headers !== null) {
$params['http']['header'] = $optional_headers;
}
$ctx = stream_context_create($params);
$fp = fopen($url, 'rb', false, $ctx);
if (!$fp) {
throw new Exception("Problem with $url");
}
$response = #stream_get_contents($fp);
if ($response === false) {
throw new Exception("Problem reading data from $url");
}
$metaData = stream_get_meta_data($fp);
fclose($fp);
if(!preg_match('~^HTTP.*([0-9]{3})~',
$metaData['wrapper_data'][0], $matches)){
throw new Exception('MALFORED RESPONSE - COULD NOT UNDERSTAND HTTP CODE');
}
if (substr($matches[1], 0, 1) != '2') {
throw new Exception('SERVER REPORTED HTTP ERROR ' . $matches[1]);
}
return $response;
}
$data = 'hello';
$paul =
do_http_request('http://apitestserver.co.uk:9000/Service.svc/Items',$data);
echo $paul;
If I change the URL to a simple script on another one of our servers which just grabs the IP of the incoming connection and returns it:
$ip=$_SERVER['REMOTE_ADDR'];
echo 'IP equals = ' . $ip;
Then it works fine with no errors.
Update -
with errors on it throws the following warning, probably because the script is not sending the correct info to the API
Warning: fopen(http://apitestserver.co.uk:9000/Service.svc/Items)
[function.fopen]: failed to open stream: HTTP request failed! HTTP/1.1
500 Data at the root level is invalid. Line 1, position 1.
Also note that I can access the api server fine using fiddler to send manually created items across, its just when this script tries to connect and send data there is an issue. I wrote another quick script which connects and prints out the default response (an rss feed of submitted items)
When I run the 'main' connector script it throws the following two errors
Warning: fopen() [function.fopen]: php_network_getaddresses:
getaddrinfo failed: Name or service not known in
/var/www/html/e/sequence.php on line 65
Warning: fopen(http://apitestserver.co.uk:9000/Service.svc/Items)
[function.fopen]: failed to open stream: Operation now in progress
in /var/www/html/e/sequence.php on line 65
I would suggest using curl instead of fopen(). curl is both more flexible, and more secure. Many servers disable allow_url_fopen, curl also fails more gracefully when there are problems on the remote server.
I just tested the URL in my browser and got a connection error.
Maybe the problem is with their server (or you have the wrong URL)?
Edit:
Looks like ther server is throwing a 500 error - maybe because the data you're posting is invalid.
You could use a packet sniffer and check the exact data you're sending to the server (or use cURL as suggested by #acrosman)
apitestserver.co.uk doesn't respond to ping and the url is inaccessable. The API-server doesn't seem to be up.
or You can change the headers:
$optional_headers = 'Content-Type: application/json')
First error message suggests problems with apitestserver - it returns error code 500, which states for "Internal server error".
Second one informs that PHP can't resolve host name into IP address - I'd look into DNS configuration and/or hosts file.
Apparently 'fopen' function does not return proper stream. There can be many possibilities when that happens, but according to PHP manual, fopen function on fail should present E_WARNING message with proper commentary. It is possible that you have errors of that class silenced - call error_reporting(E_ALL) at the beginning of your script and look for some relevant error messages.