I'm making a request to URL to get data using Goutte. But the server where I'm making request is slow. So sometimes laravel throws error of time out. When this error comes, I have to make entry of this error log in databse with some additional data (i.e, id etc). I have searched on internet. But I found all solutions related to customise error message etc. What I want is when laravel throws error of time out, I have to make entry in database with additional data and then redirect page. If any one knows the solution, it will be appreciated.
Here is my code.
use Goutte\Client;
class WebScrapingController extends Controller {
public function index() {
try {
$this->crawler = $this->client->request('GET', $url . '?' . $data);
}catch(Exception $e){
// Here I want to make entry in database and then redirect to another page
dd(['Connection time out', $i, $e]);
}
}
}
Here is my error message
ConnectException in CurlFactory.php line 186:
cURL error 7: Failed to connect to myurl port 80: Connection timed out (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)
Also getting this error sometimes
RequestException in CurlFactory.php line 187:
cURL error 56: Recv failure: Connection timed out (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)
I'm using laravel 5.3 and this scraper.
Well, this is how I would do it:
use Goutte\Client;
class WebScrapingController extends Controller {
public function index() {
try {
$this->crawler = $this->client->request('GET', $url . '?' . $data);
} catch(\ConnectException $e){
$log = new Log();//define this as a model
$log->setMessage($e->getMessage());
$log->save();
} catch(\RequestException $e){
$log = new Log();//define this as a model
$log->setMessage($e->getMessage());
$log->save();
} finally {
$yourModel = YourNamespace\YourModel::find($url);//or, depends on your model structure and DB
$yourModel = YourNamespace\YourModel::where('url',$url)->first();
}
}
}
You can also move the saving of the log in a private method, I left it like this so you can see that it is possible to treat several exceptions differently, or you could catch as a general exception:
public function index() {
try {
$this->crawler = $this->client->request('GET', $url . '?' . $data);
} catch(\Exception $e){
$log = new Log();//define this as a model
$log->setMessage($e->getMessage());
$log->save();
} finally {
$yourModel = YourNamespace\YourModel::find($url);//or, depends on your model structure and DB
$yourModel = YourNamespace\YourModel::where('url',$url)->first();
}
}
If you want to log in some files you have the Log facade: use Illuminate\Support\Facades\Log;
Related
I have a task to identify the existence of the links. As a result I am using guzzle client to identify if the link is existence or not, i.e if the response header received is 200, then the link is existence else not.
Below is my code snippet
public function checkUrl($url) {
$result['isValid'] = false;
try {
$response = $this->client->get($url, ['verify', false]);
} catch (\Exception $ex) {
$result['isValid'] = false;
$result['message'] = 'Some error message';
return $result;
}
if ($response->getStatusCode() == Response::STATUS_CODE_200) {
$result['isValid'] = true;
}
$result['message'] = 'Success - ' . $response->getStatusCode();
$response->getBody()->close();
return $result;
}
where $this->client is initialized to GuzzleHttp\Client object once in the constructor
When I run my script, after some time it throws me the error as follows:
PHP Fatal error: Uncaught ErrorException: include(/project/vendor/zendframework/zend-view/src/Model/ConsoleModel.php): failed to open stream: Too many open files
And when I check the list of open files using the command lsof -p <process id> -n, I noticed that there lots of open files are as a result of guzzle request (i.e curl responses) and it seems to be the cause of this exception.
Is there any suggestion for the solution through which I can close those responses?
I've written some code in Laravel 5.2 to retrieve results from an unrelible API source. However, it needs to be able to automatically retry the request on failed attempts, as the API call results in a 503 about a third of the time.
I'm use Guzzle to do this, and I think I know where to put the code that will intercept the 503 responses before they are processed; but I'm not sure what to actually write there.
The guzzle documentation doesn't offer much as far as retries go, and all of the examples I've come across of Guzzle 6 only show how to retrieve results (which I can already do), but not how to get it to repeat the request if needed.
I'm by no means asking anyone to do the work for me - but I think I'm approaching the limits of my understanding on this. If anybody can point me in the right direction, it'd be much appreciated :)
EDIT:
I will try and revise. Please consider the following code. In it, I want to send a GET request which should normally yield a JSON response.
DataController.php
$client = new \GuzzleHttp\Client();
$request = $client->request('GET', 'https://httpbin.org/status/503'); // URI is for testing purposes
When the response from this request is a 503, I can intercept it here:
Handler.php
public function render($request, Exception $e)
{
if ($e->getCode() == 503)
{
// Code that would tell Guzzle to retry the request 5 times with a 10s delay before failing completely
}
return parent::render($request, $e);
}
I don't know that that is the best place to put it, but the real problem is I don't know is what to write inside the if ($e->getCode() == 503)
Guzzle by default throws exceptions when a non 2** response is returned. In your case you're seeing a 503 response. Exceptions can be thought of as errors that the application can recover from. The way this works is with try catch blocks.
try {
// The code that can throw an exception will go here
throw new \Exception('A generic error');
// code from here down won't be executed, because the exception was thrown.
} catch (\Exception $e) {
// Handle the exception in the best manner possible.
}
You wrap the code that could throw an exception in the try portion of the block. Then you add your error handling code in the catch portion of the block. You can read the above link for more information on how php handles exceptions.
For your case, lets move the Guzzle call to it's own method in your controller:
public function performLookUp($retryOnError = false)
{
try {
$client = new \GuzzleHttp\Client();
$request = $client->request('GET', 'https://httpbin.org/status/503');
return $request->send();
} catch (\GuzzleHttp\Exception\BadResponseException $e) {
if ($retryOnError) {
return $this->performLookUp();
}
abort(503);
}
}
Now in your controller you can execute, $this->performLookUp(true);.
Just to add some information to clarify a few points that Logan made.
Guzzle "can" throw exceptions on a Response other then 2**/3**. It all depends upon how the GuzzleHttp\HandlerStack is created.
$stack = GuzzleHttp\HandlerStack::create();
$client = new Client(['handler'=> $stack]);
$client = new Client();
// These two methods of generating a client are functionally the same.
$stack = New GuzzleHttp\HandlerStack(new GuzzleHttp\Handler\CurlHandler());
$client = new Client(['handler'=> $stack]);
// This client will not throw exceptions, or perform any of the functions mentioned below.
The create method adds default handlers to the HandlerStack. When the HandlerStack is resolved, the handlers will execute in the following order:
Sending request:
http_errors - No op when sending a request. The response status code is checked in the response processing when returning a response promise up the stack.
allow_redirects - No op when sending a request. Following redirects occurs when a response promise is being returned up the stack.
cookies - Adds cookies to requests.
prepare_body - The body of an HTTP request will be prepared (e.g., add default headers like Content-Length, Content-Type, etc.).
send request with handler
Processing response:
prepare_body - no op on response processing.
cookies - extracts response cookies into the cookie jar.
allow_redirects - Follows redirects.
4.http_errors - throws exceptions when the response status code >= 300.
When provided no $handler argument, GuzzleHttp\HandlerStack::create() will choose the most appropriate handler based on the extensions available on your system. As indicated within the Handler Documentation
By manually creating your GuzzleHttp\HandlerStack, you can add middleware to the application. Given the context of your original question "how do i repeat the request" I believe you are most interested in the Retry Middleware that is provided within Guzzle 6.1. This is a middleware that retries requests based on the result of the provided decision function.
Documentation has yet to catch up with this class.
final class HttpClient extends \GuzzleHttp\Client
{
public const SUCCESS_CODE = 200;
private int $attemptsCount = 3;
private function __construct(array $config = [])
{
parent::__construct($config);
}
public static function new(array $config = []): self
{
return new self($config);
}
public function postWithRetry(string $uri, array $options = []): Response
{
$attemptsCount = 0;
$result = null;
do {
$attemptsCount++;
$isEnd = $attemptsCount === $this->attemptsCount;
try {
$result = $this->post(
$uri,
$options,
);
} catch (ClientException $e) {
$result = $e->getResponse();
} catch (GuzzleException $e) {
if ($isEnd) {
Logger::error($e->getMessage());
$result = $e->getResponse();
}
}
} while ($this->isNeedRetry($result, $attemptsCount));
return $result;
}
private function isNeedRetry(?Response $response, int $attemptsCount): bool
{
return $response === null && $attemptsCount < $this->attemptsCount;
}
I have php code that execute python cgi and I want to pass python trace (returned from cgi) as extra data to php exception how can I do this and how can I get that value from catch(Exception e) { (It should check if that extra value exesit or not).
I have code like this:
$response = json_decode(curl_exec($ch));
if (isset($response->error)) {
// how to send $response->trace with exception.
throw new Exception($response->error);
}
return $response->result;
and I use json-rpc library that should return that data to the user:
} catch (Exception $e) {
//catch all exeption from user code
$msg = $e->getMessage();
echo response(null, $id, array("code"=>200, "message"=>$msg));
}
Do I need to write new type of exception or can I do this with normal Exception? I would like to send everything that was thrown in "data" =>
You need to extend Exception class:
<?php
class ResponseException extends Exception
{
private $_data = '';
public function __construct($message, $data)
{
$this->_data = $data;
parent::__construct($message);
}
public function getData()
{
return $this->_data;
}
}
When throwing:
<?php
...
throw new ResponseException($response->error, $someData);
...
And when catching:
catch(ResponseException $e) {
...
$data = $e->getData();
...
}
Dynamic Property (not recommended)
Please note that this will cause deprecation error in PHP 8.2 and will stop working in PHP 9 according to one of the PHP RFC https://wiki.php.net/rfc/deprecate_dynamic_properties
As the OP asking about doing this task without extending Exception class, you can totally skip ResponseException class declaration. I really not recommend do it this way, unless you've got really strong reason (see this topic for more details: https://softwareengineering.stackexchange.com/questions/186439/is-declaring-fields-on-classes-actually-harmful-in-php)
In throwing section:
...
$e = new Exception('Exception message');
$e->data = $customData; // we're creating object property on the fly
throw $e;
...
and when catching:
catch(Exception $e) {
$data = $e->data; // Access data property
}
September 2018 edit:
As some of readers found this answer useful, I have added a link to another Stack Overflow question which explains the downsides of using dynamically declared properties.
Currently, your code converts the response text directly into an object without any intermediate step. Instead, you could always just keep the serialized (via JSON) text it and append it to the end of the Exception message.
$responseText = curl_exec($ch);
$response = json_decode($responseText);
if (isset($response->error)) {
throw new Exception('Error when fetching resource. Response:'.$responseText);
}
return $response->result;
Then you could just recover everything after "Response:" in your error log and optionally de-serialize it or just read it.
As an aside, I would also not count on the server sending JSON, you should verify that the response text was actually parseable as JSON and return a separate error for that if it isn't.
I am writing a script to install the database of an application in php. It working fine but when im trying to install a database that doesnt exist i want only my own error message but i keep getting the default Warning : Warning: mysqli::mysqli() [mysqli.mysqli]: (HY000/2005): Unknown MySQL server host 'kasdasd'.
So I know that the host is wrong and I want it to be so, with only my own errormessage. How do I get rid of this message?
My connectclass with parameter DBConfig $config:
$this->mysqli = new mysqli($config->m_host,
$config->m_user,
$config->m_passw,
$config->m_db);
if ($this->mysqli->connect_error) {
return false;
}
$this->mysqli->set_charset("utf8");
return true;
an easy solution would be to see if you can open the hostname using fsockopen and suppressing the errors:
$port = 80;
if($fp = #fsockopen($config->m_host,$port)){
$db = new mysqli($config->m_host,$config->m_user,$config->m_passw,$config->m_db);
}else{
echo 'hostname not recognized';
}
#fclose($fp);
Edited:
You can use
if ($mysqli->connect_error) {
/** handle your error here **/
// Throw a custom exception if you like! (see below)
// or just echo "There was an error";
}
You can further use mysqli_connect_errno() to find out wht happened and handle it accordingly.
Edit: mysqli doesn't throw erros so below is incorrect.
Wrap your code in a try-catch, then you can throw whatever kind of error you like:
try {
/** your code here **/
} catch (Exception $e) {
/** your handling here **/
// i.e.
throw new BadHostNameException($config->m_host);
// or
echo "Could not connect!":
}
Note: you should replace Exception $e with the specific kind of exception a bad host throws so catch(MysqlBadHostnameException $e) (the type of error will be in your error log from your previous attempts or you can do get_class($e) in my example above.
// Isusing a custom exception: Add this outside of your class..
class BadHostNameException extends Exception {}
I'm extending my previous question (Handling exceptions within exception handle) to address my bad coding practice.
I'm trying to delegate autoload errors to a exception handler.
<?php
function __autoload($class_name) {
$file = $class_name.'.php';
try {
if (file_exists($file)) {
include $file;
}else{
throw new loadException("File $file is missing");
}
if(!class_exists($class_name,false)){
throw new loadException("Class $class_name missing in $file");
}
}catch(loadException $e){
header("HTTP/1.0 500 Internal Server Error");
$e->loadErrorPage('500');
exit;
}
return true;
}
class loadException extends Exception {
public function __toString()
{
return get_class($this) . " in {$this->file}({$this->line})".PHP_EOL
."'{$this->message}'".PHP_EOL
. "{$this->getTraceAsString()}";
}
public function loadErrorPage($code){
try {
$page = new pageClass();
echo $page->showPage($code);
}catch(Exception $e){
echo 'fatal error: ', $code;
}
}
}
$test = new testClass();
?>
the above script is supposed to load a 404 page if the testClass.php file is missing, and it works fine, UNLESS the pageClass.php file is missing as well, in which case I see a
"Fatal error: Class 'pageClass' not found in D:\xampp\htdocs\Test\PHP\errorhandle\index.php on line 29" instead of the "fatal error: 500" message
I do not want to add a try/catch block to each and every class autoload (object creation), so i tried this.
What is the proper way of handling this?
Have you tried checking for pageClass early on in the process, since it seems to be necessary even to get the error page out? If it doesn't exist, and if you don't want to write the 404 page w/o any objects (e.g. just HTML), bombing out of execution where that class doesn't exist would seem to be a good path.
Hope that helps.
Thanks,
Joe