I am using Symfony 3.1 and I try to configure Monolog in such a way, that requests from the Googlebot are not logged. For this I wrote a UserAgentProcessorwhich already works as intended. In the next step I tried to write BotFilter which looks like this:
<?php
namespace AppBundle\Handler;
use Monolog\Handler\HandlerWrapper;
class FilterBotsHandler extends HandlerWrapper
{
/**
* {#inheritdoc}
*/
public function isHandling(array $record)
{
if (stripos($record['extra']['userAgent'], 'bot') !== false){
return false;
} else {
return $this->handler->isHandling($record);
}
}
}
This was inspired by the comments in the HandlerWrapper abstract class (take a look here).
Now I want to add that filter to my monolog yml-configuration. I tried adding it to my services but this was not possible as HandlerWrapper needs a Handler instance for its constructor. I researched how I could use the filter without a service but as far as I can see, the monolog bundle only accepts built-in types and the generic service type.
Now the question is: How can I use the filter in my configuration?
I am using Symfony 3.1 and I try to configure Monolog in such a way, that requests from the GoogleBot are not logged...
The quick way to prevent robots visiting your site is put these two lines into the /robots.txt file on your server. Create a robots.txt file in 'web' directory and paste the follows content:
User-agent: *
Disallow: /
https://support.google.com/webmasters/answer/6062608?hl=en&visit_id=1-636097099675465769-3677253464&rd=1
It's the recommended option when you need avoid access fully, meaning your sites will not longer be index by search engines and other bots. You don't need to configure/implement anything in your application to achieve it.
Now, if you need the bot to enter, but you don't want register it in logs. Instead of writing log files somewhere, some handlers are used to filter or modify log entries before sending them to other handlers. One powerful, built-in handler called fingers_crossed is used in the prod environment by default. It stores all log messages during a request but only passes them to a second handler if one of the messages reaches an action_level:
# app/config/config.yml
monolog:
handlers:
filter_for_errors:
type: fingers_crossed
# if *one* log is error or higher, pass *all* to file_log
action_level: error
handler: file_log
# now passed *all* logs, but only if one log is error or higher
file_log:
type: stream
path: "%kernel.logs_dir%/%kernel.environment%.log"
Thus, in your prod.log file just will register the messages/requests that contains some error, so the bots don't have effect in this level.
More details about this http://symfony.com/doc/current/logging.html
What you try to do is not advisable, because the handler will depend from http request instead of log records, which will be out of context, however you can register its own handler in Symfony easily:
Let's create the custom handler class:
namespace AppBundle\Monolog\Handler;
use Monolog\Handler\AbstractHandler;
class StopBotLogHandler extends AbstractHandler
{
public function isBotRequestDetected()
{
// here your code to detect Bot requests, return true or false
// something like this:
// return isset($_SERVER['HTTP_USER_AGENT']) && preg_match('/bot|crawl|slurp|spider/i', $_SERVER['HTTP_USER_AGENT']);
}
/**
* Checks whether the given record will be handled by this handler.
*
* This is mostly done for performance reasons, to avoid calling processors for nothing.
*
* Handlers should still check the record levels within handle(), returning false in isHandling()
* is no guarantee that handle() will not be called, and isHandling() might not be called
* for a given record.
*
* #param array $record Partial log record containing only a level key (e.g: array('level' => 100) for DEBUG level)
*
* #return bool
*/
public function isHandling(array $record)
{
return $this->isBotRequestDetected();
}
/**
* Handles a record.
*
* All records may be passed to this method, and the handler should discard
* those that it does not want to handle.
*
* The return value of this function controls the bubbling process of the handler stack.
* Unless the bubbling is interrupted (by returning true), the Logger class will keep on
* calling further handlers in the stack with a given log record.
*
* #param array $record The record to handle
*
* #return bool true means that this handler handled the record, and that bubbling is not permitted.
* false means the record was either not processed or that this handler allows bubbling.
*/
public function handle(array $record)
{
// do nothing, just returns true whether the request is detected as "bot", this will break the handlers loop.
// else returns false and other handler will handle the record.
return $this->isBotRequestDetected();
}
}
Whenever you add a record to the logger, it traverses the handler stack. Each handler decides whether it fully handled the record, and if so, the propagation of the record ends there.
Important: Read the phpdoc from isHandling() and handle() methods for more details.
Next, let's register the class as service "without tags":
# app/config/services.yml
services:
monolog.handler.stop_bot_log:
class: AppBundle\Monolog\Handler\StopBotLogHandler
public: false
Then, add its handler to handlers list:
# app/config/config_prod.yml
monolog:
handlers:
# ...
stopbotlog:
type: service
id: monolog.handler.stop_bot_log
priority: 1
Note the type property must be equal to service, id must be the service name before defined and priority must be greater than 0 to ensure that its handler will be executed before that any other handler.
When the GoogleBot performs a request to website application the stopbotlog handler stops all handlers after him and don't register any log message.
Remember it's not the recommended way to do that! According to your needs, implementing option 1 or 2 should be enough.
If you want ignore bot requests for handlers group, you can override the monolog.handler.group.class container parameter and override the group handler behavior:
namespace AppBundle\Handler;
use Monolog\Handler\GroupHandler;
class NoBotGroupHandler extends GroupHandler
{
public function isBotRequestDetected()
{
// here your code to detect Bot requests, return true or false
}
public function handle(array $record)
{
if ($this->isBotRequestDetected()) {
// ignore bot request for handlers list
return false === $this->bubble;
}
return parent::handle($record);
}
}
in your config_prod.yml or services.yml:
parameters:
monolog.handler.group.class: AppBundle\Handler\NoBotGroupHandler
That's it! Now, you can stop bot logs for custom handles list:
# config_prod.yml
monolog:
handlers:
grouped:
type: group
members: [main, console, chromephp]
Finally, if you have difficulty to analyze your logs files I recommend using this amazing tool: https://github.com/EasyCorp/easy-log-handler
It's quite a dirty trick, but if you really need it, you may make it like this.
Supposing you want to wrap a handler with a type stream:
Add a constructor in you FilterBotsHandler:
public function __constructor($path, $level, $bubble, $permissions) {
$this->handler = new Monolog\Handler\StreamHandler($path, $level, $bubble, $permissions);
}
And then redefine a parameter monolog.handler.stream.class:
parameters:
monolog.handler.stream.class: AppBundle\Handler\FilterBotsHandler
Make sure that this parameter will be defined after it was defined by MonologBundle.
That's it. Should work.
You may write CompilerPass in your AppBundle which adds configurator to monolog service. Such configurator can be also a request event listener which can replace all handlers dynamically on request and bot detection and push empty handlers array to Logger which can be hold on configurator call.
In other words configurator added to DI by CompilerPass and added to EventDispatcher as Listener to Kernel events which onRequest check User-Agent header looking for bot and then clears Monolog\Logger (passed in configurator) all handlers (or putting an NullHandler if empty handlers array fails).
DI configurator is only way to change your services during runtime which can be applied as service definition level. Such definition can be attached or detached if not needed and it doesn't really change anything in your application.
Related
I have a Listener, which behaves differently depending on the HTTP_USER_AGENT:
if ($request->server->get('HTTP_USER_AGENT') == $this->zabbixUserAgent) {
VisitorHolder::set($visitor);
} else {
VisitorHolder::set($this->visitorService->persist($visitor));
}
I want to avoid saving all Zabbix requests to our database. That works fine, but how can I fake the user agent in my unit test, so that my tests cover this case?
Creating a new Request and setting the user agent there is thoroughly ignored:
$this->currentRequest = new Request(
[], // GET parameters
[], // POST parameters
[], // request attributes (parameters parsed from the PATH_INFO, ...)
[], // COOKIE parameters
[], // FILES parameters
['HTTP_USER_AGENT' => 'zbx'], // SERVER parameters
null // raw body data
);
$this->requestStack
->expects($this->any())
->method('getCurrentRequest')
->willReturn($this->currentRequest);
A var_dump in the unit test tells me, that my user agent is still null and my case is not covered.
Any idea how I can set the user agent for this case?
If you extracted the actual check to a function elsewhere in the class, you can then mock or otherwise override that check within the class and keep it as a unit-test that does not need to fake a HTTP request at all.
For full integration tests, If you extracted the actual check to a separate service, then you can override the check with a difference configuration in a config_test.yml file, and using a different copy of the service that will always report false in a test-environment.
# config_test.yml file:
app_zabbix_detect.detector:
class: AppBundle\Services\ZabbixDetectorAlwaysFalse
In the main file it would be
# config.yml file: (or services.yml)
app_zabbix_detect.detector:
class: AppBundle\Services\ZabbixDetector # real test
I'm pretty new to Codeception and I have come across a problem I cannot figure out. I have about 40 tests in my test suite, and if a test fails, I need to send an email with the reason it failed. For example, if Codeception cannot find an element on the page resulting in a failed test, I need to send an email with just the error, like this:
Failed to verify emailing wish list behaves as expected in ThisClass::thisTest (/home/qauser/codeception_tests///acceptance-mobile/Wishlist/EmailWishlistCest.php)
Couldn't see "Success!","//*[#id="wish-list-confirm-popup"]/div/div/div[1]/h4":
I don't want to send the full stack trace, just the actual error. Does anyone know if this is possible?
Codeception exposes a useful collection of events that will come in handy for this use case. Take a look at the Customization: Events section of Codeception's documentation for more information.
I'd recommend intercepting two of the events described on that page:
The test.fail event, to aggregate information about each failed test.
The test.fail.print event, to process the aggregated data (eg. by sending a summary email) when Codeception has completed the test suite and prints its own summary of the failures to the screen.
To accomplish this, you simply need to build a custom event handler class and register it as an extension in the config file:
# codeception.yml
extensions:
enabled: [MyCustomEventHandler]
# MyCustomEventHandler.php
<?php
// Note: this was drafted using Codeception 2.0. Some of the namespaces
// maybe different if you're using a more-recent version of Codeception.
class MyCustomEventHandler extends \Codeception\Platform\Extension
{
/**
* #var \Exception[]
*/
protected $testFailures = [];
/**
* Maps Codeception events to method names in this class.
*
* Defining an event/method pair in this array essentially subscribes
* the method as a listener for its corresponding event.
*
* #var array
*/
public static $events = [
\Codeception\Events::TEST_FAIL => 'singleTestJustFailed',
\Codeception\Events::TEST_FAIL_PRINT => 'allTestFailuresAreBeingDisplayed',
];
/**
* This method will automatically be invoked by Codeception when a test fails.
*
* #param \Codeception\Event\FailEvent $event
*/
public function singleTestJustFailed(\Codeception\Event\FailEvent $event)
{
// Here we build a list of all the failures. They'll be consumed further downstream.
$this->testFailures[] = $event->getFail();
}
/**
* This method will automatically be invoked by Codeception when it displays
* a summary of all the test failures at the end of the test suite.
*/
public function allTestFailuresAreBeingDisplayed()
{
// Build the email.
$emailBody = '';
foreach ($this->testFailures as $failure) {
// Methods in scope include: $failure->getMessage(), $failure->getFile(), etc.
$emailBody .= $failure->getMessage() . "\r\n";
}
// Now send the email!
}
}
Hope this helps!
In monolog's fingers_crossed handler, the log records are held back until a certain level triggers to unload/show all those with-held records. So far, so good.
When those withheld messages are dumped after a fingers_crossed trigger, you may now actually want to know all the source lines. (And NOT before the fingers_crossed). For this the IntrospectionProcessor is used - but, as its creator specifies: 'Warning: This only works if the handler processes the logs directly. If you put the processor on a handler that is behind a FingersCrossedHandler for example, the processor will only be called once the trigger level is reached, and all the log records will have the same file/line/.. data from the call that triggered the FingersCrossedHandler.' - not good!.
This issue was raised for the 'Laravel 5' similar issue, without a satisfying answer. Here I present a working Symfony (2/3) solution.
src/AppBundle/IntrospectionPreprocessor.php:
<?php
namespace AppBundle;
class IntrospectionPreprocessor // ATTENTION TO THE 'PRE'
extends \Monolog\Processor\IntrospectionProcessor // SO USE THE ORIGINAL CLASS
{
public function processRecord(array $record)
{
return $this( $record ); // CALL TO ITS __INVOKE(..)
}
}
The services.logger.yml:
services:
monolog.processor.introspection_patch:
class: AppBundle\IntrospectionPreprocessor
arguments: [ DEBUG, [Helper, Google] ] # FILTER OUT CLASSES THAT CONTAIN [..], similar to IntrospectionProcessor
tags:
- { name: monolog.processor, method: processRecord }
This now inserts the correct file/line/class/function info, AFTER fingers_crossed triggers subsequent handlers.
Q. Is there a way to properly make this a part of my Symfony framework, so not in my application area (AppBundle), but neither touching the Symfony code (eg. gets lost with an upgrade?)
I want to implement a data log for attempts against my application. One of this will be when someone without the security rights wants to go to a certain page. For example a normal user trying to go to a url only avaiable for an administrator.
Symfony offers this security annotation:
/**
* #Security("has_role('ROLE_ADMIN')")
*/
And for now I use it to display an error page. But what I would like to do is to send the data to a database in case someone attempts to go in the admin only site recurrently (three or more times in less than a minute). The kind of data I will store is user, in case someone is logged in, IP, timestamp, among others. I already have a service that does the storing I just want to know if there is a way to know that someone is trying to access the page repeatedly without authorization and how to call my service in that case.
I have been looking all over the symfony documentation and couldn't find any information relevant to my problem. I would appreciate your help!
Thanks in advance.
SOLVED
I did what #ShiraNai7 told me to plus this in the service declaration in order to be able to use my other service. Thanks.
app.exception_listener:
class: InnoGames\Bundle\OfficeITBundle\EventListener\ExceptionListener
arguments: [#service_container]
tags:
- { name: kernel.event_listener, event: kernel.exception }
You could create a listner for the kernel.exception event and do your logging there.
use Symfony\Component\HttpKernel\Event\GetResponseForExceptionEvent;
use Symfony\Component\HttpFoundation\Response;
public function onKernelException(GetResponseForExceptionEvent $event)
{
$exception = $event->getException();
$request = $event->getRequest();
// do your logging here
}
Also see Symfony docs - How to Create Event Listeners and Subscribers
In my local environment I want all logs (all flags) to go the browser's console (BrowserConsoleHandler) and then to the default StreamHandler.
In production, I want the errors and other critical messages to go to an e-mail then stored in the database or (if fails) to a log file (default StreamHandler)
I want to set up this in a global middle-ware that I have created, which looks like this now:
<?php
namespace App\Http\Middleware;
use Closure;
use Illuminate\Support\Facades\App;
use Illuminate\Support\Facades\Log;
use Monolog\Logger;
use Monolog\Handler\BrowserConsoleHandler;
class GlobalConfig
{
/**
* Handle an incoming request.
*
* #param \Illuminate\Http\Request $request
* #param \Closure $next
* #return mixed
*/
public function handle($request, Closure $next)
{
// Get Monolog instance
$monolog = Log::getMonolog();
// In local environment the logs will be shown on browser
if (App::environment('local')) {
// Show logs in browser console
$monolog -> pushHandler(new BrowserConsoleHandler());
} else {
// Here we set up for production
}
Log::debug("Browser handler working");
return $next($request);
}
}
This doesn't work (the message is stored in the log file only, not shown on console). What I can't figure out is how to let the Log facade know about this new handler, because here, as is obvious, things are only changed within the function's scope. I know I can do it in bootstrap/app.php but isn't it to early to get the environment? Also, if I need to save logs to the database, it must already be connected, I guess
You can try to do next in bootstrap\app.php
if(env('APP_ENV') == 'local') {
$app->configureMonologUsing(function($monolog) use ($app) {
$monolog->pushHandler(
$handler = new \Monolog\Handler\RotatingFileHandler(
$app->storagePath().'/logs/laravel.log',
$app->make('config')->get('app.log_max_files', 30),
\Monolog\Logger::DEBUG
)
);
$handler->setFormatter(new \Monolog\Formatter\LineFormatter(null, null, true, true));
$monolog->pushHandler(new \Monolog\Handler\NativeMailerHandler(
'to#mail.com',
'Log::error!',
'from#mail.com'
));
});
}
Idea here is to fully control your logging on live/local. Bad news - configureMonologUsing replace all default laravel loggers, so you need to configure all your logs manually here.
Middleware isn't really the place for this, unless you wanted to actually make a particular log entry at some point in the request cycle.
Your call to pushHandler belongs in bootstrap/app.php (in Laravel 5.2), according to the docs.
I would think it would be possible to extract this logic out into a provider in case it becomes complicated and you need to move it out of bootstrap/app.php
solution here
BrowserConsoleHandler sends the script snipped after finishing the php
script by register_shutdown_function(). At this time, Laravel already
sent the full response to the browser. So the script snipped from
BrowseConsoleHandler gets generated but never sent to the browser.