Lumen changed log format from raw json after update - php

I've upgraded lumen from 5.8 to 9.1 and php from 7.x to 8.1
In my env we load json logs to kibana, but after update log format has changed to something like this:
2022-11-08 11:56:47 App\Domain\Email\Listeners\MoveEmailToProcessed{"message":"Marked email as processed","context":{"object_key":"incoming/0i02qrl9t4llea3fhfm2t6184r3nv9rs7nb4cu01","email":"wx1vx6fnek7rqhcr#lead-import.test.com"},"level":200,"level_name":"INFO","channel":"production","datetime":"2022-11-08T11:56:48.006053+00:00","extra":{}}
447.64ms DONE
while it was and should stay raw JSON like this:
{"message":"Marked email as processed","context":{"object_key":"incoming/0i02qrl9t4llea3fhfm2t6184r3nv9rs7nb4cu01","email":"wx1vx6fnek7rqhcr#lead-import.test.com"},"level":200,"level_name":"INFO","channel":"production","datetime":"2022-11-08T11:56:48.006053+00:00","extra":{}}
I have config for this as different channel in lumen using monolog:
'stdout-json' => [
'driver' => 'monolog',
'handler' => StreamHandler::class,
'with' => [
'stream' => 'php://stdout',
],
'formatter' => Monolog\Formatter\JsonFormatter::class,
],
And since format has changed it don't end up in kibana as expected. Question is what am I missing? Config didn't changed and I didn't saw anything specific in any upgrade guide I was following. Any idea what might be wrong?
Oh, and I don't use any lumen 'shortcuts' so logger is injected as it should. So code looks like this i.e.
public function __construct(Cloud $s3, LoggerInterface $logger)
{
$this->s3 = $s3;
$this->logger = $logger;
}
public function handle(EmailProcessed $emailProcessed): void
{
$filename = explode('/', $emailProcessed->getObjectKey())[1];
$newPath = sprintf('%s/%s', self::PROCESSED_DIR, $filename);
if ($this->s3->exists($newPath)) {
$this->s3->delete($newPath);
}
$this->s3->move($emailProcessed->getObjectKey(), $newPath);
$this->logger->info(
'Marked email as processed',
[
'object_key' => $emailProcessed->getObjectKey(),
'email' => $emailProcessed->getEmail(),
]
);
}

Related

How can I rename a webdav collection using Symfony Client?

I am using an http Client with Symfony. In this example I am creating a folder.
$client = HttpClient::create();
/* #var $response ResponseInterface */
$createFolder = $client->request('MKCOL', $filePath, [
'auth_basic' => [$user, $authKey],
]);
This works well.
Now I want to rename a folder:
$renameFolder = $client->request('MOVE', $filePath, [
'auth_basic' => [$user, $authKey],
]);
But what I cannot figure out is, where do I define the new folder name?
You seem to be accessing a Webdav server, from the MKCOL command.
If you want to use MOVE, the destination should be passed on a Destination header, as shown here.
With Symfony Http Client, the request would be something like:
$moveCollection = $client->request('MOVE', $collectionPath,
[
'auth_basic' => [$user, $authKey],
'headers' => [
'Destination' => $newCollectionPath
]
]
);

Slim v4 Creating Log File

So, my issue is that I'm having trouble to make Slim record all its actions inside a file (for example : app.log). I ran across a lot of tutorials and other forum similar to this one but the issue was that they were using the v3 of the Slim Framework.
I saw some post suggesting things like this inside a settings.php :
return [
'settings' => [
'displayErrorDetails' => true, // set to false in production
'addContentLengthHeader' => false, // Allow the web server to send the content-length header
// Renderer settings
'renderer' => [
'template_path' => __DIR__ . '/../templates/',
],
// Monolog settings
'logger' => [
'name' => 'my-app',
'path' => __DIR__ . '/../logs/' . $logDate->format('Y-m-d') . 'app.log',
],
],
];
But the issue with that method is that, well, settings aren't set up this way anymore in v4. So here I am. Stuck. If anybody could give me a hand, it'll help a lot !
To load the settings within a container you have to add a container definition for it.
Example settings file: config/settings.php
return [
// set to false in production
'displayErrorDetails' => true,
// Renderer settings
'renderer' => [
'template_path' => __DIR__ . '/../templates/',
],
// Monolog settings
'logger' => [
'name' => 'my-app',
'path' => __DIR__ . '/../logs/' . date('Y-m-d') . '_app.log',
],
];
Example container entry in config/container.php:
use Psr\Container\ContainerInterface;
// ...
return [
// ...
'settings' => function () {
return require __DIR__ . '/settings.php';
},
// Add more entries here ...
}
To fetch the settings within the container use this:
use Psr\Container\ContainerInterface;
use Psr\Log\LoggerInterface;
// ...
return [
// ...
LoggerInterface::class => function (ContainerInterface $container) {
$settings = $container->get('settings');
$name = $settings['logger']['name'];
$logger = new Logger($name);
// Add logger handler...
return $logger;
},
// ...
}
Tip: For autowiring support it's better to use an collection object instead of an simple "string" as container identifier. Read more

How to test upload file in laravel 5.0?

I want to test my file upload action .
I checked file type validation on that action . It is working from browser interaction , but I tried with phpunit it fail on validation .
I validate for accepting jpg or jpeg file only .
Here is my unit code
public function testWIthJpgImage()
{
$_FILES = [
'front_image' => [
'name' => 'uitest',
'type' => 'image/jpeg',
'size'=> filesize(storage_path('data/uitest.jpg')),
'error' => 0,
'tmp_name' => storage_path('data/uitest.jpg')
]
];
$response = $this->call('POST', '/api/test',[],[],$_FILES);
$data = json_decode($response->getContent());
dd($data); // here validation fail always
$this->assertEquals(200, $response->getStatusCode());
}
How can I test with file upload correctly on phpunit ? My laravel version 5.0 (old version) Laravel UploadFile is not available
You can use Illuminate\Http\UploadedFile Facade for this.
Here is an example of one of my projects where I upload & store a file. It also has a files table & model.
/** #test */
function create_new_file ()
{
$payload = [
'file' => UploadedFile::fake()->image('avatar.jpg')
];
// act
$response = $this->post("/api/files", $payload);
// test
$response->assertStatus(201);
Storage::disk('local')->assertExists(File::first()->path);
}
Don't forget
use Illuminate\Http\UploadedFile;
use Illuminate\Support\Facades\Storage;
In addition you can use
Storage::fake('local');
in order to not persist the uploaded files (if you are using the Storage as I do). This is very handy so your tests don't create a bunch of files in your uploads folder.
I think my problem is sovled but it is so weired to do that :) I set $_FILES and Symfony UploadedFile both :<
$_FILES = [
'front_image' => [
'name' => 'uitest',
'type' => 'image/jpeg',
'size'=> filesize(storage_path('data/uitest.jpg')),
'error' => 0,
'tmp_name' => storage_path('data/uitest.jpg'),
'test' => true
]
];
$uploadedFile = new Symfony\Component\HttpFoundation\File\UploadedFile(
storage_path('data/uitest.jpg'),
'uitest.jpg',
'image/jpeg',
filesize(storage_path('data/uitest.jpg')),
0,
true
);
$response = $this->call('POST', '/api/test',[],[],['front_image' => $uploadedFile]);

Add Metadata, headers (Expires, CacheControl) to a file uploaded to Amazon S3 using the Laravel 5.0 Storage facade

I am trying to find out how to add in Metadata or headers (Expires, CacheControl etc.) to a file uploaded using the Laravel 5.0 Storage facade. I have use the page here as reference.
http://laravel.com/docs/5.0/filesystem
The following code works correctly:
Storage::disk('s3')->put('/test.txt', 'test');
After digging I also found that there is a 'visibility' parameter which sets the ACL to 'public-read' so the following also works correctly.
Storage::disk('s3')->put('/test.txt', 'test', 'public');
But I would like to be able to set some other values to the header of the file. I have tried the following:
Storage::disk('s3')->put('/index4.txt', 'test', 'public', array('Expires'=>'Expires, Fri, 30 Oct 1998 14:19:41 GMT'));
Which doesn't work, I have also tried:
Storage::disk('s3')->put('/index4.txt', 'test', array('ACL'=>'public-read'));
But that creates an error where the 'visibility' parameter can not be converted from a string to an array. I have checked the source of AwsS3Adapter and it seems there is code for options but I can not seem to see how to pass them correctly. I think it takes the following:
protected static $metaOptions = [
'CacheControl',
'Expires',
'StorageClass',
'ServerSideEncryption',
'Metadata',
'ACL',
'ContentType',
'ContentDisposition',
'ContentLanguage',
'ContentEncoding',
];
Any help on how to accomplish this would be appreciated.
First, you need to call getDriver so you can send over an array of options. And then you need to send the options as an array.
So for your example:
Storage::disk('s3')->getDriver()->put('/index4.txt', 'test', [ 'visibility' => 'public', 'Expires' => 'Expires, Fri, 30 Oct 1998 14:19:41 GMT']);
Be aware that if you're setting Cache-Control it has to be passed as CacheControl. This may well be true for other keys with non-alphanumierc characters.
If you want to have global defaults with headers, this works in Laravel 5.4. Change your config/filesystems.php file like so:
s3' => [
'driver' => 's3',
'key' => env('AWS_KEY'),
'secret' => env('AWS_SECRET'),
'region' => env('AWS_REGION'),
'bucket' => env('AWS_BUCKET'),
'options' => ['CacheControl' => 'max-age=315360000, no-transform, public',
'ContentEncoding' => 'gzip']
],
After attempting the above answers and failing to be able to add customer user-metadata it turns out that after digging through the SDK code it is a bit easier than I thought (Assume $path is a path to an image file). I didn't appear to need to call the getDriver() method either, not too sure if that makes any difference with the current version of the AWS SDK.
Storage::put(
'image.jpg',
file_get_contents($path),
[
'visibility' => 'public',
'Metadata' => [
'thumb' => '320-180',
],
]
);
So now if you view the newly uploaded file in S3 you will see the custom metadata:
Hope this helps someone.
The answer from #Paras is good. But there is one thing that can confuse newcommers:
'options' => [
'Expires' => gmdate('D, d M Y H:i:s GMT', strtotime('+1 month')),
>>> WRONG visibility' => 'public', WRONG <<<
]
If you want to define global options for the HEADERS, the options array is the right way to go. But if you also want to define the visibility, you can not mix it up. Visibility has to be defined outside of options array.
👍
'visibility' => 'public',
'options' => ['Expires' => gmdate('D, d M Y H:i:s GMT', strtotime('+1 month'))]
This is an example of how to upload a file to S3 as of Laravel 5.8 with expiry and cache control headers, for example:
Storage::put($directory . '/' . $imageName,
$image, [
'visibility' => 'public',
'Expires' => gmdate('D, d M Y H:i:s \G\M\T', time() + (60 * 60 * 24 * 7)),
'CacheControl' => 'max-age=315360000, no-transform, public',
]);
Also don't forget to uncheck the 'Disable cache' checkbox in Chrome if you're testing and it never seems to work, that got me bad for an hour when my browser wouldn't cache things even though I finally got the headers right in S3.
For Laravel 9 users this has became more easy. You do not need to call ->getDriver() anymore. You can directly pass options to the put command.
Storage::disk('s3')->put('/index.txt', 'file content', [
// S3 Object ACL
'visibility' => 'public', // or 'private',
// HTTP Headers
'CacheControl' => 'public,max-age=315360000',
'ContentDisposition' => 'attachment; filename="index.txt"',
'Expires' => 'Thu, 12 Feb 2032 08:24:43 GMT',
// Metadata or other S3 options
'MetadataDirective' => 'REPLACE'
'Metadata' => [
'Custom-Key' => 'test',
],
])
In case you need other headers or options, please checkout the flysystem source code for all available headers and options.
https://github.com/thephpleague/flysystem-aws-s3-v3/blob/master/src/AwsS3Adapter.php#L38
public const AVAILABLE_OPTIONS = [
'ACL',
'CacheControl',
'ContentDisposition',
'ContentEncoding',
'ContentLength',
'ContentType',
'Expires',
'GrantFullControl',
'GrantRead',
'GrantReadACP',
'GrantWriteACP',
'Metadata',
'MetadataDirective',
'RequestPayer',
'SSECustomerAlgorithm',
'SSECustomerKey',
'SSECustomerKeyMD5',
'SSEKMSKeyId',
'ServerSideEncryption',
'StorageClass',
'Tagging',
'WebsiteRedirectLocation',
];
Hey I solved this problem, you need to create a custom S3 filesystem
First, create a new file CustomS3Filesystem.php and save into app/providers, this custom S3 filesystem uses the S3 Adapter, but you can add metadata and headers.
<?php namespace App\Providers;
use Storage;
use League\Flysystem\Filesystem;
use Aws\S3\S3Client;
use League\Flysystem\AwsS3v2\AwsS3Adapter as S3Adapter;
use Illuminate\Support\ServiceProvider;
class CustomS3Filesystem extends ServiceProvider {
public function boot()
{
Storage::extend('s3_custom', function($app, $config)
{
$s3Config = array_only($config, ['key', 'region', 'secret', 'signature', 'base_url']);
$flysystemConfig = ['mimetype' => 'text/xml'];
$metadata['cache_control']='max-age=0, no-cache, no-store, must-revalidate';
return new Filesystem(new S3Adapter(S3Client::factory($s3Config), $config['bucket'], null, ['mimetype' => 'text/xml', 'Metadata' => $metadata]), $flysystemConfig);
});
}
public function register()
{
//
}
}
Add provider into providers list at config/app.php
'App\Providers\CustomS3Filesystem',
create new filesistem name in config/filesystems
's3-new' => [
'driver' => 's3_custom',
'key' => 'XXX',
'secret' => 'XXX',
'bucket' => 'XXX',
],
Use the new created custom s3 adapter
Storage::disk('s3-new')->put(filename, file_get_contents($file), public);
I used laravel documentation to customize the s3 adapter
http://laravel.com/docs/5.0/filesystem#custom-filesystems
I hope this may help you.
I am using Laravel 4.2, but I think my solution might also help on Laravel 5.0 (cannot say for sure, as I have not tried to upgrade yet). You need to update the meta options in the config for the Flysystem driver that you are using. In my case, I created a connection called s3static to access the bucket where I am storing images that will not be changing.
My config file:
's3static' => [
'driver' => 'awss3',
'key' => 'my-key',
'secret' => 'my-secret',
'bucket' => 'my-bucket',
// 'region' => 'your-region',
// 'base_url' => 'your-url',
'options' => array(
'CacheControl' => 'max_age=2592000'
),
// 'prefix' => 'your-prefix',
// 'visibility' => 'public',
// 'eventable' => true,
// 'cache' => 'foo'
],
Now when I put any files on to S3 using this connection, they have the Cache-Control meta data set.
To expand on #sergiodebcn 's answer, here is the same CustomS3Filesystem class working for S3 v3 and the latest Laravel. Note I have removed the XML mimetype and set up a 5 day cache time:
namespace App\Providers;
use Illuminate\Support\Arr;
use Storage;
use League\Flysystem\Filesystem;
use Aws\S3\S3Client;
use League\Flysystem\AwsS3v3\AwsS3Adapter as S3Adapter;
use Illuminate\Support\ServiceProvider;
class CustomS3Filesystem extends ServiceProvider
{
/**
* Format the given S3 configuration with the default options.
*
* #param array $config
* #return array
*/
protected function formatS3Config(array $config)
{
$config += ['version' => 'latest'];
if ($config['key'] && $config['secret']) {
$config['credentials'] = Arr::only($config, ['key', 'secret']);
}
return $config;
}
/**
* Bootstrap a custom filesystem
*
* #return void
*/
public function boot()
{
Storage::extend('s3_custom', function($app, $config)
{
$s3Config = $this->formatS3Config($config);
return new Filesystem(
new S3Adapter(
new S3Client($s3Config),
$config['bucket'],
null,
[
'CacheControl' => 'max-age=432000'
]
)
);
});
}
public function register()
{
//
}
}
Using Laravel 8 here:
I didn't see this mentioned elsewhere, but the metadata option key => values listed by Christoph Kluge
appear to only accept string values, and fail silently if passed an integer, bool, etc... So if you're passing in a variable you'll need to convert to a string value:
$fileID = $fileData['FileId'];
$fileExt = $fileData['FileExtension'];
$fileUnique = $fileData['UniqueFileId'];
$isImage = $fileData['IsImage'];
$isDefault = $fileData['IsDefaultImage'];
$filePath = $fileUnique . "." . $fileExt;
$file = $mp->fileID($fileID)->get();
if (Storage::disk('s3')->missing('img/' . $filePath)) {
Storage::disk('s3')->put(
'img/' . $filePath,
$file,
[
// Metadata or other S3 options
'MetadataDirective' => 'REPLACE',
'Metadata' => [
'is-image' => strval($isImage),
'is-default' => strval($isDefault),
'unique-file-id' => strval($fileUnique),
'file-extension' => strval($fileExt),
]
]
);
echo nl2br('uploading file: ' . $filePath . "\n");
} else {
echo nl2br('file already exists:' . $filePath . "\n");
}

Test an upload with PHPUnit in Laravel

Currently, i try to test a file upload with PHPUnit for Laravel. After some searches, i found a first solution :
$file = new \Symfony\Component\HttpFoundation\File\UploadedFile(
app_path() . '/tests/Files/default.png', 'default.png');
$response = $this->call('POST', 'students',
[
'firstname' => 'firstname',
'lastname' => 'lastname',
'promotion' => '2016',
'isCoop' => 1
],
[
'avatar' => [$file]
]
);
It works, but it failed when I push this code on Travis, and i'm not sure it's very clean to do that...
You can see my test failed here.
Thank's you.
You should use virtual file system for your tests. Check mocking file system in the phpunit documentation
You can use this class \Illuminate\Http\UploadedFile to test your upload file as you can see on this answer
mine working using test case like this :
public function testUploadLanscapseValid()
{
$uploadFile= new \Illuminate\Http\UploadedFile(
base_path()."/resources/fakerFile/mobileScreen/bg_land.jpg", //path image
'example.jpg',
'image/jpeg',
filesize(base_path()."/resources/fakerFile/mobileScreen/bg_land.jpg"), // file size
null,
true
);
//checking UI validation response
$this->visit('/mobile-screen')
->attach($uploadFile, 'image-landscape')
->press('upload-image-landscapse')
->seePageIs('/mobile-screen')
->see('Mobile screen successfully uploaded.');
//checking database is inserted
$this->seeInDatabase('mobile_screen',['link_lanscapse'=>'bg_land.jpg']);
//checking file exist
if(file_exists(base_path() . '/public/mobileScreen/bg_land.jpg')){
$this->assertTrue(true);
}else{
$this->assertTrue(false);
}
}
when you are testing upload using file please use \Illuminate\Http\UploadedFile instead of \Symfony\Component\HttpFoundation\File\UploadedFile
UPDATE
when seeing your test I think it is incomplete you can get the response of your test like mine here :
public function testUploadFunctionalTest()
{
$uploadedFile= new \Illuminate\Http\UploadedFile(
base_path()."/resources/fakerFile/mobileScreen/bg_land.jpg", //path image
'example.jpg',
'image/jpeg',
filesize(base_path()."/resources/fakerFile/mobileScreen/bg_land.jpg"), // file size
null,
true
);
// you can use this or
$parameters = [];
$response = $this->action(
'POST',
'AdminWeb\MobileScreenController#store',
[],
$parameters,
[],
['image' => $uploadedFile]
);
// you can use this choose One !
$response =$this->call('POST', 'mobile-screen#store',
[
'firstname' => 'firstname',
'lastname' => 'lastname',
'promotion' => '2016',
'isCoop' => 1
],
[
'image' => [$uploadedFile]
]);
$result = json_decode($response->content()); // you can dump or whatever you want with the test
var_dump($result->your_response);
$this->assertEquals($result->your_response,'expected output');
}
Note : I'm using laravel 5.2, and this test will write the file on your disk
$file = new \Symfony\Component\HttpFoundation\File\UploadedFile(
app_path() . '/tests/Files/default.png', 'default.png', "image/jpeg", null, null, true);

Categories