I am using Maatwebsite\Excel to handle data exports in my application. Some of the exports are pretty big, so I want to queue the exports. I have followed the documentation, but I get the following error when attempting to export:
You cannot serialize or unserialize PDO instances
I understand that PDO instances cannot be serialized, I just don't understand why it's telling me that since I am following what is said in the docs. Here's my code:
Controller
$path = 'public/validations/' . $filename;
//the $client var is a record retrieved from a model, $input is the result of $request->all()
(new ValidationsExport($client, $input))->queue($path)->chain([
// the script never reaches here, but $user is from \Auth::user()
new NotifyUserOfValidationExport($user, $filename),
]);
Export
class ValidationsExport implements FromQuery, WithHeadings, WithMapping, WithStrictNullComparison, WithCustomQuerySize, WithEvents, WithColumnFormatting
{
use Exportable;
private $client;
private $input;
public function __construct($client, $input)
{
$this->client = $client;
$this->input = $input;
}
public function query()
{
// this does return a query builder object, but this is required for FromQuery and is shown in the docs as an example of how to queue an export
$this->validations = $this->getValidations();
return $this->validations;
}
public function querySize(): int
{
$query = ....
$size = $query->count();
return $size;
}
public function headings(): array
{
// this is an array
return $this->columns;
}
public function columnFormats(): array
{
return [
'A' => NumberFormat::FORMAT_TEXT,
'B' => NumberFormat::FORMAT_TEXT,
'C' => NumberFormat::FORMAT_TEXT
];
}
public function map($row): array
{
$mapping = [];
foreach($row as $key => $value) {
if(is_bool($value)) {
if($value) {
$mapping[$key] = "Yes";
} else {
$mapping[$key] = "No";
}
}else{
$mapping[$key] = $value;
}
}
return $mapping;
}
//.......
}
I assume that the problem comes from using FromQuery, but I can't use FromCollection because I run out of memory since the export is so big. I need the built in chunking that FromQuery uses. Is there a way I can queue an export using FromQuery?
You probably don't need to be setting a member variable $this->validations to that builder. That is what is ending up trying to be serialized. If you are just going to return it you don't need to store a copy on the class.
Related
I can't understand why the data is not transmitted in the custom facade (Format) method parameter. I'm doing exception handling for the API and using my visualization handler (JsonExceptionFormatter).
Interface CanFormat:
interface CanFormat
{
public function format($data);
}
Facade Format:
class Format extends Facade
{
protected static function getFacadeAccessor(): string
{
return \Hopex\VkSdk\Foundation\Format::class;
}
}
Class Format:
namespace Hopex\VkSdk\Facades;
class Format implements CanFormat
{
private array $formatters = [];
public function with(string $formatter): static
{
$formatter = new $formatter();
if ($formatter instanceof CanFormat) {
$this->formatters[] = $formatter;
}
return $this;
}
public function format($data): mixed
{
foreach ($this->formatters as $formatter) {
$data = $formatter->format($data);
}
return $data;
}
}
Exception render method:
final public function render(Request $request)
{
if (env('LOG_LEVEL') === 'debug') {
dump($this->getMessage()); // added for test (next "dump 1")
return new JsonResponse(
Format::with(JsonExceptionFormatter::class)->format($this->getMessage()),
$this->getCode()
);
}
}
Visualization handler:
class JsonExceptionFormatter implements CanFormat
{
public function format($data): array
{
dump($data); // added for test (next "dump 2")
return [
'type' => 'error',
'message' => $data instanceof SdkException ? $data->getMessage() : $data
];
}
}
It's dumps:
[dump 1]: "ApiException: User authorization failed"
[dump 2]: []
P.S. Other formats work without problems.
I'm making a Laravel package, which is a basic API Wrapper to practice. I want my code completely re-usable and neat, well that's the reason we learn OOP I think :P
Let me first attach my code, and I'll explain what I'm trying to achieve via comments.
// This is how I'm calling my class
Shiprocket::
withCredential('other-than-default') // this is optional
->order(203504661) // pass order id
->details() // finally fetch the details
// This is my main class it's behind a Larvel Facade Accessor
class Shiprocket
{
protected $credentials;
protected $token;
// I'm using it as a constructor to initilize with a different credentil pair.
public function withCredential($credential_id)
{
$this->credentials = config('shiprocket.credentials')[$credential_id];
$this->token = $this->getToken();
return $this;
}
public function __construct()
{
$this->credentials = config('shiprocket.credentials')[config('shiprocket.default_credentials')];
$this->token = $this->getToken();
}
public function order($order_id = null)
{
return new OrderResource($order_id);
// Here my doubt starts
// I want to return another class (OrderResource) for Order related methods
// so that we can call Order related methods like:
// Shiprocket::withCredential('my-credential')->order()->getAll()
// and those methods will also use methods & properties of this Main class
// like the token, get(), post()
}
public function shipment($shipment_id = null)
{
return new ShipmentResource($shipment_id);
// and maybe I can also have more child classes like OrderResource
// So that I can call similar methods as OrderResource for shipments like ... ->getAll()
// or ... ->status()
// but these methods won't be reusable - they'll be completely different, just sometimes
// might have same names.
}
public function getToken(): string
{
$duration = config('shiprocket.token_cache') ? config('shiprocket.token_cache_duration') : 0;
return cache()->remember("shiprocket-{$this->credentials['email']}", $duration, function () {
return Http::post("https://apiv2.shiprocket.in/v1/external/auth/login", [
'email' => $this->credentials['email'],
'password' => $this->credentials['password'],
])->json()['token'];
});
}
public function get($url, $data = null)
{
return Http::withToken($this->token)->get($url, $data)->json();
}
public function post($url, $data = null)
{
return Http::withToken($this->token)->post($url, $data)->json();
}
}
It's okay even if you don't attach any code, maybe just guide me a bit what would be the best way to achieve something like this.
The chain methods that you want to apply it's called the Builder pattern
Builder is a creational design pattern that lets you construct complex objects step by step. The pattern allows you to produce different types and representations of an object using the same construction code.
you can learn and find snippets from here https://refactoring.guru/design-patterns/builder
back to your case, I cant agree that we need the builder pattern here, but let's try to have the small steps with your code, let's say you want to build Shiprocket object that contains the Order and the Shipment
the simple change you need is to return the Shiprocket so the code should look like this
<?php
class Shiprocket
{
protected $credentials;
protected $token;
private $order;
private $shipment;
public function withCredential($credential_id)
{
$this->credentials = config('shiprocket.credentials')[$credential_id];
$this->token = $this->getToken();
$this->order = null;
$this->shipment = null;
return $this;
}
public function __construct()
{
$this->credentials = config('shiprocket.credentials')[config('shiprocket.default_credentials')];
$this->token = $this->getToken();
$this->order = null;
$this->shipment = null;
}
public function order($order_id = null)
{
$this->order = new OrderResource($order_id);
return $this;
}
public function shipment($shipment_id = null)
{
$this->shipment = new ShipmentResource($shipment_id);
return $this;
}
public function getOrder(){
return $this->order;
}
public function getShipment(){
return $this->shipment;
}
public function getToken(): string
{
$duration = config('shiprocket.token_cache') ? config('shiprocket.token_cache_duration') : 0;
return cache()->remember("shiprocket-{$this->credentials['email']}", $duration, function () {
return Http::post("https://apiv2.shiprocket.in/v1/external/auth/login", [
'email' => $this->credentials['email'],
'password' => $this->credentials['password'],
])->json()['token'];
});
}
public function get($url, $data = null)
{
return Http::withToken($this->token)->get($url, $data)->json();
}
public function post($url, $data = null)
{
return Http::withToken($this->token)->post($url, $data)->json();
}
}
Note: the code could not be perfect when it comes to the standard and the best practice I just change it to follow your idea
I hope it's helpful
I'm working on a project
and I made an exportable class
this is the class
I'm sending a query to exportal tyope of 'Illuminate\Database\Eloquent\Builder'
<?php
namespace App\Exports;
use Maatwebsite\Excel\Concerns\FromQuery;
use Maatwebsite\Excel\Concerns\Exportable;
use Maatwebsite\Excel\Concerns\WithMapping;
use Illuminate\Contracts\Queue\ShouldQueue;
use Maatwebsite\Excel\Concerns\WithHeadings;
class ExcelExport implements FromQuery, WithMapping, WithHeadings
{
use Exportable;
private $query;
public function __construct($query)
{
$this->query = $query;
}
public function query()
{
return $this->query;
}
public function headings():array
{
$return = [
'H1',
'H2',
];
return $return;
}
public function map($row):array
{
$return = [];
foreach ($row->relation as $rel) {
$return[] = $rel->column;
}
return $return;
}
}
and call it into my controller as
public function export(Request $request, string $type = 'excel')
{
// returns Builder
$query = $this->service->findByReportable($request, 1);
$file = new ExcelExportLead($query);
if ($file){
switch (strtolower($type)) {
case 'pdf':
$file_name = 'export-' . time() . '.pdf';
// OriginExcel refers to 'Maatwebsite\Excel\Excel'
return $file::queue($file, $file_name, OriginExcel::DOMPDF);
break;
default:
$file_name = 'export-' . time() . '.xls';
return $file->queue($file_name);
break;
}
} else{
return back()->withErrors(__('common.Sorry But there Was an issue in exporting Data please try again'));
}
But there the error appeared is 'Serialization of 'Doctrine\DBAL\Driver\PDOConnection' is not allowed'
i don't know how to solve it I've used SerializesModels but it didn't fix the issue
Serialization it's a tool where you encode a variable/object into a text (or binary) representation to then move that variable to another thread/process.
The underlying problem it's that the Builder object has a PDOConnection attribute (the connection to the database) and those objects by definition can't be serialized, as they are usually a file descriptor in the operating system that can't be moved to another process/thread.
The solution should go along the lines of either
Sending the query as a string to ExcelExport using the builder->toSql() method
Encoding the query in some other way (IE anourvalar/eloquent-serialize https://packagist.org/packages/anourvalar/eloquent-serialize).
I have a controller in laravel, AppExportController. In one of my functions on that controller, I iterate over many records and return a file download. I decided I wanted to create a little function so I could cache a certain thing, a Zone Name in this instance.
This was my first attempt at writing a function to cache the zone names (the getZoneName function obviously):
<?php
namespace App\Http\Controllers;
class AppExportController extends Controller {
/**
* Create a new controller instance.
*
* #return void
*/
public function __construct() {
$this->middleware('auth');
$this->middleware('client.approved');
}
public function prices(Request $request) {
$user = Auth::user();
...
$zoneNameCache = [];
function getZoneName($zoneId) use (&$zoneNameCache) {
try {
if (!empty($zoneNameCache[$zoneId])) {
return $zoneNameCache[$zoneId];
} else {
$zone = ServiceZone::find($zoneId);
$zoneNameCache[$zoneId] = $zone->name;
return $zone->name;
}
} catch(Exception $e) {
return '';
}
};
$prices = []; // I actually do a database query here, don't worry about that
$records = [];
foreach($prices as $price) {
// output to $records here
$records[] = [
...
getZoneName($price->service_zone_id),
...
];
}
return response();
}
}
This was making that route 500 error, and I tracked it down to being for sure the closure aspect of the function -- when I took out the use (&$zoneNameCache) part, it worked (but didn't cache anything of course).
So I tried another thing -- assigning the function to a variable instead. And that worked! With the closure, and caching was working!
<?php
namespace App\Http\Controllers;
class AppExportController extends Controller {
/**
* Create a new controller instance.
*
* #return void
*/
public function __construct() {
$this->middleware('auth');
$this->middleware('client.approved');
}
public function prices(Request $request) {
$user = Auth::user();
...
$zoneNameCache = [];
$getZoneName = function ($zoneId) use (&$zoneNameCache) {
try {
if (!empty($zoneNameCache[$zoneId])) {
return $zoneNameCache[$zoneId];
} else {
$zone = ServiceZone::find($zoneId);
$zoneNameCache[$zoneId] = $zone->name;
return $zone->name;
}
} catch(Exception $e) {
return '';
}
};
$prices = []; // I actually do a database query here, don't worry about that
$records = [];
foreach($prices as $price) {
// output to $records here
$records[] = [
...
$getZoneName($price->service_zone_id),
...
];
}
return response();
}
}
I don't know why the second one should work but not the first one. Can anyone shed light on this?
Without assigning it to a variable, or returning it, it is not a closure.
This way you have function declaration, within another function or method in this case.
Which is not allowed, and therefore will give you a 500 for sure.
If you check your php error_log and probably your laravel log. It will tell you that.
If your do not want to assign it to a variable at that point, you could return it immediately
return function().......
Inside my processor class I have a statement that grabs all the projects from a db table and formats them to be displayed. This method does not work and halts at the getCollection call.
class GlobalLinkSettingsProcessor extends modObjectGetListProcessor{
public function initialize() {
return parent::initialize();
}
public function process() {
$result = $this->modx->getCollection('ManagerProjects');
$project_names = array();
foreach ($result as $row) {
$projects = unserialize($row->get('manager_projects'));
foreach($projects as $short_code => $project) {
$project_names[] = array('project_name' => $project, 'project_short_code' => $short_code);
}
}
return '{"total":' . count($project_names) . ',"results":' . $this->modx->toJSON($project_names) . ',"success":true}';
}
...
}
This code that uses plain SQL does work:
class GlobalLinkSettingsProcessor extends modObjectGetListProcessor{
public function initialize() {
return parent::initialize();
}
public function process() {
$leadersql = "SELECT * FROM `modx_manager_projects`";
$query = $this->modx->query($leadersql);
$project_names = array();
while ($row = $query->fetch(PDO::FETCH_ASSOC)) {
$projects = unserialize($row['manager_projects']);
foreach($projects as $short_code => $project) {
$project_names[] = array('project_name' => $project, 'project_short_code' => $short_code);
}
};
return '{"total":' . count($project_names) . ',"results":' . $this->modx->toJSON($project_names) . ',"success":true}';
}
...
}
I use similar method to the first which saves ManagerProjects and works fine, so I don't think it has to do with the model declaration. I could easily just use the second method above since it seems to work, but I want to use the best method.
What is wrong with the first method?
Is the first method the proper way to implement SQL in the Modx processor? Or is there a better way?
We can do this task easier a little bit.
#Vasis is right but we can use base prepareRow method instead of reloading iterate method:
<?php
class GlobalLinkSettingsProcessor extends modObjectGetListProcessor{
public $classKey = 'ManagerProjects';
protected $projects = array();
public function prepareRow(xPDOObject $object) {
$_projects = unserialize($object->get('manager_projects'));
foreach($_projects as $short_code => $project) {
$this->projects[] = array('project_name' => $project, 'project_short_code' => $short_code);
}
return parent::prepareRow($object);
}
public function outputArray(array $array,$count = false) {
$count = count($this->projects);
return parent::outputArray($this->projects,$count);
}
}
return 'GlobalLinkSettingsProcessor';
There we can see one of modx ‘features’. In modObjectGetListProcessor process method we can see this:
public function process() {
$beforeQuery = $this->beforeQuery();
if ($beforeQuery !== true) {
return $this->failure($beforeQuery);
}
$data = $this->getData();
$list = $this->iterate($data);
return $this->outputArray($list,$data['total']);
}
getData method returns a list of objects and it goes to iterate method (where we can check if the object is accessible and change the list of these objects on demand). If you don't have access to some of objects we'll get changed list. And it goes to outputArray method but second parameter is still old for it. So you should count them again.
This is solution is quite well but you tried to get data which is stored in object's field. So afterIteration method will be unusable for further extension in my version of processor. But who cares? :)
P.S.: About your first version of processor. modObjectGetList processor is ready for getting collection. So you have not to use getcollection method. Just add proper classKey property to it.
Another way is in modProcessor extension. It gives to you a base structure. But you can make your own kind of stuff.
Because you do it wrong! Just see this. The right way to do it, is something like this:
<?php
class GlobalLinkSettingsProcessor extends modObjectGetListProcessor{
public $classKey = 'ManagerProjects';
public function iterate(array $data) {
$list = array();
$list = $this->beforeIteration($list);
$this->currentIndex = 0;
/** #var xPDOObject|modAccessibleObject $object */
foreach ($data['results'] as $object) {
if ($this->checkListPermission && $object instanceof modAccessibleObject && !$object->checkPolicy('list')) continue;
$projects = unserialize($object->get('manager_projects'));
foreach($projects as $short_code => $project) {
$objectArray = array('project_name' => $project, 'project_short_code' => $short_code);
if (!empty($objectArray) && is_array($objectArray)) {
$list[] = $objectArray;
$this->currentIndex++;
}
}
}
$list = $this->afterIteration($list);
return $list;
}
}