How to make File Upload with Laravel excel using chunk method? - php

public function uploadNotaCorte(Request $request, EstadoRepository $estadoRepository)
{
$error = array();
$path = $request->file('file')->getRealPath();
$notasCorte = Excel::load($path, function($reader) {
})->get();
$chunk = $notasCorte->chunk(100);
foreach ($notasCorte as $key => $notaCorte) {
//RULES
}return $error;
}
**
Hi everyone, I'm new to programming and I'm having a hard time implementing the chunk method, so the dodigo above usually works on small files plus larger error files because of the size.
I need to upload a file with 120,000 records and I am trying to use the chunk for this, I do not know what I can do wrong already looked at the documentation more and very simple and I could not solve the problem can anyone help me ??**

Assuming you're using the maatwebsite/excel package, this link should help: http://www.maatwebsite.nl/laravel-excel/docs/import#chunk
You'll want to change your code to something like this:
public function uploadNotaCorte(Request $request, EstadoRepository $estadoRepository)
{
$error = array();
$path = $request->file('file')->getRealPath();
Excel::filter('chunk')->load($path)->chunk(100, function($results)
{
foreach($results as $row)
{
// RULES
}
});
return $error;
}
This isn't tested and I've never used that package (though good to know it exists) so your mileage may vary.

Related

Search files with prefix on S3 in Laravel 9

I'm working on a project that save file on S3. But I've never worked with S3 before.
I want to retrieve the files that match this pattern: {id}_{YYYYMMDD}.pdf
I could do this Storage::disk('s3')->files(); with Storage, but I think it's not the solution because there are thousand of files.
I search through topics and this is one of the things I've tried so far:
public static function searchS3ByPrefix($path, $prefix) {
try {
$storage = Storage::disk('s3');
$client = $storage->getAdapter()->getClient(); // ** error on this line
$command = $client->getCommand('ListObjects');
$command['Bucket'] = $storage->getAdapter()->getBucket();
$command['Prefix'] = $path . $prefix;
$result = $client->execute($command);
return array_column($result['Contents'], 'Key');
}
catch (\Exception $e) {
Log::error($e);
return [];
}
}
The error message said that getClient() is undefined in League\Flysystem\AwsS3V3\AwsS3V3Adapter
Do you have a solution for this? Thank you very much
On the line where I commented error, just modify it into this:
$client = $storage->getClient();
Note: In my case, the result will be empty if the prefix has slash / at the very beginning, so make sure to remove it. For example: '/data/1_' will not work, change it to 'data/1_'

Elasticsearch doesn't index something in a php application

I've a problem using elasticsearch in a php application. The application is built with zend and uses a .env to hold the following configuration:
ELASTICSEARCH_MAX_DOCUMENTS=250
ELASTICSEARCH_MAX_BULK_SIZE=3M
ELASTICSEARCH_HOST=my-elasticsearch.intern.rz
ELASTICSEARCH_PORT=80
ELASTICSEARCH_USER=user
ELASTICSEARCH_PASSWORD=pw
The call to index the new files is part of a import service class and looks like this: 
public function flush(FlushInterface $flushInterface = null) {
$bulk = $this->getSearchDocumentBulk();
if (!empty($bulk->getActions())) {
$response = $bulk->send();
$this->resetSearchDocumentBulk();
if (0 === $response->count()) {
$data = $response->getData();
throw new BulkException(isset($data['message']) ? strip_tags($data['message']) : '');
}
}
$this->documentCache = [];
if ($flushInterface instanceof FlushInterface) {
$flushInterface->flush();
}
return $this;
}
protected function getSearchDocumentBulk() {
if (!($this->searchDocumentBulk instanceof Bulk)) {
$this->searchDocumentBulk = new Bulk($this->getSearchClient()->getClient());
$this->searchDocumentBulk->setIndex(self::INDEX);
}
return $this->searchDocumentBulk;
}
I know it's only a short snippet but it's quite difficult to pull out the relevant code. So please let me know if I have to post some more methods. 
The application is started by a symfony command and I'm able to curl to elasticsearch (version 5.1) without any errors.
My problem is that no document is indexed. If I check elasticsearch-head I see that there was no data transfer anymore. 
But there's also no error, no exception or something like that. The process is completed with 100% (100,000 of 100,000 files imported). So I've no idea what happens or how to find out the bug.

PHP include external method and class

I'm new to PHP and I have an issue I can't seem to fix or find a solution to.
I'm trying to create a helper function that will return an 'object' filled with information pulled from an XML file. This helper function, named functions.php contains a getter method which returns a 'class' object filled with data from an SVN log.xml file.
Whenever I try to import this file using include 'functions.php'; none of the code after that line runs the calling function's page is blank.
What am I doing wrong?
Here is what the functions.php helper method and class declaration looks like:
<?php
$list_xml=simplexml_load_file("svn_list.xml");
$log_xml=simplexml_load_file("svn_log.xml");
class Entry{
var $revision;
var $date;
}
function getEntry($date){
$ret = new Entry;
foreach ($log_xml->logentry as $logentry){
if ($logentry->date == $date){
$ret->date = $logentry->date;
$ret->author = $logentry->author;
}
}
return $ret;
}
I'm not sure what the point of having a separate helper function from the class is, personally I'd combine the two. Something like this
other-file.php
require './Entry.php';
$oLogEntry = Entry::create($date, 'svn_log.xml');
echo $oLogEntry->date;
echo $oLogEntry->revision;
Entry.php
class Entry
{
public $revision;
public $date;
public $author;
public static function create($date, $file) {
$ret = new Entry;
$xml = simplexml_load_file($file);
foreach($xml->logentry as $logentry) {
if($logentry->date == $date) {
$ret->date = $logentry->date;
$ret->author = $logentry->author;
$ret->revision = $logentry->revision;
}
}
return $ret;
}
}
EDIT
In light of the fact OP is new to PHP, I'll revise my suggestion completely. How about ditching the class altogether here? There's hardly any reason to use a class I can see at this point; let's take a look at using an array instead.
I might still move the simplexml_load_file into the helper function though. Would need to see other operations to merit keeping it broken out.
entry-helper.php
function getEntry($date, $file) {
$log_xml = simplexml_load_file($file);
$entry = array();
foreach($log_xml->logentry as $logentry) {
if($logentry->date == $date) {
$entry['date'] = $logentry->date;
$entry['author'] = $logentry->author;
$entry['revision'] = $logentry->revision;
}
}
return $entry;
}
other-file.php
require './entry.php';
$aLogEntry = Entry::create($date, 'svn_log.xml');
echo $aLogEntry['date'];
echo $aLogEntry['revision'];
EDIT
One final thought.. Since you're seemingly searching for a point of interest in the log, then copying out portions of that node, why not just search for the match and return that node? Here's what I mean (a return of false indicates there was no log from that date)
function getEntry($date, $file) {
$log_xml = simplexml_load_file($file);
foreach($log_xml->logentry as $logentry) {
if($logentry->date == $date) {
return $logentry;
return false;
}
Also, what happens if you have multiple log entries from the same date? This will only return a single entry for a given date.
I would suggest using XPATH. There you can throw a single, concise XPATH expression at this log XML and get back an array of objects for all the entries from a given date. What you're working on is a good starting point, but once you have the basics, I'd move to XPATH for a clean final solution.

PHP APC problems when storing objects

I am trying to build an configuration parser for my application I installed APC today, but everytime I try to put an serialized object in the store, it does not get in there and does not. (I am checking with apc.php for my version[3.1.8-dev] on PHP 5.3.16 [My Dev Environment], so I am sure that the data is not in the cache). this is how I pass the data to the cacher:
// The data before the caching
array (
'key' => md5($this->filename),
'value' => serialize($this->cfg)
);
// The caching interface
function($argc){
$key = $argc['key'];
Cache\APC::getInstance()->set($key,$argc['value']);
}
// The caching method described above
public function set($key, $val) {
if (apc_exists($key)) {
apc_delete ($key);
return apc_store($key, $val);
}
else
return false;
}
// the constructor of the configuration class.
// It 1st looks for the configuration in
// the cache if it is not present performs the reading from the file.
public function __construct($filename = '/application/config/application.ini',
$type = self::CONFIG_INI)
{
if (defined('SYSTEM_CACHE') && SYSTEM_CACHE === 'APC'){
$key = md5($filename);
$cfg = APC::getInstance()->get($key);
if (!empty($cfg)) {
print "From Cache";
$this->cfg = unserialize($cfg);
return;
} else {
print "From File";
}
}
}
I did a few tests and there is not a problem with the MD5() key (which I thought while writing this question) nor with APC itself. I am really stuck on this one, nothing odd in the logs, so if anyone can give me at least some directions will be very appreciated.
Thanks in advance!
The problem is was in my code:\
public function set($key, $val) {
/*
*
* If the key exists in the cache delete it and store it again,
* but how could it exist when the else clause is like that...
*/
if (apc_exists($key)) {
apc_delete ($key);
return apc_store($key, $val);
}
// This is very wrong in the current case
// cuz the function will never store and the if will
// never match..
else
return false;
}
NOTE:
Always think and keep your eyes open, if you still can't find anything get off the PC and give yourself a rest. Get back after 10-15 minutes and pown the code. It helps! :D

How do I pass $_FILE and $_POST data into a form?

So far this is what I know to be the way to pass the data from a post into the form.
$form->setData( $this->getRequest()->getPost() );
I thought that this might work
$form
->setData( $this->getRequest()->getPost() )
->setData( $this->getRequest()->getFiles() );
Which it does not. Looking through the framework source I confirmed that it shouldn't. So I was thinking about merging the file data into post data. Surely this cannot be the desired solution? It's not as if getPost() and getFiles() return easily mergeable arrays, they return Parameter objects.
Please note this is Zend Framework 2 specific.
Have you tried getFileInfo knowing now or paying mind to the fact that your using Zend. Typically on a per file basis $_FILE is an array based on the information of the file being uploaded. Filename, extension, etc.. Zends getFileInfo outputs that information in a similar fashion. Though I haven't played with it in sometime, its worth looking into
Example concept (more for multiple file uploads I know, but works with one, good concept to leave in tact just incase you wanna add a second or more files down the road)
$uploads = new Zend_File_Transfer_Adapter_Http();
$files = $uploads->getFileInfo();
foreach($files as $file => $fileInfo) {
if ($uploads->isUploaded($file)) {
if ($uploads->isValid($file)) {
if ($uploads->receive($file)) {
$info = $uploads->getFileInfo($file);
$tmp = $info[$file]['tmp_name'];
$data = file_get_contents($tmp);
// here $tmp is the location of the uploaded file on the server
// var_dump($info); to see all the fields you can use
}
}
}
}
After attempting to use Zend's file transfer adapter I went with a workaround in the controller. I think that the setData() in the form class should merge the items into the data instead of replacing them. (IMHO)
protected function getPostedData()
{
if ( is_null($this->postedData) )
{
$this->postedData = array_merge(
(array) $this->getRequest()->getPost(),
(array) $this->getRequest()->getFiles()
);
}
return $this->postedData;
}
I am using array_merge:
$form = $this->getForm('my_form');
$request = $this->getRequest();
if($request->isPost())
{
$file = $this->params()->fromFiles('name_of_file');
$form->setData(array_merge(
$request->getPost()->toArray(),
array('arquivo' => $file['name'])
));
if ($form->isValid()) {
// now i can validate the form field
I use variable variables like this article explains to create variables and then echo them as the values for each form entry.
example:
// create array of GET/POST variables then convert each variable to a local variable
$fields = array_keys($_REQUEST);
foreach ($fields as $field) {
$$field = $_REQUEST[$field];
}

Categories