Yii2: PHPExcel error when process Excel contain 90k rows - php

I have an application that has upload Excel file process. Then PHPExcel will process the uploaded Excel file.
This is the flow of process:
Upload file
Save file into local directory
Get file from local dir
Read the file using PHPExcel
I've successfully save the Excel file to local directory, reading and get the data from the Excel file.
But when I use an Excel file that contain 90K of rows x 25 columns, that file didn't saved into local directory. Then I'm trying to print_r the data and it return error=> 1
This is the print_r result
1. Excel with less than 8k rows and successfully saved to local dir:
2. Excel with 90K of rows and didn't saved into local dir:
This is my code in controller
public function actionCreate() {
$connection = \Yii::$app->db;
$model = new MasterProduct();
$userId = Yii::$app->user->id;
$date = new DateTime('now', new \DateTimeZone('Asia/Bali'));
$created_at = $date->format('Y:m:d H:i:s');
$dirTrash = Yii::getAlias('#webroot/trash');
$dirSubTrash = Yii::getAlias('#webroot/trash/trash_app');
//create directory in local if doesn't exist
if (!is_dir($dirTrash)) {
mkdir(Yii::getAlias('#webroot/trash'));
if (!is_dir($dirSubTrash)) {
mkdir(Yii::getAlias('#webroot/trash/trash_app'));
}
} else {
if (!is_dir($dirSubTrash)) {
mkdir(Yii::getAlias('#webroot/trash/trash_app'));
}
}
if (Yii::$app->request->post()) {
Yii::$app->response->format = Response::FORMAT_JSON;
$result = [];
$fileMaster = UploadedFile::getInstancesByName('master_product');
$middleName = substr(md5(microtime() * 100000), rand(0, 9), 5);
//named the uploaded file, then save into local directory
if ($fileMaster !== null) {
$nameMaster = $userId . '_MP' . '_' . $middleName . '_' . date('Y-m-d') . '_' . $fileMaster[0]->getBaseName() . "." . $fileMaster[0]->getExtension();
$pathMaster = Yii::getAlias('#webroot/trash/trash_app/') . $nameMaster;
$fileMaster[0]->saveAs($pathMaster);
} else {
$error[] = "Choose the" . "<strong>" . " Excel " . "</strong>" . "file first.";
$result = [
'error' => $error
];
}
//this the code i've used to print_out the array in above images
echo '<pre>';
print_r($fileMaster);
die();
//Access, and get the data from local directory
$objPHPExcelMaster = new \PHPExcel();
$fileNameMaster = Yii::getAlias('#webroot/trash/trash_app/') . $nameMaster;
$inputFilesMaster = fopen(Yii::getAlias('#webroot/trash/trash_app/') . $nameMaster, "r");
try {
$inputFileTypeMaster = \PHPExcel_IOFactory::identify($fileNameMaster);
$objReaderMaster = \PHPExcel_IOFactory::createReader($inputFileTypeMaster);
$objPHPExcelMaster = $objReaderMaster->load($fileNameMaster);
} catch (Exception $ex) {
die('Error');
}
$sheetMaster = $objPHPExcelMaster->getSheet(0);
$highestRowMaster = $sheetMaster->getHighestDataRow();
$highestColumnMaster = $sheetMaster->getHighestDataColumn();
//read the Excel file and set into array
for ($row = 1; $row <= $highestRowMaster; ++$row) {
$rowDataMaster = $sheetMaster->rangeToArray('A' . $row . ':' . $highestColumnMaster . $row, NULL, TRUE, NULL);
}
$tempDataMaster = mysql_escape_string(json_encode($rowDataMaster));
$commandCheckExist = "SELECT COUNT(user_id) FROM `gm_json_master` WHERE id=$userId";
$queryCheckExist = Yii::$app->db->createCommand($commandCheckExist)->execute();
//save data into db
if ($queryCheckExist == 0) {
$command_2 = "INSERT INTO json_master(json_m_product) VALUES('$tempDataMaster')";
$query_2 = Yii::$app->db->createCommand($command_2)->execute();
} else {
$commandUpdate = "UPDATE json_master SET json_m_product='$tempDataMaster'";
$queryUpdate = Yii::$app->db->createCommand($commandUpdate)->execute();
}
$result = [
'url' => Url::to(['/gm/json-master-product/save'])
];
return $result;
} else {
return $this->render('create', [
'model' => $model,
]);
}
}
How does this happen? and how do I can read big data of Excel using PHPExcel?

An error code of 1 means that the file size exceeds the max filesize allowed by your server's configuration.
From the docs:
UPLOAD_ERR_INI_SIZE
Value: 1; The uploaded file exceeds the upload_max_filesize directive in php.ini.
I would try upping that limit as well as the max POST size limit and see if that helps.

Related

If file exist on destination append number to file

i know this question is asked before but i cant figure out how to get this to work. I have a function that triggers whenever i delete a post. It will then move the file to a existing folder on the server. what i would like for it to do is to check if there is already a file with the same name in this folder. if this file exist it should append a number to this file. for example if file 1234.jpg exist it should rename the file that i want to move to 1234-1.jpg.
The code i have so far is:
add_action('before_delete_post', function ($id) {
$rs_image_path = "/var/www/html/wp-content/uploads/hexon/";
$rs_image_dest_path = "/var/www/html/wp-content/uploads/wpallimport/files/images/";
// $rs_image_filenames="30946280-1.JPG,30946280-2.JPG,30946280-3.JPG,30946280-4.JPG";
// $rs_image_klantnummer = "1232";
$rs_image_filenames = get_field('afbeeldingen_bestandsnamen', $id);
$rs_image_klantnummer = get_field('hexon_klantnummer', $id);
$variableAry = explode(", ", $rs_image_filenames);
foreach ($variableAry as $rs_image_url) {
/* Store the path of source file */
$filePath = $rs_image_path . "" . $rs_image_klantnummer . "/" . $rs_image_url;
/* Store the path of destination file */
$destinationFilePath = $rs_image_dest_path . "" . $rs_image_klantnummer . "/" . $rs_image_url;
/* Move Files */
if (!file_exists($filePath) || !is_readable($filePath)) {
// some error handling here
} else {
$b = !rename($filePath, $destinationFilePath);
}
}
}
);
How about this?
$n = '';
while (file_exists($destinationFilePath.$n) || !rename($filePath, $destinationFilePath.$n)) {
$n++;
// escape loop
if ($n > 100) {
throw new Exception('Error saving');
}
}
$destinationFilePath .= $n;

Can't insert file's path into database.(I'm New to HTML and PHP)

I'm trying to upload the file's path into my database. But nothing is being inserted. My file gets uploaded to target directory successfully. I want to insert the path too, but can't do it. I believe I'm doing some mistake in the Insert Into statement. Please let me know what's wrong?
My upload.php code is below:
<?php
// variables
$conn = mysqli_connect('localhost','root','abcdef','trademark');
if(!$conn) {
echo "Not Connected To Server";
}
else {
define('UPLOAD_DIR', 'uploads/');
$fileName = $_FILES['file'];
// check for which action should be taken if file already exist
//Rename file name
if(file_exists(UPLOAD_DIR . $fileName['name']))
{
$updatedFileName = update_file_name(UPLOAD_DIR.$fileName['name']);
move_uploaded_file($fileName['tmp_name'], $updatedFileName);
echo"FILE SUCCESSFULLY UPLOADED!! " . "<br/><br/><br/>"; //after renaming
}
// If no such file already exists, then upload it as it is
else
{
move_uploaded_file($fileName['tmp_name'], UPLOAD_DIR.$fileName['name']);
echo " FILE SUCCESSFULLY UPLOADED!! ". "<br/><br/>";
}
// function to rename file
function update_file_name($file)
{
$pos = strrpos($file,'.');
$ext = substr($file,$pos);
$dir = strrpos($file,'/');
$dr = substr($file,0,($dir+1));
$arr = explode('/',$file);
$fName = trim($arr[(count($arr) - 1)],$ext);
$exist = FALSE;
$i = 2;
while(!$exist)
{
$file = $dr.$fName.'_'.$i.$ext;
if(!file_exists($file))
$exist = TRUE;
$i++;
}
return $file;
} // function to rename ends
$sql = "INSERT INTO file (Path) VALUES (' " . mysqli_real_escape_string( UPLOAD_DIR.$fileName['name']) . " ')";
$r = mysqli_query($conn,$sql);
echo 'file info inserted';
}
?>
Check syntax for function mysqli_real_escape_string
getting warning message as,
Warning: mysqli_real_escape_string() expects exactly 2 parameters, 1
given in

memory limit and slow upload using class upload php

i am creating upload image using class upload from https://www.verot.net/
my problem: output image 2 (real photo and edit-resize photo).
all work but it wasting time while processing upload.
and when i check size real photo before and after upload, size not same.
$counter=1;
foreach ($files as $file) {
$handle = new Upload($file);
if ($handle->uploaded)
{
$handle->Process("real");
$handle->dir_chmod = 0777;
//$handle->jpeg_quality = 85;
//jenis file yang diijinkan
$handle->allowed = array('image/*');//VALIDASI UPLOAD GAMBAR
//$handle->image_resize = false;
/*----first-START----*/
if ($handle->processed) {
${'pathPhotoReal'.$counter} = 'real/'.$handle->file_dst_name;
echo ${'pathPhotoReal'.$counter}."<br>";
${'namaFoto'.$counter} = $_POST['text'.$counter];
echo ${'namaFoto'.$counter}."<br>";
}
else {
// one error occured
echo ' Error: ' . $handle->error . '';
}
/*----first-FINISH----*/
// we now process the image a second time, with some other settings
//$handle->allowed = array('image/*');
$handle->image_ratio = true;
$handle->image_ratio_fill = true;
$handle->image_resize = true;
$handle->image_x = 300;
$handle->image_y = 300;
$handle->Process('edit');
/*----second-START----*/
if ($handle->processed) {
${'pathFotoEdit'.$counter} = 'edit/'.$handle->file_dst_name;
echo ${'pathFotoEdit'.$counter}."<br>";
}
else {
// one error occured
echo ' Error: ' . $handle->error . '';
}
/*----second-FINISH----*/
$counter = $counter + 1;
//$handle->clean();
else {
// if we're here, the upload file failed for some reasons
// i.e. the server didn't receive the file
echo ' File not upload ' . $handle->error . '';
echo($handle->log);
}
}
in my last code using else{echo ' File not upload ' . $handle->error . '';}
it always show, inside folder pathPhoto have saved photo inside. i think upload success.
try changing the folder permissions from filezilla.
With this you can know if the server allows you to make those persmisos

how to unzip txt.gz file and store into database using php

My zip file name is Product_Catalog.txt.gz. this zip file contain one txt file.
how can i extract and store into my database
i have already done in zip file unzip and store into my database. but i can't understand .gz format. so please advise
my zip file code is here
if (realpath($destinationname."/".$filename)){
if ($zip = zip_open(realpath($directory."/".$filename)))
{
while (($zip_entry = zip_read($zip))){
$zipfilename=zip_entry_name($zip_entry );
$zipfilename."<br>";
$tmpfname = tempnam("/tmp", "");
$handle = fopen($tmpfname, "w+");
while($data = zip_entry_read($zip_entry,50000000000)){
fwrite($handle,$data);
}// end while $data
fseek($handle,0);
if ($separatetables);
$table=strtok($zipfilename,"-");
$sql="CREATE TABLE IF NOT EXISTS `$table` LIKE `cjfeeds`";
mysql_query($sql);
how can i convert this code into txt.gz
gzip, which is generally used for single files, is not the same file format as ZIP. Although the same compression algorithm (DEFLATE) is used in both formats, the headers are entirely different, so PHP's Zip functions will not recognize your file.
Instead, you can use the compress.zlib:// wrapper to open gzip-compressed files. You can then use the normal stream functions to read the file.
$handle = fopen("compress.zlib://$filename", 'r');
However, there are some limitations; for example, opening a gzipped file in read-write mode is not possible, and seeking may be slow. If necessary, you can work around these by making a temporary uncompressed copy:
copy("compress.zlib://$filename", $tmpfname);
You don't extract.
if ($rrpath){
if ($zip = gzopen($rrpath, "rb"))
{
$filenamen = $_SESSION['files_list'][$i];
$zipfilename = str_replace('.gz', '', $_SESSION['files_list'][$i]);
$tmpfname = tempnam("/home/demoosiz/tmp", "");
$handle = fopen($tmpfname, "w+");
ini_set("max_execution_time",3000000000000);
while($data = gzread($zip, 4096)){
fwrite($handle,$data);
}// end while $data
fseek($handle,0);
if ($separatetables);
$table=strtok($zipfilename,"-");
// $sqly = "TRUNCATE TABLE `cjfeeds`";
//mysql_query($sqly);
// I'm not too sure if this is windows specific
$tmpfile=addslashes($tmpfname);
//if the script times out uncomment the next line
ini_set("max_execution_time",3000000000000);
$sql22 ="LOAD DATA LOCAL INFILE '$tmpfile' REPLACE INTO TABLE `cjfeeds` FIELDS TERMINATED BY '$fieldseparator' IGNORE 1 LINES;";
You can use this code. This code is very helpful for you.
You keep calling it a zip file. It's not a zip file (.zip). It's a gzip file (.gz). Different format.
You can use gzdecode() to decompress a gzip file.
You can interact with .gz files through the functions of the zlib module.
Responding to your comment:
You won't need to do the while($zip_entry = zip_read($zip)) since gz files don't hold multiple files within them. A .gz file is just a single file that has been compressed. Unfortunately, this also means that there is no equivalent to zip_entry_name either, since there is only one file.
If you're trying to read a tarball - an archive of files which have been concatenated together and gzipped to form a .tar.gz file - then Zlib isn't for you, since it's not designed to handle that. You'll want something like PharData instead:
$phar = new PharData($filename);
foreach ($phar as $phar_stream) {
$file_data = file_get_contents($phar_stream);
// process $file_data how you like
}
If these are just single-file gz files, then you can read the files with the gz versions of the usual filesystem functions.
You can use gzread to process chunks of the file the way you're currently doing with zip_entry_read, or you can read all lines into an array with gzfile. There is a readgzfile function which grabs the whole file, but unfortunately it seems like it just dumps the output directly to the client, rather than do a file. You could use output buffering to capture that output, but it seems like too much of a hassle considering your other options.
Try this....
if(isset($_FILES['zipfile']))
{
$filename = $_FILES['zipfile']['name'];
$source = $_FILES['zipfile']['tmp_name'];
$type = $_FILES['zipfile']['type'];
/*$name = explode('.zip', $filename); # $name[0] returns the name of the file. $name[1] returns the extension (zip)
; */ # Where the file will be saved. I.E. 'extracted/myFile-02151985/'
$target = PHYSICAL_PATH."upload/";
// Ensures that the correct file type was chosen.
$accepted_types = array('application/zip', 'application/x-zip-compressed', 'multipart/x-zip', 'application/x-compressed');
foreach($accepted_types as $mime_type)
{
if($mime_type == $type)
{
$okay = true;
break;
}
}
$okay = strtolower($name[1]) == 'zip' ? true : false;
if(!$okay)
{
$msg="Please choose a zip file, dummy!";
}
$saved_file_location = $target.$filename;
if(move_uploaded_file($source, $target . $filename))
{
global $target;
global $unique_folder;
$zip = new ZipArchive();
$x = $zip->open($saved_file_location);
if ($x === true)
{
$folder=$zip->extractTo($target);
for ( $i=0; $i < $zip->numFiles; $i++ )
{
echo $entry = $zip->getNameIndex($i);
$filename=explode('/',$entry);
print_r($filename);
if($filename[1] != '')
{
$filename_folder=$filename[0];
$filename = $filename[1]; //
$name = explode('.zip', $filename_folder); # $name[0] returns the name of the file. $name[1] returns the extension (zip)
//echo 'sfkdf'.$name[0];
$target = PHYSICAL_PATH."upload/";
$sql = "select `id` from `tracks` order by `id` desc";
$list = mysql_query($sql);
$list_num = mysql_num_rows($list);
if($list_num > 0)
{
$res = mysql_fetch_array($list);
$new_id=$res['id'];
} else {
$new_id=1;
}
$check_for_jpg = strpos($filename, '.mp3');
if ($check_for_jpg === false ){
echo 'Check file format.';
$source=$target . $name[0].'/'.$filename;
unlink($source);
} else {
$srch = array(' ','--','"','!','#','#','$','%','^','&','*','(',')','_','+','{','}','|',':','"','<','>','?','[',']','\\',';',"'",',','/','*','+','~','`','=');
$rep = array('_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_');
$name_song[$i]= str_replace($srch,$rep,$filename);
$title_name=explode('.',$filename);
$new_id=$new_id+1;
$new_filename=$new_id.'-'.$name_song[$i];
mysql_query("insert into `tracks` set track='".addslashes(stripslashes($new_filename))."',title='".addslashes(stripslashes($title_name[0]))."',tape_id='".$id_last."',addDate=NOW(),status=1 ");
$source=$target . $name[0].'/'.$filename; $target_move=$target.$new_filename;
if(rename($source,$target_move)) { unlink($source); } else { echo 'not';}
echo 'folder zip';
echo '<li>' . $new_filename . '</li>';
}
rmdir($target . $name[0]);
}
else { echo $target = PHYSICAL_PATH."upload/";
$sql = "select `id` from `test_table` order by `id` desc";
$list = mysql_query($sql);
$list_num = mysql_num_rows($list);
if($list_num > 0) {
//and, instead of using "while":
$res = mysql_fetch_array($list); // will return the highest "id" number
echo $new_id=$res['id'];
} else {
$new_id=1;
}
$check_for_jpg = strpos($entry, '.mp3');
if ($check_for_jpg === false ){
echo 'Check file format.';
$source=$target .$entry;
unlink($source);
} else {
$srch = array(' ','--','"','!','#','#','$','%','^','&','*','(',')','_','+','{','}','|',':','"','<','>','?','[',']','\\',';',"'",',','/','*','+','~','`','=');
$rep = array('_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_','_');
$name_song[$i]= str_replace($srch,$rep,$entry);
$title_name=explode('.',$entry);
$new_id=$new_id+1;
$new_filename=$new_id.'-'.$name_song[$i];
mysql_query("insert into `tracks` set track='".addslashes(stripslashes($new_filename))."',title='".addslashes(stripslashes($title_name[0]))."',tape_id='".$id_last."',status=1 ");
$source=$target .$entry; $target_move=$target.$new_filename;
if(rename($source,$target_move)) { unlink($source); } else { echo 'not';}
echo 'only zip';
echo '<li>' . $new_filename . '</li>';
}
}
}
$zip->close();
unlink($saved_file_location); #deletes the zip file. We no longer need it.
} else {
die("There was a problem. Please try again!");
}
} else {
$msg="There was a problem";
}
}

swfupload destroy session? php

hy, i need a little help here:
i use SWFupload to upload images!
in the upload function i make a folder call $_SESSION['folder'] and all the files i upload are in 1 array call $_SESSION['files'] after uploads finish i print_r($_SESSION) but the array is empty? why that?
this is my upload.php:
if($_FILES['image']['name']) {
list($name,$error) = upload('image','jpeg,jpg,png');
if($error) {$result = $error;}
if($name) { // Upload Successful
$result = watermark($name);
print '<img src="uploads/'.$_SESSION['dir'].'/'.$result.'" />';
} else { // Upload failed for some reason.
print 'noname'.$result;
}
}
function upload($file_id, $types="") {
if(!$_FILES[$file_id]['name']) return array('','No file specified');
$isimage = #getimagesize($_FILES[$file_id]['tmp_name']);
if (!$isimage)return array('','Not jpg');
$file_title = $_FILES[$file_id]['name'];
//Get file extension
$ext_arr = split("\.",basename($file_title));
$ext = strtolower($ext_arr[count($ext_arr)-1]); //Get the last extension
//Not really uniqe - but for all practical reasons, it is
$uniqer = substr(md5(uniqid(rand(),1)),0,10);
//$file_name = $uniqer . '_' . $file_title;//Get Unique Name
//$file_name = $file_title;
$file_name = $uniqer.".".$ext;
$all_types = explode(",",strtolower($types));
if($types) {
if(in_array($ext,$all_types));
else {
$result = "'".$_FILES[$file_id]['name']."' is not a valid file."; //Show error if any.
return array('',$result);
}
}
if((!isset($_SESSION['dir'])) || (!file_exists('uploads/'.$_SESSION['dir']))){
$dirname = date("YmdHis"); // 20010310143223
$pathtodir = $_SERVER['DOCUMENT_ROOT']."/ifunk/uploads/";
$newdir = $pathtodir.$dirname;
if(!mkdir($newdir, 0777)){return array('','cannot create directory');}
$_SESSION['dir'] = $dirname;
}
if(!isset($_SESSION['files'])){$_SESSION['files'] = array();}
//Where the file must be uploaded to
$folder = 'uploads/'.$_SESSION['dir'].'/';
//if($folder) $folder .= '/'; //Add a '/' at the end of the folder
$uploadfile = $folder.$file_name;
$result = '';
//Move the file from the stored location to the new location
if (!move_uploaded_file($_FILES[$file_id]['tmp_name'], $uploadfile)) {
$result = "Cannot upload the file '".$_FILES[$file_id]['name']."'"; //Show error if any.
if(!file_exists($folder)) {
$result .= " : Folder don't exist.";
} elseif(!is_writable($folder)) {
$result .= " : Folder not writable.";
} elseif(!is_writable($uploadfile)) {
$result .= " : File not writable.";
}
$file_name = '';
} else {
if(!$_FILES[$file_id]['size']) { //Check if the file is made
#unlink($uploadfile);//Delete the Empty file
$file_name = '';
$result = "Empty file found - please use a valid file."; //Show the error message
} else {
//$_SESSION['files'] = array();
$_SESSION['files'][] .= $file_name;
chmod($uploadfile,0777);//Make it universally writable.
}
}
return array($file_name,$result);
}
SWFUpload doesn't pass the session ID to the script when you upload, so you have to do this yourself. Simply pass the session ID in a get or post param to the upload script, and then in your application do this before session_start:
if(isset($_REQUEST['PHPSESSID'])) {
session_id($_REQUEST['PHPSESSID']);
}
you must pass the session ID to the upload file used by swfupload.
more details here

Categories