I'm trying to export a list of contacts with a specified set of fields from in Codeigniter. PHP is exiting early and I can't figure out why. It exports 4,092 contacts then exits but the count of the array being exported is 140,699. PHP gives me no errors, and on my test server the export function works fine. Here's the code:
function admin_export()
{
set_time_limit(0);
if(!$this->ion_auth->in_group(array('admin')) && !$this->input->is_cli_request())
die();
$contacts = $this->contacts_model->get_contacts();
$export_fields = unserialize($this->contacts_model->get_contact_export_fields());
if(!file_exists($this->config->item('tmp_path')))
mkdir($this->config->item('tmp_path'));
if($export_fields == false)
{
echo json_encode(false);
}
else
{
$fh = fopen($this->config->item('tmp_path').'export.csv', 'w');
fputcsv($fh, $export_fields, ',');
foreach($contacts as $i => $contact)
{
$id = $contact['id'];
foreach($contact as $k => $v)
{
if(!in_array($k, $export_fields))
{
unset($contacts[$i][$k]);
}
}
if(in_array('role', $export_fields))
{
$contacts[$i]['role'] = '';
$roles = $this->contacts_model->get_contact_roles($id);
foreach($roles as $role)
{
$contacts[$i]['role'] .= $role['role'].';';
}
$contacts[$i]['role'] = rtrim($contacts[$i]['role'], ';');
}
if(in_array('role_id', $export_fields))
{
$contacts[$i]['role_id'] = '';
$role_ids = $this->contacts_model->get_contact_roles($id);
foreach($role_ids as $role_id)
{
$contacts[$i]['role_id'] .= $role_id['id'].';';
}
$contacts[$i]['role_id'] = rtrim($contacts[$i]['role_id'], ';');
}
fputcsv($fh, $contacts[$i], ',');
}
fclose($fh);
echo json_encode(true);
}
}
What about memory? It's possible that it is maxing out the allocated memory and stopping.
Related
Thank you StackOverflow experts for looking at my question.
First, It is possible this question has been asked before but my situation is a bit unique. So, please hear me out.
When our users want to edit an existing record, they would also like to have the ability to delete an existing pdf file if one exists before adding a new one.
To display an existing file, I use this code.
<td class="td_input_form">
<?php
// if the BidIDFile is empty,
if(empty($result["BidIDFile"]))
{
//then show file upload field for Bid File
echo '<input type="file" name="BidIDFile[]" size="50">';
}
else
{
// Bid file already upload, show checkbox to delete it.
echo '<input type="checkbox" name="delete[]" value="'.$result["BidIDFile"].'"> (delete)
'.$result["BidIDFile"].'';
}
</td>
Then to delete this file, I use the following code:
// Connect to SQL Server database
include("connections/Connect.php");
// Connect to SQL Server database
include("connections/Connect.php");
$strsID = isset($_GET["Id"]) ? $_GET["Id"] : null;
if(isset($_POST['delete']))
{
// whilelisted table columns
$fileColumnsInTable = array( 'BidIDFile', 'TabSheet', 'SignInSheet', 'XConnect',
'Addend1', 'Addend2','Addend3','Addend4','Addend5', 'Addend6');
$fileColumns = array();
foreach ($_POST['delete'] as $fileColumn)
{
if(in_array($fileColumn, $fileColumnsInTable))
$fileColumns[] = $fileColumn;
}
// get the file paths for each file to be deleted
$stmts = "SELECT " . implode(', ', $fileColumns) . " FROM bids WHERE ID = ? ";
$querys = sqlsrv_query( $conn, $stmts, array($strsID));
$files = sqlsrv_fetch_array($querys,SQLSRV_FETCH_ROW);
// loop over the files returned by the query
foreach ($files as $file )
{
//delete file
unlink($file);
}
// now remove the values from the table
$stmts = "UPDATE bids SET " . impload(' = '', ', $fields) . " WHERE ID = ? ";
$querys = sqlsrv_query( $conn, $stmts, array($strsID));
This works fine. However, the edit file points to an existing file with an INSERT and UPDATE operation in this one file (great thanks to rasclatt) and I am having problem integrating the two together.
Can someone please help with integrating the two files into one?
Thanks in advance for your assistance.
Here is the INSERT and UPDATE file:
<?php
error_reporting(E_ALL);
class ProcessBid
{
public $data;
public $statement;
public $where_vals;
protected $keyname;
protected $conn;
public function __construct($conn = false)
{
$this->conn = $conn;
}
public function SaveData($request = array(),$skip = false,$keyname = 'post')
{
$this->keyname = $keyname;
$this->data[$this->keyname] = $this->FilterRequest($request,$skip);
return $this;
}
public function FilterRequest($request = array(), $skip = false)
{
// See how many post variables are being sent
if(count($request) > 0) {
// Loop through post
foreach($request as $key => $value) {
// Use the skip
if($skip == false || (is_array($skip) && !in_array($key,$skip))) {
// Create insert values
$vals['vals'][] = "'".ms_escape_string($value)."'";
// Create insert columns
$vals['cols'][] = "".str_replace("txt","",$key)."";
// For good measure, create an update string
$vals['update'][] = "".str_replace("txt","",$key)."".' = '."'".ms_escape_string($value)."'";
// For modern day binding, you can use this array
$vals['bind']['cols'][] = "".$key."";
$vals['bind']['cols_bind'][] = ":".$key;
$vals['bind']['vals'][":".$key] = $value;
$vals['bind']['update'][] = "".$key.' = :'.$key;
}
}
}
return (isset($vals))? $vals:false;
}
public function AddFiles($name = 'item')
{
// If the files array has been set
if(isset($_FILES[$name]['name']) && !empty($_FILES[$name]['name'])) {
// Remove empties
$_FILES[$name]['name'] = array_filter($_FILES[$name]['name']);
$_FILES[$name]['type'] = array_filter($_FILES[$name]['type']);
$_FILES[$name]['size'] = array_filter($_FILES[$name]['size']);
$_FILES[$name]['tmp_name'] = array_filter($_FILES[$name]['tmp_name']);
// we need to differentiate our type array names
$use_name = ($name == 'item')? 'Addend':$name;
// To start at Addendum1, create an $a value of 1
$a = 1;
if(!empty($_FILES[$name]['tmp_name'])) {
foreach($_FILES[$name]['name'] as $i => $value ) {
$file_name = ms_escape_string($_FILES[$name]['name'][$i]);
$file_size = $_FILES[$name]['size'][$i];
$file_tmp = $_FILES[$name]['tmp_name'][$i];
$file_type = $_FILES[$name]['type'][$i];
if(move_uploaded_file($_FILES[$name]['tmp_name'][$i], $this->target.$file_name)) {
// Format the key values for addendum
if($name == 'item')
$arr[$use_name.$a] = $file_name;
// Format the key values for others
else
$arr[$use_name] = $file_name;
$sql = $this->FilterRequest($arr);
// Auto increment the $a value
$a++;
}
}
}
}
if(isset($sql) && (isset($i) && $i == (count($_FILES[$name]['tmp_name'])-1)))
$this->data[$name] = $sql;
return $this;
}
public function SaveFolder($target = '../uploads/')
{
$this->target = $target;
// Makes the folder if not already made.
if(!is_dir($this->target))
mkdir($this->target,0755,true);
return $this;
}
public function where($array = array())
{
$this->where_vals = NULL;
if(is_array($array) && !empty($array)) {
foreach($array as $key => $value) {
$this->where_vals[] = $key." = '".ms_escape_string($value)."'";
}
}
return $this;
}
public function UpdateQuery()
{
$this->data = array_filter($this->data);
if(empty($this->data)) {
$this->statement = false;
return $this;
}
if(isset($this->data) && !empty($this->data)) {
foreach($this->data as $name => $arr) {
$update[] = implode(",",$arr['update']);
}
}
$vars = (isset($update) && is_array($update))? implode(",",$update):"";
// Check that both columns and values are set
$this->statement = (isset($update) && !empty($update))? "update bids set ".implode(",",$update):false;
if(isset($this->where_vals) && !empty($this->where_vals)) {
$this->statement .= " where ".implode(" and ",$this->where_vals);
}
return $this;
}
public function SelectQuery($select = "*",$table = 'bids')
{
$stmt = (is_array($select) && !empty($select))? implode(",",$select):$select;
$this->statement = "select ".$stmt." from ".$table;
return $this;
}
public function InsertQuery($table = 'bids')
{
$this->data = array_filter($this->data);
if(empty($this->data)) {
$this->statement = false;
return $this;
}
$this->statement = "insert into ".$table;
if(isset($this->data) && !empty($this->data)) {
foreach($this->data as $name => $arr) {
$insert['cols'][] = implode(",",$arr['cols']);
$insert['vals'][] = implode(",",$arr['vals']);
}
}
$this->statement .= '(';
$this->statement .= (isset($insert['cols']) && is_array($insert['cols']))? implode(",",$insert['cols']):"";
$this->statement .= ") VALUES (";
$this->statement .= (isset($insert['vals']) && is_array($insert['vals']))? implode(",",$insert['vals']):"";
$this->statement .= ")";
return $this;
}
}
include("../Connections/Connect.php");
function render_error($settings = array("title"=>"Failed","body"=>"Sorry, your submission failed. Please go back and fill out all required information."))
{ ?>
<h2><?php echo (isset($settings['title']))? $settings['title']:"Error"; ?></h2>
<p><?php echo (isset($settings['body']))? $settings['body']:"An unknown error occurred."; ?></p>
<?php
}
// this function is used to sanitize code against sql injection attack.
function ms_escape_string($data)
{
if(!isset($data) || empty($data))
return "";
if(is_numeric($data))
return $data;
$non_displayables[] = '/%0[0-8bcef]/'; // url encoded 00-08, 11, 12, 14, 15
$non_displayables[] = '/%1[0-9a-f]/'; // url encoded 16-31
$non_displayables[] = '/[\x00-\x08]/'; // 00-08
$non_displayables[] = '/\x0b/'; // 11
$non_displayables[] = '/\x0c/'; // 12
$non_displayables[] = '/[\x0e-\x1f]/'; // 14-31
foreach($non_displayables as $regex)
$data = preg_replace($regex,'',$data);
$data = str_replace("'","''",$data);
return $data;
}
// New bid save engine is required for both sql statement generations
$BidSet = new ProcessBid($conn);
$strId = null;
if(isset($_POST["Id"]))
{
$strId = $_POST["Id"];
//echo $strId;
}
If ($strId == "") {
//echo "This is an insert statement";
// This will generate an insert query
$insert = $BidSet->SaveData($_POST)
->SaveFolder('../uploads/')
->AddFiles('BidIDFile')
->AddFiles('item')
->AddFiles('SignInSheet')
->AddFiles('TabSheet')
->AddFiles('Xcontract')
->InsertQuery()
->statement;
// Check that statement is not empty
if($insert != false) {
sqlsrv_query($conn,$insert);
render_error(array("title"=>"Bid Successfully Saved!","body"=>'Go back to Solicitation screen'));
$err = false;
}
//echo '<pre>';
//print_r($insert);
// echo '</pre>';
}
else
{
//echo "This is an update statement";
// This will generate an update query
$update = $BidSet->SaveData($_POST,array("Id"))
->SaveFolder('../uploads/')
->AddFiles('BidIDFile')
->AddFiles('item')
->AddFiles('SignInSheet')
->AddFiles('TabSheet')
->AddFiles('Xcontract')
->where(array("Id"=>$_POST["Id"]))
->UpdateQuery()
->statement;
//echo '<pre>';
//print_r($update);
//echo '</pre>';
// Check that statement is not empty
if($update != false) {
sqlsrv_query($conn,$update);
render_error(array("title"=>"Bid Successfully Saved!","body"=>'Go back to admin screen'));
$err = false;
}
}
// This will post an error if the query fails
if((isset($err) && $err == true) || !isset($err))
render_error(); ?>
Here is the snippet of code in question:
function hoursearned_func($user) {
$key = 'Service Hours Earned';
if(isset($user[$key])) {
$result = $user[$key];
}
return "Service Hours Earned: ".$result." Hours";
}
add_shortcode('hoursearned', 'hoursearned_func');
This small function simply returns the value of an array given the key, but when calling the shortcode, [hoursearned], it only displays:
Service Hours Earned: Hours
rather than:
Service Hours Earned: 200 Hours
Where 200 is a string that is returned by the function hoursearned_func given $user.
For some reason Wordpress is not passing the returned variables from the functions to the shortcodes. How can I get Wordpress to show the result from the function?
Here is the full code if that helps... (note that all of this is being loaded as a plugin)
//Dump CSV file (SalesForce Report) to php array to be parsed..
add_action('dump_csv_array', 'dump_csv_array', 10);
function dump_csv_array(){
$data = array();
$header = NULL;
if (($file = fopen('http://blah.com/info.csv', 'r')) !==FALSE) { //The link to the CSV is insignificant
while (($row = fgetcsv($file, 1000, ",")) !==FALSE) {
if(!$header)
$header = $row;
else
$data[] = array_combine($header, $row);
}
fclose($file);
}
return $data;
}
$multiarray = (array) dump_csv_array();
global $current_user;
get_currentuserinfo();
$user_email = $current_user->user_email;
//Parses the multidimensional array of Dumped CSV info. Looks up the current user's email and returns a single array of the current user's SalesForce information.
function parse_array($multiarray, $user_email){
$array = (array) $multiarray;
$email = $user_email;
if(is_user_logged_in() === true ) {
foreach($array as $subarray) {
if(isset($subarray['Email']) && $subarray['Email'] == $email) {
$data = $subarray;
}
}
}
else {
return NULL;
}
return $data;
}
$user = parse_array($multiarray, $user_email);
//Defines Shortcodes and their Functions
function hoursearned_func($user) {
$key = 'Service Hours Earned';
if(isset($user[$key])) {
$result = $user[$key];
}
return "Service Hours Earned: ".$result." Hours";
}
function hoursawarded_func($user) {
$key = 'Service Hours Awarded';
if(isset($user[$key])) {
$result = $user[$key];
}
return "Service Hours Awarded: ".$result." Hours";
}
function rank_func($user) {
$key = 'Rank';
if(isset($user[$key])) {
$result = $user[$key];
}
return "Rank: ".$result;
}
function memstatus_func($user) {
$key = 'Membership Status';
if(isset($user[$key])) {
$result = $user[$key];
}
return "Status: ".$result;
}
function register($atts) {
add_shortcode( 'hoursearned', 'hoursearned_func');
add_shortcode( 'hoursawarded', 'servicehours_func');
add_shortcode( 'rank', 'rank_func');
add_shortcode( 'memstatus', 'memstatus_func');
}
add_action('init', 'register');
var_dump(hoursearned_func($user));
Try to create a shortcode with attributes, (or) retrieve the $user data inside the shortcode's function and set them as default values.
function example_func($atts) {
$user = get_user_array(); //Retrieve the data
$atts = shortcode_atts($user, $atts);
return 'example: ' . $atts['foo'] . ' ' . $atts['bar'];
}
add_shortcode('example', 'example_func');
Have a look at the Shortcode API it basically explains everything.
Consider the following: I am using gzipped .cvs reports downloaded through an API to render data live (constructTable). First I check to see if the file has been downloaded, then if it's complete (more thorough check should be implemented, just don't know how yet). If everything's okay, it should read the file and turn it into an array - which I return to the recursive function which in turn returns it to my constructTable function. But the array won't return from the checkReport function to the constructTable. What am I doing wrong?
function fileToArray($filePath){
$file = fopen($filePath,'rb');
$gzipcsv = fread($file, filesize($filePath));
fclose($file);
$ungzipcsv = gzdecode($gzipcsv);
return parse_csv($ungzipcsv);
}
function checkReport($filePath,$queryStr,$status=''){
if($status!='ok'){
if(!file_exists($filePath)){
if($status!='creating_report'){
DownloadReport($_SESSION['currentUser'],$filePath,$queryStr);
$status='creating_report';
} else {
usleep(100000);
$status='downloading_report';
}
} else {
if(filesize($filePath)!=0){
$fs = filesize($filePath);
usleep(100000);
clearstatcache(TRUE, $filePath);
if($fs==filesize($filePath)){
$status='ok';
} else {
$status='downloading_report';
}
} else {
$status='zero_filesize';
}
}
checkReport($filePath,$queryStr,$status);
} else {
return fileToArray($filePath);
}
}
function constructTable($user, $fields, $dataType, $reportName, $conditionItem, $conditionId, $dateRange) {
$filePath = dirname(dirname(__FILE__)) . '/csv/'.$GLOBALS['customer'].'__'.$dataType.'__'.$reportName.'.csv.gz';
$queryStr = 'SELECT '.implode(',',$fields);
$queryStr .= ' FROM '.$reportName;
$queryStr .= '';
if($conditionId!='') $queryStr .= ' WHERE '.$conditionItem.' = '.$conditionId;
$queryStr .= ' DURING '.$dateRange;
$csv_array = NULL;
$i = 0;
while(!$csv_array){
if($i++>2){
break;
}
$csv_array = checkReport($filePath,$queryStr);
var_dump(checkReport($filePath,$queryStr));
}
Trying to prevent a file from being duplicated in a users directory. This is what I have come up with,
$data = $connection->getMedia($fileid);
/**
* The test begins
*/
$dir = 'media/download/'.$username;
$files1 = scandir($dir);
foreach ($files1 as $i => $file) {
if (md5($file) == md5($data)) {
//statement
echo "Duplicate entry detected.";
}
else {
file_put_contents($finalPath, $data);
}
This, however, gives me an endless loop and I cant see what it is doing. So, can anyone give me a better idea.?
EDIT: the full code:
foreach ($accounts as $username => $password) {
$connection= new Connection($username,$password);
$fileList= $connection->getFiles();
echo '<br><p>Last available files for '.$username.': <br>';
if (!file_exists('media/download/'.$username)) {
mkdir('media/download/'.$username, 0777, true);
}
for ($i=0;$i<7;$i++)
{
$fileID= $fileList[$i]->id;
$data = $connection->getMedia($fileID);
if (!strlen($data) < 1 ) {
$recipient = $fileList[$i]->recipient;
$finalPath = 'media/download/'.$username.'/to'.$recipient.'_id'.$fildID.rand(0,200).'.jpg';
file_put_contents($finalPath, $data);
echo " <img src='".$finalPath."' height='100' width='100'> ";
}
}
}
i found a very good PHP solution here for compareing 2 CSV Files.
The Only Problem is the Result Page..
index.php?f1=re1.csv&f2=re2.csv
<?php
//---- init
$strFileName1=isset($_REQUEST['f1'])?$_REQUEST['f1']:'';
$strFileName2=isset($_REQUEST['f2'])?$_REQUEST['f2']:'';
if ( !$strFileName1 ) { die("I need the first file (f1)"); }
if ( !$strFileName2 ) { die("I need the second file (f2)"); }
try {
$arrFile1 = parseData($strFileName1);
$arrFile2 = parseData($strFileName2);
} catch (Exception $e) {
die($e->getMessage());
}
$rowCount1=count($arrFile1);
$rowCount2=count($arrFile2);
$colCount1=count($arrFile1[0]);
$colCount2=count($arrFile2[0]);
$highestRowCount = $rowCount1>$rowCount2 ? $rowCount1:$rowCount2;
$highestColCount = $colCount1>$colCount2 ? $colCount1:$colCount2;
$row = 0;
$err = 0;
//---- code
echo "<h2>Vergleich $strFileName1 and $strFileName2</h2>";
echo "\n<table border=1>";
echo "\n<tr><th>Err<th>Tabelle<th>Row#<th>Col#<th>Data in $strFileName1<th>Data in $strFileName2";
while($row<$highestRowCount) {
if(!isset($arrFile1[$row])) {
echo "\n<tr><td>Row missing in $strFileName1<th>$row";
$err++;
} elseif(!isset($arrFile1[$row])) {
echo "\n<tr><td>Row missing in $strFileName2<th>$row";
$err++;
} else {
$col=0;
while($col<$highestColCount) {
if ( !isset($arrFile1[$row][$col]) ) {
echo "\n<tr><td>Data missing in $strFileName1<td>$row<td>$col<td><td>".htmlentities($arrFile2[$row][$col]);
$err++;
} elseif ( !isset($arrFile2[$row][$col]) ) {
echo "\n<tr><td>Data missing in $strFileName1<td>$row<td>$col<td>".htmlentities($arrFile1[$row][$col]) ."<td>";
$err++;
} elseif ( $arrFile1[$row][$col] != $arrFile2[$row][$col] ) {
echo "\n<tr><td>Data mismatch";
echo "<td>Tabelle <td>$row <td>$col";
echo "<td>".htmlentities($arrFile1[$row][$col]);
echo "<td>".htmlentities($arrFile2[$row][$col]);
$err++;
}
$col++;
}
}
$row++;
}
echo "</table>";
if ( !$err ) {
echo "<br/>\n<br/>\nThe two csv data files seem identical<br/>\n";
} else {
echo "<br/>\n<br/>\nThere are $err differences";
}
//---- functions
function parseData($strFilename) {
$arrParsed = array();
$handle = fopen($strFilename , "r");
if ($handle) {
while (!feof($handle)) {
$data = fgetcsv($handle , 0 , ',' , '"' );
if ( empty($data)) continue; //empty row
$arrParsed[]=$data;
}
fclose($handle);
} else {
throw new Exception("File read error at $strFilename");
}
return $arrParsed;
}
?>
The Result displays the Row# and Col# of the file..
but what i need is the Name of the Col..
This is how my CSV looks like:
ISO_COUNTRY_CODE,CNT_POSTAL_CODE,ORDER1_ADMIN_TYP_1112,ORDER2_ADMIN_TYP_1113,ORDER8_ADMIN_TYP_1119,BUILTUP_ID
AND,7,0,0,7,40
ARG,1970,25,0,514,4076
AUS,2612,9,0,10978,4597
AUT,2213,9,95,2354,10304
BEL,1148,3,11,589,3781
BGR,4507,28,0,264,5375
I want to see in which "ISO_COUNTRY_CODE" is the CNT_POSTAL_CODE wrong..
$rowname = $row; // here i want to see the RAW of the Table
"like AND or ARG etc"
$colname = $col; // here i want to see the COL of the Table
"like CNT_POSTAL_CODE
or ORDER1_ADMIN_TYP_1112 etc "