Generate multiple CSV files - mysql PHP - php

I have made this PHP script that should take an array and for each element in the array - generate a csv file. Unfortunately something is wrong. It doesn't store any of the files in the directory specified. But it doesn't return any errors neither. Maybe someone can see the problem?
$ids = json_decode($_POST['jsonarray']); // array sent with ajax
$start = $_POST['start']; // date sent with ajax
$end = $_POST['end']; // date sent with ajax
$start_date = date('yyyy-mm-dd', strtotime($start)); // format dates to sql firendly
$end_date = date('yyyy-mm-dd', strtotime($end));
$toZip = array(); // Prepare array to files for zip
if(is_array($ids)) {
foreach ($ids as $key => $qr)
{
// Get labels first
// Here we prepare the first line in the .CSV file
$tb = $qr . '_labels';
$sql = $user_pdo->query("SELECT * FROM $tb");
$head_array = array('Log ID', 'Timestamp');
while ($row = $sql->fetch(PDO::FETCH_ASSOC))
{
// This array is the first line in the .CSV file
$head_array[] = $row['label'];
}
// Get ready for looping through the database
$table = $qr . '_data';
$results = $user_pdo->prepare("SELECT * FROM $table WHERE timestamp BETWEEN :start_date AND :end_date;");
$results->bindParam(':start_date', $start_date, PDO::PARAM_STR);
$results->bindParam(':end_date', $$end_date, PDO::PARAM_STR);
$results->execute();
// Pick a filename and destination directory for the file
$filename = "temp/db_user_export_".time().".csv";
// Actually create the file
// The w+ parameter will wipe out and overwrite any existing file with the same name
$handle = fopen($filename, 'w+');
// Write the spreadsheet column titles / labels
fputcsv($handle, $head_array);
// Write all the user records to the spreadsheet
foreach($results as $row)
{
// amount of rows is unknown
$rows = $row->rowCount();
$insert_array = array();
for ($i=0; $i<=$rows; $i++)
{
// function goes here
$insert_array[] = $row[$i];
}
fputcsv($handle, $insert_array);
}
// Finish writing the file
fclose($handle);
$toZip[] = $filename;
}
}
Example on var_dump($ids);
array(4) {
[0]=>
string(5) "t23ry"
[1]=>
string(5) "6us32"
[2]=>
string(5) "se43z"
[3]=>
string(5) "o00gq"
}

I found the answer. After a long time searching and playing around, I saw that this function
foreach($results as $row)
{
// amount of rows is unknown
$rows = $row->rowCount();
$insert_array = array();
for ($i=0; $i<=$rows; $i++)
{
// function goes here
$insert_array[] = $row[$i];
}
fputcsv($handle, $insert_array);
}
didn't work because of following:
$rows = $row->rowCount(); has to be $rows = count($row);
The number of string in the returned $row array was higher than expected so I needed to change my select statement to $results = $user_pdo->query("SELECT * FROM $table WHERE timestamp >= '$start' AND timestamp <= '$end'";, PDO::FETCH_NUM);. This will only give me the rows in numeric order, which will make the $row[$i] -> array work.
Also as you can see, I changed the prepared statement to a query instead, and also changes the start date and end date variables to be unformatted.
This really took some time, but it is finally working. Thanks a lot for all the support guys.

fputcsv only outputs a line at a time. Change this:
for ($i=0; $i<=$rows; $i++)
{
// function goes here
$insert_array[] = $row[$i];
}
fputcsv($handle, $insert_array);
To this:
for ($i=0; $i<=$rows; $i++)
{
// function goes here
fputcsv($handle, $row[$i]);
}

Related

Check if all values in array are in database and store ones that aren't in new array [duplicate]

This question already has answers here:
Check if all values in an array exist in a database column
(2 answers)
Closed 1 year ago.
I have an array that looks something like this -> ["john.smith#gmail.com", "jane.doe#gmail.com", "jack.smith#gmail.com"]. I want to increment $count for each email that exists in the database. If it doesn't exist (invalid), then I want to push it to the $invalidEmails array.
After that, I want to set my $output according to whether all the emails in the original array are valid or not. They're valid if all of them exist in the database. I'd appreciate some help with this as I'm not sure how to go about it from here. It doesn't work for all cases right now, for example if first email is valid but second one is invalid.
This is what I have so far:
$result = $conn->query("SELECT mail FROM dej_colleagues");
$rows = mysqli_fetch_all($result, MYSQL_ASSOC);
$tags = preg_split("/\,/", $_POST['tags']);
$invalidEmails = array();
$count = 0;
for ($i = 0; $i < sizeof($tags); $i++) {
$trim_brackets = trim($tags[$i], '[]');
$trim_quotes = trim($trim_brackets, '"');
foreach($rows as $row) {
if ($trim_quotes == $row["mail"]) {
$count += 1;
break;
}
}
if ($count == 0) {
array_push($invalidEmails, $tags[$i]);
}
}
$output = array();
if (sizeof($tags) == $count) {
$output = array("validity => "valid emails");
}
else {
$output = array("validity" => "invalid emails", "emails" => $invalidEmails;
}
echo json_encode($output);
Your code seems convoluted, so rather than debug it I started with a more focussed query and worked from there.
Basically, the query asks the database for a list of emails that appear in your $tags array, then uses array_diff() to find any that appear in $tags, but not in the database.
From there you can produce your output directly.
ini_set('display_errors',1);
$mysqli = new mysqli('mysql.lv.local','userName', 'userPassword','schemaName' );
// Assuming the input is a string and not an array, json_decode it.
$tags = '["john.smith#gmail.com", "Jane.doe#gmail.com", "jack.smith#gmail.com","fred.jones#gmail.com"]';
$tags = json_decode($tags);
// switch everything to lower case
$tags = array_map('strtolower', $tags);
// Build and prepare a query with placeholders. Note conversion to lower case
$sql = 'select distinct lower(`mail`) from `emails` where lower(`mail`) in (?'.str_repeat(',?', count($tags)-1).')';
//echo $sql;
$stmt = $mysqli->prepare($sql);
// Bind the values from $tags to the query
$stmt->bind_param(str_repeat('s', count($tags)), ...$tags);
// Execute
$stmt->execute();
// Bind a variable for the result
$stmt->bind_result($email);
// Retrieve the emails in to $dbMails
$dbMails = [];
while ($stmt->fetch()) {
$dbMails[] = $email;
}
//var_dump($dbMails);
// Anything that's in $tags but not in $dbMails is invalid
$absentEmails = array_diff($tags, $dbMails);
//var_dump($absentEmails);
if ($absentEmails) {
$op= ["validity"=>"Invalid enails", 'emails'=>array_values($absentEmails)];
} else {
$op= ["validity"=>"Valid enails"];
}
echo json_encode($op);

create multiple arrays from csv file using PHP

I have csv file with 1500+ entries in a column.I can able to read csv file's all values of column with this.
$rowcount = 1;
$srcFileName = "input/test.csv";
$file = fopen($srcFileName,"r");
$inputfielscount = count(file($srcFileName, FILE_SKIP_EMPTY_LINES));
while($rowcount < $inputfielscount)
{
$row = fgetcsv($file);
$result=array("id" =>$row[0],"des"=>"I am jhon",salery="10000");
$Final=array("listingsEmp"=>$result);
}
After reading first (1-10) value i will create an array (like array [0] =>$result) and Then wantto repeat same task from (11-20) and create another array (like array [1] =>$Final this time $final array contain information about the next ids whic we read from csv file (11-10)) and so on.
For the above requirment i changed code to this :
$rowcount = 1;
$srcFileName = "input/test.csv";
$file = fopen($srcFileName,"r");
while($rowcount < 20)
{
if(($rowcount % 10 == 0) && ( $rowcount != 0)) {
$rowcount++;
break;
}else{
$row = fgetcsv($file);
// some curl code for fetching data according to csv file field(Id)
$result=array("id" =>$row[0],"des"=>"I am jhon",salery="10000"); //contain 10 array
}
}
$Final=array("listingsEmp"=>$result);
Now i will post this $final array which has (0-10 index array ,each has unique id and corresponding values) using curl and get response which i am save in csv file.
$currenttime=date("Y-m-d-H_i_s");
$opfile='output'.$currenttime.'.csv'; //path wher op csv file exist
if(!#copy($srcFileName,'/output/'.$opfile))
{
$errors= error_get_last();
echo "COPY ERROR: ".$errors['type'];
echo "<br />\n".$errors['message'];
}else { // echo "File copied from remote!";
$fp = fopen('output/output'.$currenttime.'.csv',"a");
$fr = fopen($srcFileName,"r");
$rowcounts=0;
$FinalRES=$Final->response;
while($rowcounts< $inputfielscount) {
$resultBulk=$FinalRES[$rowcounts];
$resultBulkStatus=$FinalRES->status;
$resultBulkErrors=$FinalRES->errors;
$errorMsgArray=$resultBulkErrors[0];
$BulkErrorsMessage=$errorMsgArray->message;
$rows = fgetcsv($fr);
if($resultBulkStatus=='failure'){
$list = array ($rows[0],$rows[1],$resultBulkStatus,$BulkErrorsMessage);
}else {
$list = array ($rows[0],$rows[1],$resultBulkStatus,"successfully");
}
fputcsv($fp,$list);
//$p++;
$rowcounts++;
}
}
This full code runs once and give response for 10 ids ,i want repeat this code again for next 10 id (11-20)and then for (21-30) so on .
Once all response write in output csv file After that it display download output file link,Output file contain full response for all Ids which is in csv file(1500 +)
<?php $dnldfilw='output'.$currenttime.'.csv';?>
<a href='download.php?filename=<?php echo $dnldfilw; ?>'>Download Output file</a>
?>
The easiest method is to just use the file() function you are already using...
So to shorten the code to some pseudocode:
<?php
$indexedArray = array();
$indexedSplit = 10;
$lines = file($srcFileName);
$tempArray = array();
foreach($lines as $line) {
if(count($tempArray) % $indexedSplit === 0) {
$indexedArray[] = $tempArray;
$tempArray = array();
}
$tempArray[] = $line;
}
foreach($indexedArray as $index => $valueArray) {
// do the curl magic
// write results of curl into csv
}
Your question is poorly phrased, but I think this would be your aim, right?

Checking while loop key exists in seperate array

Having checked a variety of questions but not being able to find quite what I need, I am at a bit of a loss.
I am trying to chose the columns from MySQL I want exported to CSV by parsing the column names and adding the valid column names to a $colnames array, then adding those values as headers to the CSV and then only displaying the relevant data from the database through a while loop.
I have looked at the following in particular having been guided there from other questions: How to get all the key in multi-dimensional array in php
Here is the code:
function query_to_csv($query, $filename, $attachment = false, $headers = true, $specs_off = false) {
if($attachment) {
// send response headers to the browser
header( 'Content-Type: text/csv; charset=UTF-8' );
header( 'Content-Disposition: attachment;filename='.$filename);
$fp = fopen('php://output', 'w');
} else {
$fp = fopen($filename, 'w');
}
$result = mysql_query($query) or die( mysql_error() );
if($headers) {
// output header row (if at least one row exists)
$row = mysql_fetch_assoc($result);
if($row) {
// PRODUCTS TABLE SPECIFIC - get rid of specs_ and free_ columns so have nicer data set for user
if($specs_off) {
$columnames = array_keys($row);
$colnames = array();
//$colnames = array_keys($row);
foreach($columnames as $key => $value) {
if((substr_count($value, "spec_") < 1) && (substr_count($value, "free_") < 1)) {
array_push($colnames, $value);
}
}
}
else {
$colnames = array_keys($row);
}
// add in additional columns if exporting client data
if($table == 'clients') {array_push($colnames, "products", "last_order_date");}
//write the colnames to the csv file
fputcsv($fp, $colnames);
// reset pointer back to beginning
mysql_data_seek($result, 0);
}
} // done with the headers etc, now lets get on with the data
// clear out and create the $row_data array
$row_data = array();
// run through the row array adding values to row_data as we go
while($row = mysql_fetch_assoc($result)) {
// create the array_keys_multi from https://stackoverflow.com/questions/11234852/how-to-get-all-the-key-in-multi-dimensional-array-in-php/11234924#11234924
function array_keys_multi(array $array) {
$keys = array();
foreach ($array as $key => $value) {
$keys[] = $key;
if (is_array($array[$key])) {
$keys = array_merge($keys, array_keys_multi($array[$key]));
}
}
return $keys;
}
// run the function on the $row array
array_keys_multi($row);
// now use the $keys array
foreach($keys as $key => $value) {
// check if the value is in the colnames array and if so push the data on to the $row_data array ready for export to CSV
if(in_array($value, $colnames)) {
array_push($row_data, $row[$value]);
}
}
// now we are ready to write the CSV
fputcsv($fp, $row_data);
}
fclose($fp);
exit;
} // end of query_to_csv
// Write the sql statement
$sql = "SELECT * FROM ".$table." ";
if(isset($where_1_col)) { $sql .= " WHERE `".$where_1_col."` = '".$where_1_val."'"; }
if(isset($where_2_col)) { $sql .= " AND `".$where_2_col."` = '".$where_2_val."'"; }
if(isset($where_3_col)) { $sql .= " AND `".$where_3_col."` = '".$where_3_val."'"; }
if(isset($where_4_col)) { $sql .= " AND `".$where_4_col."` = '".$where_4_val."'"; }
if(isset($order_by_col)) { $sql .= " ORDER BY `".$order_by_col."` ". strtoupper($order_by_dir) ." "; }
// output as an attachment
query_to_csv($sql, $table."_export.csv", true, true, true);
All I am getting is a huge export of the chosen column names repeated as many times as there are values from the initial query. I don't know how to get the values in.
Any suggestions on where I am going wrong or how I can undertake this more neatly are welcomed.
It seems that you just append the new row data to $row_data but never clear that array.
array_push($row_data, $row[$value]);
What I did to fix it:
Move
// clear out and create the $row_data array
$row_data = array();
into the while loop.
Change
// clear out and create the $row_data array
$row_data = array();
while($row = mysql_fetch_assoc($result)) {
...
}
To
while($row = mysql_fetch_assoc($result)) {
// clear out and create the $row_data array
$row_data = array();
...
}
Note:
You are using $table everywhere but never define it. Like here if($table == 'clients')
If it is a global var you need to add global $table or a parameter to your function, too.
Edit:
As mentioned in my comment on your question you could just use array_keys() to get the keys.
php.net/manual/de/function.array-keys.php
And then change
array_keys_multi($row);
to
$keys = array_keys($row);
After that you can remove array_keys_multi()
Further you could move that part in front of your while-loop because you only need to calculate the column names you need once and not in every iteration.

php output txt files with record limits plus grouped by unique county names

When I ouput these txt files, I am trying to group them by unique county with a count limitation per county file. For example, let's say the query returns 2 unique counties in this accessable result field: $row['county_txt'].. Let's say I set the $per_file limitation to 2500. I have the script working now with the per_file etc but not with the counties grouping. Below is somewhat of a mash of where I am at. Thanks for any guidance in helping me resolve this.
Output examples:
Green County - Total Green county results 2900 output would be 2 files.
Output files would be:
Green-#1-20130627-2500.txt
Green-#2-20130627-400.txt
Red County - Total Red county results 12650 output would be 5 files.
Output files would be:
Red-#1-20130627-2500.txt
Red-#2-20130627-2500.txt
Red-#3-20130627-2500.txt
Red-#4-20130627-2500.txt
Red-#5-20130627-150.txt
... // earlier part of script
// Functions I've been attempting
$county[] = $row['county_txt'];
function unique_county() {
foreach($county as $unq_cnty) {
echo $unq_cnty;
return $unq_cnty;
}
}
function get_unique_county() {
$column = array();
while($row = mysql_fetch_array($result)){
$column[] = array_unique($row['county_txt']);
echo $column;
}
}
get_unique_county();
$file_count = 1;
$recs = 0;
$per_file = 2500;
$footer = "FOOTER";
$default_contents = $contents = array("BODY CONTENT TOP");
while ($row = mysql_fetch_array($result)) {
$line = "...";
$contents[] = $line; // Each array element will be a line in the text file
$i++;
$recs++;
if ($county == $unq_cnty && $i == $per_file) {
$contents[] = $footer; // Add the footer to the end
file_put_contents($unq_county . "-#" . $file_count . "-" . date('Y') . "-" . $recs . '.txt', implode("\r\n", $contents));
$i = 0;
$recs = 0;
$contents = $default_contents;
$file_count++;
} // End of if()
} // End of while()
You need a counter, and then be able to reset it (upon resetting it, you save the file).
Example (untested, example only):
<?php
$rowCounter = 0;
$fileCounter = 1;
$startID = md5(microtime(1));
$fp = fopen("{$startID}.txt", "w");
while ($row = mysql_fetch_array($result)) {
$rowCounter++;
fwrite($fp, $row['county_txt']."\r\n");
if($rowCounter == 2500) {
fclose($fp);
if($startID) {
rename("{$startID}.txt", "Red-#{$fileCounter}-".date("Ymd")."-{$rowCounter}.txt");
$startID = md5(microtime(1));
}
$fp = fopen("{$startID}.txt", "w");
$rowCounter = 0;
$fileCounter++;
}
}
// Save last file
fclose($fp);
rename("{$startID}.txt", "Red-#{$fileCounter}-".date("Ymd")."-{$rowCounter}.txt");
?>
On that note, don't use mysql_* functions. Instead, use mysqli at the very least, or PDO.
Not really sure what you are trying to do here, but it seems you are making things way harder than need be. In essence, it seems that you need to work with a two-dimensional array. So why not just query the database and read the data into a 2-D array right off the bat rather than jump through all these extra hoops (i.e. functions to determine unique array values and such)?
So you code might look something like this:
$county_array = array()
while ($row = [YOUR DATABASE ROW FETCHING MECHANISM HERE]) {
$county_array[$row['county_name']][] = $row; // you can change $row here to whatever data you actually need to store.
}
$limit = 2500;
foreach ($county_array as $county_name => $county_array) {
$temp_array = array();
$i = 0;
foreach ($county_array as $item) {
$temp_array[] = $item;
$i++;
if ($i === $limit) {
// we reached file limit, so write it to file code omitted for this
$temp_array = array();
$i = 0;
}
}
if (count($temp_array) > 0) {
// there are still items in temp array so write them to file code omitted for this
}
}
If you actually order by country name in your query and detect for changes to the value when reading county names out (and thus starting a new file), you could actually write directly into files in your loop that reads from the DB saving yourself memory overhead.

Export Gmail Contacts to CSV using PHP

I'm trying to write a PHP script to backup my Gmail contacts.
I found an article which described using the Zend framework in combination with the Google Contacts API in order to query contacts, I managed to get it working however the amount of information returned is far from adequate.
Here is the article: http://www.ibm.com/developerworks/opensource/library/x-phpgooglecontact/index.html
And here is my code:
$fp = fopen('file.csv', 'w');
foreach ($results as $r) {
$master = array();
$master[0] = (string) $r->name;
$master[1] = (string) $r->orgName;
$master[2] = (string) $r->orgTitle;
$iCount = 2;
foreach($r->phoneNumber as $p) {
$iCount += 1;
$master[$iCount] = (string) $p->phoneNumber;
}
fputcsv($fp, $master);
}
fclose($fp)
Here is the output from var_dump():
object(stdClass)#7 (5)
{
["name"] => string(17) "John Doe"
["orgName"] => string(6) "Some Org"
["orgTitle"] => string(0) ""
["emailAddress"] => array(1)
{
[0]=> string(17) "user#domain.com"
}
["phoneNumber"] => array(2)
{
[0] => string(3) "123"
[1]=> string(3) "321"
}
}
Try this code:
$csvFile = 'file.csv';
// Open the CSV file for writing
if (!$fp = fopen($csvFile, 'w')) {
exit("Unable to open '$csvFile' for writing");
}
// Loop results
foreach ($results as $r) {
// Build base array
$item = array($r->name, $r->orgName, $r->orgTitle);
// Add phone numbers to array
$item = array_merge($item, $r->phoneNumber);
// Write to CSV file
fputcsv($fp, $item);
}
fclose($fp);
This code does not add the email addresses to the file, because you have not used them in your code, but it could easily be added by changing the array_merge() line to this:
$item = array_merge($item, $r->phoneNumber, $r->emailAddress);
This would result in the email addresses appearing at the end of each row. To have them appear somewhere else, you just need to change the order in which you supply the arguments to array_merge().
HOWEVER...
The code above, based on your code, will result in a CSV file that will be difficult to parse. This is because the contact can have a varying number of phone numbers and emails addresses. A CSV file should be a table, with well defined columns and the same number of columns in each row. For this reason, you would be better doing something like this:
N.B. This solution loops the data twice in order to dynamically construct the column layout. This will be a slower solution and it could be speeded up by rigidly defining the column layout, but this will potentially result in either too many columns, some with empty data, or not enough columns and some data being lost.
$csvFile = 'file.csv';
// Loop the data to construct the maximum number of emails and telephone numbers
$numTels = $numEmails = 0;
foreach ($results as $r) {
if (count($r->phoneNumber) > $numTels) $numTels = count($r->phoneNumber);
if (count($r->emailAddress) > $numEmails) $numEmails = count($r->emailAddress);
}
// Open the CSV file for writing
if (!$fp = fopen($csvFile, 'w')) {
exit("Unable to open '$csvFile' for writing");
}
// Construct the column headers row and write to file
$colHeaders = "name,orgname,orgtitle";
for ($i = 0; $i < $numTels; $i++) $colHeaders = ",tel_$i";
for ($i = 0; $i < $numEmails; $i++) $colHeaders = ",email_$i";
fwrite($fp, "$colHeaders\n");
// Construct and write rows to file
foreach ($results as $r) {
$item = array($r->name, $r->orgName, $r->orgTitle);
for ($i = 0; $i < $numTels; $i++) $item[] = (isset($r->phoneNumber[$i])) ? $r->phoneNumber[$i] : '';
for ($i = 0; $i < $numEmails; $i++) $item[] = (isset($r->emailAddress[$i])) ? $r->emailAddress[$i] : '';
fputcsv($fp, $item);
}
fclose($fp);

Categories