Getting data from mysql and creating a csv file issue - php

I want to extract some data and create a csv file. This is how my attributes table looks like. you can see it in the picture:
HERE
This is my code, what i tried:
$select = mysql_query("select * from attributes");
$final ="";
$i = 0;
while($row = mysql_fetch_array($select)) {
$id_produs = $row['id_produs'];
$n_attr = $row['nume_attr'];
$val = $row['val_attr'];
$final .= $id_produs . "%%" . $val . "%%";
$csv_array_attributes[$i] = $final;
$i++;
}
This is the part when i create the CSV file:
$file = fopen("attributes.csv", "w+");
foreach ($csv_array_attributes as $line) {
fputcsv($file, explode('%%', $line), "^", "`");
}
fclose($file);
AND this would be the wanted result: HERE. So, the nume_attr should be the header for every column in the csv and the val_attr should be the values for every nume_attr (see the image).
I tried many ways but I don't get this result from the image. I have to mention that my table "attributes" has more than 74000 rows.
This is the table structure:
CREATE TABLE `attributes` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`id_produs` text NOT NULL,
`nume_attr` text NOT NULL,
`val_attr` text NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=673912 DEFAULT CHARSET=latin1;

/*Try following code */
$select = mysql_query("select * from attributes");
$i = 0;
$csv_array_attributes[$i]=array('sku','manufacturer_article','manufacturer');
$i++;
while($row = mysql_fetch_array($select)) {
$csv_array_attributes[$i] =array('sku'=>$row['id_produs'],'manufacturer_article'=>$row['nume_attr'],'manufacturer'=>$row['val_attr']);
$i++;
}
$file = fopen("attributes.csv", "w+");
foreach ($csv_array_attributes as $line) {
fputcsv($file,$line);
}
fclose($file);

Let's assume speed isn't the issue here and neither is concurrency, i.e. you just want the result now no matter how good, bad or ugly the solution is ;-)
First query all the attribute names via SELECT DISTINCT nume_attr FROM attributes
Store those names as keys of an array with empty string values, i.e. you should have an array like
$template = array(
'Manufacturer Article' => '',
'Manufacturer' => '',
...
)
Make a variable that stores the currently processed id_produs (start with false).
Make a variable that stores the current "csv"-record ( again start with false )
Query all records ordered by id_produs and iteratore over the result.
If the current record has a different id_produs than the "currently processed" write out the current "csv"-array and start anew with the template array created in the first step and the current id_produs as ...the current id_produs. For each record write the val_attr value to the key nume_attr in the csv array.
You will have to check/write the last record after the iteration.

/* try following code,and reply if it will helpful to you or not */
$select = mysql_query("select * from attributes");
$i = 0;
$i++;
while($row = mysql_fetch_array($select)) {
$csv_array_attributes[$row['id_produs']][$row['nume_attr']] =$row['val_attr'];
$i++;
}
$arraytempheader=array();
$arraytemp=array();
$arraytempheader[0]="";
$j=1;
foreach ($csv_array_attributes as $Colname=>$alline) {
if($j!=1)
{
foreach($arraytempheader as $key=>$val)
{
if($key!=$j && $key!=0)
$arraytemp[$Colname][$key]="";
}
}
foreach ($alline as $keyline=>$valline)
{
$arraytempheader[$j]=$keyline;
$arraytemp[$Colname][$j]=$valline;
$j++;
}
}
$file = fopen("attributes.csv", "w+");
fputcsv($file,$arraytempheader);
foreach ($arraytemp as $Colname=>$alline) {
$finalArray=array();
$finalArray[]=$Colname;
$finalArray=array_merge($finalArray,$alline);
fputcsv($file,$finalArray);
}
fclose($file);

Try to fix this in other way.
Here you select the attributes and store them into array
$select = mysql_query("select * from attributes");
$rezultat=array();
while($row = mysql_fetch_array($select)) {
$id_produs = $row['id_produs'];
$n_attr = $row['nume_attr'];
$val = $row['val_attr'];
//$final .= $id_produs . "%%" . $val . "%%";
$rezultat[] = array('id'=>$id_produs,'attr'=>$val);
}
And here you write them to the file.
$file = fopen("attributes.csv", "w+");
foreach ($rezultat as $line) {
fputcsv($file, $line, "^", "`");
}
fclose($file);

Related

exporting a mysql query in php version 4 to a csv

Edit on my original post. I found the answer!!!! with help:)
I now have this working by using the below code with thanks for the advice on this in the comments:
<?php
$f = fopen('incident_csv\test.csv', 'w');
$query = "
select column1, column2, column3
from table
where columns = values
";
$var1 = mysql_query($query, $database connection variable);
/* From Monkey Zeus */
$csv_lines = array();
// Loop your records from the DB
while ($row = mysql_fetch_assoc($var1)){
$columns = array();
// Loop each column for the row
foreach($row as $k=>$v){
// Surround column item in double-quotes and escape double-quotes with double-double-quotes
$columns[] = '"'.str_replace('"', '""', $v).'"';
}
// Join on a comma
$csv_lines[] = implode(',', $columns);
}
// Create full CSV file by joining on a newline
$csv_file_string = implode("\n", $csv_lines);
/* From Monkey Zeus */
fwrite($f, $csv_file_string);
?>
You can do this:
$csv_lines = array();
// Loop your records from the DB
while ($row = mysql_fetch_assoc($var1)){
$columns = array();
// Loop each column for the row
foreach($row as $k=>$v){
// Surround column item in double-quotes and escape double-quotes with double-double-quotes
$columns[] = '"'.str_replace('"', '""', $v).'"';
}
// Join on a comma
$csv_lines[] = implode(',', $columns);
// Write to the file right away. You are using PHP4 so I imagine system memory is not plentiful
// If you use this then there is no need for the "$csv_lines[] =" from above
// nor the $csv_file_string after the while loop
// fwrite($f, implode(',', $columns)."\n");
}
// fclose($f); // If you choose to write to the file during the while loop then close the file handle when you are finished
// Create full CSV file by joining on a newline
$csv_file_string = implode("\n", $csv_lines);
You can simply wite an arrayToCsv function:
function arrayToCsv($array) {
$string = array();
foreach ($array as $field) {
$string[] = implode(';', $field);
}
return implode("\n", $string);
}
$a = array(
array('First Field of First Line', 'Second Field of First Line'),
array('First Field of Second Line', 'Second Field of Second Line')
);
fwrite($f, arrayToCsv($a));
But, remember to have your $f setted from $f = fopen('file location', 'w');
Hope I was helpful.

How to create custom CSV files using PHP?

In the $csv_array_attributes[0] represents every column for my csv file. sku is the product id. In $csv_array_attributes[0] represents all of the product ids for my csv(which is the rows value form column sku). In the query below I extract all of the products ids, attributes names and attributes values. My problem is that I have to compute my $csv array in order to assign for each attribute name(columns) the attribute values. In $final_string somehow I want to memorize all of the attributes values for a products (see the commented part). Thx in advance for the help. Im really stuck with this :(
$csv_array_attributes[0] = "sku%%".$header;
//GETTING ATTRIBUTES VALUES
$i = 1;
foreach($xml->children() as $content){
$id_produs = $content->ProductCode;
$csv_array_attributes[$i] = $id_produs;
$i++;
}
$select = mysql_query("SELECT id_prod.id_produs, GROUP_CONCAT('^^',nume_attr) AS nume_at, GROUP_CONCAT('^^',val_attr) AS val_at FROM attributes
INNER JOIN id_prod
ON id_prod.id_id_produs = attributes.id_produs
GROUP BY id_prod.id_produs");
$i = 0; $id_prod = array();
while ($row = mysql_fetch_array($select)){
$id_produs = $row['id_produs'];
$nume_attr = explode(",^^",substr($row['nume_at'],2));
$val_attr = explode(",^^",substr($row['val_attr'],2));
$id_prod[$id_produs] = $nume_attr;
$i++;
}
// var_dump(count($id_prod));
$nume = explode("%%", $csv_array_attributes[0]);
$csv[0] = $csv_array_attributes[0];
$i = 1;
foreach ($id_prod as $key => $value) {
// comment part
//$final_string = "";
//if (attr_name has attribute_value for product i ){
// $final_string.= attribute_value."%%";
//}else{
// $final_string.= "%%";
//}
//end comment part
$csv[$i] = $csv_array_attributes[$i]."%%".$final_string;
$i++;
}
//var_dump($csv_array_attributes[0])
//CREARE CSV
$file = fopen("attributes.csv","w+");
foreach ($csv as $line){
fputcsv($file,explode('%%',$line),"^","`");
}
To see the current result please click HERE
Actually I only can give you a hint how I would create a array for csv.
// header
$csv[0][0] = "one";
$csv[0][1] = "two";
$csv[0][2] = "three";
$csv[0][3] = "four";
$csv[0][4] = "five";
// body
$csv[1][0] = "some";
$csv[1][1] = "values";
$csv[1][2] = "that";
$csv[1][3] = "could";
$csv[1][4] = "be";
$csv[2][0] = "here";
$csv[2][1] = "in";
$csv[2][2] = "this";
$csv[2][3] = "little";
$csv[2][4] = "array";
This is only a sample to view the array setup.
If you have set up the header, lets say by a for you can fill the 2 demensional array.
If this is set up you can use a foreach to build your CSV
Sample
$seperator = ";";
$ln = "\r\n";
$csv_output = "";
foreach($csv as $c){
foreach($c as $value){
$csv_output .= $value . $seperator;
}
$csv_output . $ln;
}
csv output
one;two;three;four;five;
some;values;that;could;be;
here;in;this;little;array;
a row;with an;;empty;field;
empty fields are no problem since you seperate them all with a ;

Checking while loop key exists in seperate array

Having checked a variety of questions but not being able to find quite what I need, I am at a bit of a loss.
I am trying to chose the columns from MySQL I want exported to CSV by parsing the column names and adding the valid column names to a $colnames array, then adding those values as headers to the CSV and then only displaying the relevant data from the database through a while loop.
I have looked at the following in particular having been guided there from other questions: How to get all the key in multi-dimensional array in php
Here is the code:
function query_to_csv($query, $filename, $attachment = false, $headers = true, $specs_off = false) {
if($attachment) {
// send response headers to the browser
header( 'Content-Type: text/csv; charset=UTF-8' );
header( 'Content-Disposition: attachment;filename='.$filename);
$fp = fopen('php://output', 'w');
} else {
$fp = fopen($filename, 'w');
}
$result = mysql_query($query) or die( mysql_error() );
if($headers) {
// output header row (if at least one row exists)
$row = mysql_fetch_assoc($result);
if($row) {
// PRODUCTS TABLE SPECIFIC - get rid of specs_ and free_ columns so have nicer data set for user
if($specs_off) {
$columnames = array_keys($row);
$colnames = array();
//$colnames = array_keys($row);
foreach($columnames as $key => $value) {
if((substr_count($value, "spec_") < 1) && (substr_count($value, "free_") < 1)) {
array_push($colnames, $value);
}
}
}
else {
$colnames = array_keys($row);
}
// add in additional columns if exporting client data
if($table == 'clients') {array_push($colnames, "products", "last_order_date");}
//write the colnames to the csv file
fputcsv($fp, $colnames);
// reset pointer back to beginning
mysql_data_seek($result, 0);
}
} // done with the headers etc, now lets get on with the data
// clear out and create the $row_data array
$row_data = array();
// run through the row array adding values to row_data as we go
while($row = mysql_fetch_assoc($result)) {
// create the array_keys_multi from https://stackoverflow.com/questions/11234852/how-to-get-all-the-key-in-multi-dimensional-array-in-php/11234924#11234924
function array_keys_multi(array $array) {
$keys = array();
foreach ($array as $key => $value) {
$keys[] = $key;
if (is_array($array[$key])) {
$keys = array_merge($keys, array_keys_multi($array[$key]));
}
}
return $keys;
}
// run the function on the $row array
array_keys_multi($row);
// now use the $keys array
foreach($keys as $key => $value) {
// check if the value is in the colnames array and if so push the data on to the $row_data array ready for export to CSV
if(in_array($value, $colnames)) {
array_push($row_data, $row[$value]);
}
}
// now we are ready to write the CSV
fputcsv($fp, $row_data);
}
fclose($fp);
exit;
} // end of query_to_csv
// Write the sql statement
$sql = "SELECT * FROM ".$table." ";
if(isset($where_1_col)) { $sql .= " WHERE `".$where_1_col."` = '".$where_1_val."'"; }
if(isset($where_2_col)) { $sql .= " AND `".$where_2_col."` = '".$where_2_val."'"; }
if(isset($where_3_col)) { $sql .= " AND `".$where_3_col."` = '".$where_3_val."'"; }
if(isset($where_4_col)) { $sql .= " AND `".$where_4_col."` = '".$where_4_val."'"; }
if(isset($order_by_col)) { $sql .= " ORDER BY `".$order_by_col."` ". strtoupper($order_by_dir) ." "; }
// output as an attachment
query_to_csv($sql, $table."_export.csv", true, true, true);
All I am getting is a huge export of the chosen column names repeated as many times as there are values from the initial query. I don't know how to get the values in.
Any suggestions on where I am going wrong or how I can undertake this more neatly are welcomed.
It seems that you just append the new row data to $row_data but never clear that array.
array_push($row_data, $row[$value]);
What I did to fix it:
Move
// clear out and create the $row_data array
$row_data = array();
into the while loop.
Change
// clear out and create the $row_data array
$row_data = array();
while($row = mysql_fetch_assoc($result)) {
...
}
To
while($row = mysql_fetch_assoc($result)) {
// clear out and create the $row_data array
$row_data = array();
...
}
Note:
You are using $table everywhere but never define it. Like here if($table == 'clients')
If it is a global var you need to add global $table or a parameter to your function, too.
Edit:
As mentioned in my comment on your question you could just use array_keys() to get the keys.
php.net/manual/de/function.array-keys.php
And then change
array_keys_multi($row);
to
$keys = array_keys($row);
After that you can remove array_keys_multi()
Further you could move that part in front of your while-loop because you only need to calculate the column names you need once and not in every iteration.

php output txt files with record limits plus grouped by unique county names

When I ouput these txt files, I am trying to group them by unique county with a count limitation per county file. For example, let's say the query returns 2 unique counties in this accessable result field: $row['county_txt'].. Let's say I set the $per_file limitation to 2500. I have the script working now with the per_file etc but not with the counties grouping. Below is somewhat of a mash of where I am at. Thanks for any guidance in helping me resolve this.
Output examples:
Green County - Total Green county results 2900 output would be 2 files.
Output files would be:
Green-#1-20130627-2500.txt
Green-#2-20130627-400.txt
Red County - Total Red county results 12650 output would be 5 files.
Output files would be:
Red-#1-20130627-2500.txt
Red-#2-20130627-2500.txt
Red-#3-20130627-2500.txt
Red-#4-20130627-2500.txt
Red-#5-20130627-150.txt
... // earlier part of script
// Functions I've been attempting
$county[] = $row['county_txt'];
function unique_county() {
foreach($county as $unq_cnty) {
echo $unq_cnty;
return $unq_cnty;
}
}
function get_unique_county() {
$column = array();
while($row = mysql_fetch_array($result)){
$column[] = array_unique($row['county_txt']);
echo $column;
}
}
get_unique_county();
$file_count = 1;
$recs = 0;
$per_file = 2500;
$footer = "FOOTER";
$default_contents = $contents = array("BODY CONTENT TOP");
while ($row = mysql_fetch_array($result)) {
$line = "...";
$contents[] = $line; // Each array element will be a line in the text file
$i++;
$recs++;
if ($county == $unq_cnty && $i == $per_file) {
$contents[] = $footer; // Add the footer to the end
file_put_contents($unq_county . "-#" . $file_count . "-" . date('Y') . "-" . $recs . '.txt', implode("\r\n", $contents));
$i = 0;
$recs = 0;
$contents = $default_contents;
$file_count++;
} // End of if()
} // End of while()
You need a counter, and then be able to reset it (upon resetting it, you save the file).
Example (untested, example only):
<?php
$rowCounter = 0;
$fileCounter = 1;
$startID = md5(microtime(1));
$fp = fopen("{$startID}.txt", "w");
while ($row = mysql_fetch_array($result)) {
$rowCounter++;
fwrite($fp, $row['county_txt']."\r\n");
if($rowCounter == 2500) {
fclose($fp);
if($startID) {
rename("{$startID}.txt", "Red-#{$fileCounter}-".date("Ymd")."-{$rowCounter}.txt");
$startID = md5(microtime(1));
}
$fp = fopen("{$startID}.txt", "w");
$rowCounter = 0;
$fileCounter++;
}
}
// Save last file
fclose($fp);
rename("{$startID}.txt", "Red-#{$fileCounter}-".date("Ymd")."-{$rowCounter}.txt");
?>
On that note, don't use mysql_* functions. Instead, use mysqli at the very least, or PDO.
Not really sure what you are trying to do here, but it seems you are making things way harder than need be. In essence, it seems that you need to work with a two-dimensional array. So why not just query the database and read the data into a 2-D array right off the bat rather than jump through all these extra hoops (i.e. functions to determine unique array values and such)?
So you code might look something like this:
$county_array = array()
while ($row = [YOUR DATABASE ROW FETCHING MECHANISM HERE]) {
$county_array[$row['county_name']][] = $row; // you can change $row here to whatever data you actually need to store.
}
$limit = 2500;
foreach ($county_array as $county_name => $county_array) {
$temp_array = array();
$i = 0;
foreach ($county_array as $item) {
$temp_array[] = $item;
$i++;
if ($i === $limit) {
// we reached file limit, so write it to file code omitted for this
$temp_array = array();
$i = 0;
}
}
if (count($temp_array) > 0) {
// there are still items in temp array so write them to file code omitted for this
}
}
If you actually order by country name in your query and detect for changes to the value when reading county names out (and thus starting a new file), you could actually write directly into files in your loop that reads from the DB saving yourself memory overhead.

Insert many multi-level arrays into mysql base

i have a function what separate my file's content. Example file:
001;"text1"
002;"text2"
003;"text3"
999
001;"tekst11"
Sometimes some indexes have a few values:
120;"text2"
120;"text3"
My function:
function parseCSV($file) {
$lines = file($file);
$output = array();
$i = -1;
foreach($lines as $line) {
$line = trim($line);
if($line == "999") {
$i++;
} else {
$ex = explode(";", $line, 2);
$val = str_replace("\"", "", $ex[1]);
//$dodaj = mysql_query("INSERT INTO `czynsz` (`$ex[$i]`) VALUES ('$val')");
if(isset($output[$i][$ex[0]])) {
if(is_array($output[$i][$ex[0]])) {
$output[$i][$ex[0]][] = $val;
} else {
$output[$i][$ex[0]] = array($output[$i][$ex[0]], $val);
}
} else {
$output[$i][$ex[0]] = $val;
}
$dodaj = mysql_query("INSERT INTO `czynsz` (`$ex[$i]`) VALUES ('$val')");
}
}
return $output;
}
$csvdata = parseCSV("woda.txt");
echo "<pre>\r\n";
print_r($csvdata);
echo "</pre>\r\n";
Above script makes me multi-level array tree and separate arrays from themselves by separator (999), because in my file are many arrays... When an index has many values - the script makes for him internal tree. For example:
Array
(
[0] => Array
(
[001] => 06-001-03-13
[002] => 06447
[003] => F
...
[097] =>
[120] => Array
(
[0] => text1
[1] => text2
)
)
[1] => Array
...
And there is my question:
I have a problem with save this mega array in my MySQL base. Every index in array have the same in data base. When the same index will have many values (ex. 120) it can be implode. Thank you with all my heart for any form of help... :(
Create a MySQL Table:
CREATE TABLE `czynsz` (
`id` INT NOT NULL ,
`value` VARCHAR( 255 ) NOT NULL
) ENGINE = InnoDB;
ALTER TABLE `czynsz` ADD UNIQUE `unique` ( `id` )
Then use the following php code to insert the data
ini_set('error_reporting', E_ALL); // just for debugging remove later
ini_set('display_erorrs', 1); // just for debugging remove later
function parseCSV($file) {
$lines = file($file);
$output = array();
$i = 0;
foreach($lines as $line) {
$line = trim($line);
if($line == "999") {
$i++;
continue;
}
list($id, $val) = explode(";", $line, 2);
$val = str_replace("\"", "", $val);
$result = mysql_query("
INSERT INTO `czynsz` (
id,
`value`
) VALUES (
'$id',
'$val'
) ON DUPLICATE KEY UPDATE
`value` = CONCAT(`value`, ', ', '$val')"
);
if(!$result) {
die(mysql_error());
}
if(isset($output[$i][$id])) {
if(is_array($output[$i][$id])) {
$output[$i][$id][] = $val;
} else {
$output[$i][$id] = array($output[$i][$id], $val);
}
} else {
$output[$i][$id] = $val;
}
}
return $output;
}
Use the PHP sledgehammer solution...
Assuming that you just want to store the array and don't need to query against distinct nodes, use the serialize function to turn your array into a string representation. You can easily pull the data and then use unserialize to get back the original array.
$s_arr = serialize($my_array);
$my_array = unserialize($s_arr);
If you need access to the nodes in the database, then I think that MySQL is not a good solution because of the complexity involved. If you want a key-value store, then you should look at a NoSQL solution such as Redis.

Categories