export mysql table to csv with merged columns - php
I have a csv import export function where the database table has a category and a sub category column within the data structure.
The Database Fields are: 'SKU*', 'Name', 'Origin', 'Stock', 'Category', 'SubCategory', 'RRP', 'Price Enq', 'Description', 'Dimensions', 'SEO Meta Description', 'SEO Page Title', 'Priority'
I need to implode the values of each of these with a separator of >
so it would be Category>SubCategory in one column for both the heading line and the line values of each row.
now i know on an import from a csv i can run an explode to split the 2 columns out from the csv:
$temp = explode(">", strtoupper($data["productCategory"]));
$data["productCategory"] = trim($temp["0"]);
$data["productSubCategory"] = trim($temp["1"]);
but i cannot for the life of me work out how to take the 2 columns from the database and implode them correctly
id assumed that this would work:
// Merge productCategory and productSubCategory
$temp = implode(strtoupper($data["productCategory"])).">". implode(strtoupper($data["productSubCategory"]));
$data["productCategory"] = $temp;
but I cant work out how to incorporate that correctly into the export code im working with:
$field = "";
$values = array();
// Generate the Heading Row & SQL Select Keys
if (isset($layout) === true) {
foreach ($layout as $data => $ar) {
if ($ar['csv'] === true || $ar['csv'] == "Y") {
// Build SQL Select Query
if (strlen($field) > 0) {
$field .= ', ';
}
$field .= "`".$data."`";
// Add Heading to Array
array_push($values, $ar['heading']);
}
}
// Add Heading to Array
array_push($values, "Delete");
}
// Open New CSV File
$fp = fopen('product_data.csv', 'w');
// Insert Heading Row
fputcsv($fp, $values);
$sql = "SELECT ".$field." FROM 'Product' ORDER BY `productSKU` ";
$result = sql_exec($sql);
while ($line = $result->fetch_assoc()) {
// Reset values array for each line
$values = array();
// Go through data and generate array for fputcsv()
foreach ($line as $key => $data) {
// Decode HTML Entities in the Data (so people know what they're looking at)
$data = html_entity_decode($data, ENT_QUOTES | ENT_HTML401, "UTF-8");
// Add Data to array for fputcsv
array_push($values, $data);
}
// Add line to the CSV
fputcsv($fp, $values);
}
fclose($fp);
can anyone please exlain or advise how to achieve this?
TIA
Related
PHP script to rearrange csv data and output as csv
I have the following data in a csv file. I need to rearrange the data and concate it into 2 columns. the columns will be SKU and Feature. Where SKU = SKU and Feature will be derivative from other columns in the following format. For yellow marked row: Feature column data will be: Edge:Square Edge;Wide Plank|Finish:Glossy;Smooth|Grade:A(Select & Better/Prestige)|Installation Location:Second Floor;Main Floor........ I could parse the csv and stucked. $lines = explode( "\n", file_get_contents( '3b.csv' ) ); $headers = str_getcsv( array_shift( $lines ) ); $data = array(); foreach ( $lines as $line ) { $row = array(); foreach ( str_getcsv( $line ) as $key => $field ) if($headers[$key]=='sku'){ $row[ $headers[ $key ] ] = str_replace(",",";",$field); } if($headers[$key]!='sku' && $field!='') { $row['feature'] = $headers[ $key ].":".str_replace(",",";",$field)."|"; } $row = array_filter( $row ); $data[] = $row; } echo "<pre>"; print_r($data); echo "</pre>"; Anyone please help me to do this or suggest any script to do this.
You haven't provided the actual text of your incoming csv files, so I will assume that parsing it normally will work properly. I have borrow my script from your next two questions to unconditionally process your data. The header row's data is used as a lookup array for the feature names. Code: (untested) $file = fopen("3b.csv", "r"); $headers = fgetcsv($file); $final_array = []; while (($row = fgetcsv($file)) !== false) { $sku = $row[0]; unset($row[0]); foreach ($row as $featureNameIndex => $featureValues) { foreach (explode(',', $featureValues) as $featureValue) { $final_array[] = [ 'sku' => $sku, 'feature' => "{$headers[$featureNameIndex]}:{$featureValue}" ]; } } } fclose($file); var_export($final_array); This approach will generate an indexed array of associative arrays -- each containing two-elements. Features with multiple values are divided and stored as separate subarrays.
How to get some values from a 3d array into a new array inside a while loop
I would like to get a CSV export working where the user picks some optional fields to export with the required fields. I'm having trouble just adding the list of fields to export into this csv export method. The list of fields i want to be part of the export ends up in the $fields array, but I'm not sure how to integrate that into the export loop for each row. // database conection stuff and $_POST data validation/sanitation above here, not relevant. From a CSV export form. $exportToCSVQuery = " SELECT $allRequestedFields' FROM my_table "; // if the (run query) is valid then... if ($exportToCSVResult = mysqli_query($mysqli, $exportToCSVQuery)) { // count number of rows in result. $exportToCSVResultNumRows = mysqli_num_rows($exportToCSVResult); // if there is at least one result. if ($exportToCSVResultNumRows > 0 ) { if ($safeDelimiter == 'Comma') { $delimiter = ","; } elseif ($safeDelimiter == 'Semi-colon') { $delimiter = ";"; } else{ $delimiter = ","; } //create a file pointer $f = fopen('php://memory', 'w'); //set column headers $fields = array('ID', 'Name', 'em', 'type', 'reason', 'result', 'status', 'notes', 'datetime', 'requestedby', 'requestedbycitrixid', 'week', 'period', 'year', 'historylog'); // put required and optional fields together from the form. $fields = $fields+$safeFieldsArray; fputcsv($f, $fields, $delimiter); //create $linedata information, formatted in a php assoc row format. //flip the $fields array so the values become the keys, needed for the next step. $fieldsFlipped = array_flip($fields); //output each row of the data, format line as csv and write to file pointer while($exporttocsvrow = $exporttocsvresult->fetch_assoc()){ $fieldsarrayforlinedata = array_intersect_key($exporttocsvrow,$fieldsFlipped); // flip array $flippedfieldsarrayforlinedata = array_flip($fieldsarrayforlinedata); fputcsv($f, $flippedfieldsarrayforlinedata, $delimiter); } //move back to beginning of file fseek($f, 0); //set headers to download file rather than displayed header('Content-Type: text/csv'); header('Content-Disposition: attachment; filename="' . $safeFilename . '";'); //output all remaining data on a file pointer fpassthru($f); } else { Echo 'No Data to Export'; } } else { $exportToCSVResultErr = "Error: " .mysqli_error($mysqli) ; trigger_error("Query Error: ".$exportToCSVQuery." Error:" $exportToCSVResultErr, E_USER_WARNING); } Edit: I tried adding something based on Pradeep’s answer below to the loop like so: while($exporttocsvrow = $exporttocsvresult->fetch_assoc()){ $fieldsarrayforlinedata = array_intersect_key($exportToCSVRow,$fieldsFlipped); unset($valuesArray); $valuesArray = array(); foreach ($fieldsFlipped as $key){ foreach ($exportToCSVRow[$key] as $values) { $valuesArray[] = $values; } } fputcsv($f, $valuesArray, $delimiter); } But this just results in the CSV with headings and blank data. I guess I'm probably adding that loop incorrectly.
I hope this will help to get the desired output. $row=array('fieldA'=>array('data1','data2','data2','data3'),'fieldB'=>array('data1','data2','data3'),'fieldC'=>array('data1','data2','data3')); $key=array('fieldA','fieldB'); foreach($key as $key){ foreach($row[$key] as $values){ echo $values; } }
How to convert few columns as JSON array in php while csv import
i have a csv file, please see this below image of my CSV and this is my table i need a help to split few columns and convert as an array and insert in to table, lets say i want to exclude the first column which is 'product_id' from the array and remaining all columns as array and insert in to table, so the expected query will be like insert into process_detail (product_id,product_attributes) values ($product_id,'[{"color":"blue","style":"classic","material_type":"etc","length":"35 cm","price":"11.25","product_description":"etc"}]'; note: my csv headers which are in the first row in CSV file will match exactly with Mysql table headers this is my php function convert($string) { return htmlspecialchars($string,ENT_QUOTES); } $columnArray = array(); $dataArray = array(); $firstRule = true; while ($data = fgetcsv ($source, 1000, ",")) { if($firstRule) { foreach($data as $columnName) { $columnArray[] = $columnName; } $firstRule = false; } else { $rule = array(); for($i = 0; $i < count($data) ; $i++) { $rule[$columnArray[$i]] = $data[$i]; } $dataArray[] = $rule; } } foreach($dataArray as $data) { // here i am stuck, i would want to json_encode of few columns and insert into table } echo $product_id; var_dump($bound_values); mlarray(2) { ["product_id"]=> string(2) "ml" ["product_attributes"]=> string(1588) "{"cv_id":"weight_2_kg","sv_id":"label_3","collection_id":"length_3_cm","season":"width_3_cm","hit":"height_3_cm","tier":"diameter_3_cm","style_no":"size_3_ml","base_product_id":"weight_3_kg","product_title":"label_4","introduction":"length_4_cm","color_name":"width_4_cm","price_a":"height_4_cm","color_price_a":"diameter_4_cm","sku_price_a":"size_4_ml","price_b":"weight_4_kg","color_price_b":"label_5","price_c":"length_5_cm","color_price_c":"width_5_cm","sku_price_c":"height_5_cm","product_availability_b":"diameter_5_cm","product_availability_c":"size_5_ml","product_availability_d":"weight_5_kg","returnable_a":"label_6","returnable_b":"length_6_cm","returnable_c":"width_6_cm","collect_a":"height_6_cm","collect_b":"diameter_6_cm","collect_c":"size_6_ml","brand":"weight_6_kg","group_name":"label_7","department":"length_7_cm","class_name":"width_7_cm","sub_class":"height_7_cm","category_1":"diameter_7_cm","category_2":"size_7_ml","category_3":"weight_7_kg","category_4":"label_8","category_5":"length_8_cm","copy_writing_agency":"width_8_cm","photographer_name":"height_8_cm","photography_date":"diameter_8_cm","event":"size_8_ml","status":"weight_8_kg","style":"label_9","p_family":"length_9_cm","pf_name":"width_9_cm","feature_1":"height_9_cm","primary_material":"diameter_9_cm","secondary_material":"size_9_ml","type_name":"weight_9_kg","material_finish_1":"label_10","material_finish_2":"length_10_cm","for_name":"width_10_cm","gender":"height_10_cm","care_instructions":"diameter_10_cm","fragile":"size_10_ml","assembly_required":"weight_10_kg","unit_components":"segment"}" } 2536789array(2) { ["product_id"]=> string(7) "2536789" ["product_attributes"]=> string(1570) "{"cv_id":"","sv_id":"","collection_id":"","season":"JJ17","hit":"JJ17","tier":"TIER1","style_no":"11DD DV","base_product_id":"","product_title":"something","introduction":"","color_name":"blue","price_a":"169","color_price_a":"","sku_price_a":"","price_b":"189","color_price_b":"","price_c":"1.5","color_price_c":"","sku_price_c":"","product_availability_b":"","product_availability_c":"","product_availability_d":"","returnable_a":"","returnable_b":"","returnable_c":"","collect_a":"","collect_b":"","collect_c":"","brand":"brand a","group_name":"group b","department":"something else","class_name":"something","sub_class":"","category_1":"","category_2":"","category_3":"","category_4":"","category_5":"","copy_writing_agency":"","photographer_name":"","photography_date":"","event":"exclusive","status":"Active","style":"Basic","p_family":"something","pf_name":"something else","feature_1":"","primary_material":"Glass","secondary_material":"Glass","type_name":"something","material_finish_1":"something","material_finish_2":"","for_name":"","gender":"","care_instructions":"","fragile":"Yes","assembly_required":"","unit_components":"","unit_pack_length_cm":"24","unit_pack_width_cm":"16","unit_pack_height_cm":"32","unit_pack_weight_g":"","length_cm":"","width_cm":"","height_cm":"","weight_kg":"","size_ml":"250","diameter_cm":"","set_includes":"","label_1":"","length_1_cm":"","width_1_cm":"","height_1_cm":"","diameter_1_cm":"","size_1_ml":"","weight_1_kg":"","label_2":"","length_2_cm":"","width_2_cm":"","height_2_cm":"","diameter_2_cm":"","size_2_":"","":""}" }INSERT INTO process_detail (process_id,product_id,product_attributes) values(1,'2536789','[{"cv_id":"","sv_id":"","collection_id":"","season":"JJ17","hit":"JJ17","tier":"TIER1","style_no":"11DD DV","base_product_id":"","product_title":"something","introduction":"","color_name":"blue","price_a":"169","color_price_a":"","sku_price_a":"","price_b":"189","color_price_b":"","price_c":"1.5","color_price_c":"","sku_price_c":"","product_availability_b":"","product_availability_c":"","product_availability_d":"","returnable_a":"","returnable_b":"","returnable_c":"","collect_a":"","collect_b":"","collect_c":"","brand":"brand a","group_name":"group b","department":"something else","class_name":"something","sub_class":"","category_1":"","category_2":"","category_3":"","category_4":"","category_5":"","copy_writing_agency":"","photographer_name":"","photography_date":"","event":"exclusive","status":"Active","style":"Basic","p_family":"something","pf_name":"something else","feature_1":"","primary_material":"Glass","secondary_material":"Glass","type_name":"something","material_finish_1":"something","material_finish_2":"","for_name":"","gender":"","care_instructions":"","fragile":"Yes","assembly_required":"","unit_components":"","unit_pack_length_cm":"24","unit_pack_width_cm":"16","unit_pack_height_cm":"32","unit_pack_weight_g":"","length_cm":"","width_cm":"","height_cm":"","weight_kg":"","size_ml":"250","diameter_cm":"","set_includes":"","label_1":"","length_1_cm":"","width_1_cm":"","height_1_cm":"","diameter_1_cm":"","size_1_ml":"","weight_1_kg":"","label_2":"","length_2_cm":"","width_2_cm":"","height_2_cm":"","diameter_2_cm":"","size_2_":"","":""}]') this is my csv content product_id,cv_id,sv_id,collection_id,season,hit,tier,style_no,base_product_id,product_title,introduction,color_name,price_a,color_price_a,sku_price_a,price_b,color_price_b,price_c,color_price_c,sku_price_c,product_availability_b,product_availability_c,product_availability_d,returnable_a,returnable_b,returnable_c,collect_a,collect_b,collect_c,brand,group_name,department,class_name,sub_class,category_1,category_2,category_3,category_4,category_5,copy_writing_agency,photographer_name,photography_date,event,status,style,p_family,pf_name,feature_1,primary_material,secondary_material,type_name,material_finish_1,material_finish_2,for_name,gender,care_instructions,fragile,assembly_required,unit_components,unit_pack_length_cm,unit_pack_width_cm,unit_pack_height_cm,unit_pack_weight_g,length_cm,width_cm,height_cm,weight_kg,size_ml,diameter_cm,set_includes,label_1,length_1_cm,width_1_cm,height_1_cm,diameter_1_cm,size_1_ml,weight_1_kg,label_2,length_2_cm,width_2_cm,height_2_cm,diameter_2_cm,size_2_ml,weight_2_kg,label_3,length_3_cm,width_3_cm,height_3_cm,diameter_3_cm,size_3_ml,weight_3_kg,label_4,length_4_cm,width_4_cm,height_4_cm,diameter_4_cm,size_4_ml,weight_4_kg,label_5,length_5_cm,width_5_cm,height_5_cm,diameter_5_cm,size_5_ml,weight_5_kg,label_6,length_6_cm,width_6_cm,height_6_cm,diameter_6_cm,size_6_ml,weight_6_kg,label_7,length_7_cm,width_7_cm,height_7_cm,diameter_7_cm,size_7_ml,weight_7_kg,label_8,length_8_cm,width_8_cm,height_8_cm,diameter_8_cm,size_8_ml,weight_8_kg,label_9,length_9_cm,width_9_cm,height_9_cm,diameter_9_cm,size_9_ml,weight_9_kg,label_10,length_10_cm,width_10_cm,height_10_cm,diameter_10_cm,size_10_ml,weight_10_kg,segment 2536789,,,,JJ17,JJ17,TIER1,11DD DV,,something,,blue,169,,,189,,1.5,,,,,,,,,,,,brand a,group b,something else,something,,,,,,,,,,exclusive,Active,Basic,something,something else,,Glass,Glass,something,something,,,,,Yes,,,24,16,32,,,,,,250,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, thanks in advance.
There are a handful of ways to approach this problem, however this is the way that I would solve it. Note that this is untested code, so please either troubleshoot or else ask in comments if there's some issue / error. NOTE: the size limit in the fgetcsv is optional, and not necessary, and in this case, causing problems - so remove the limit per the code below. I've commented the code below to tell you what's going on at each step: // declare an empty array to contain the column keys $keys = []; // load the keys into the $keys array // load the first row $data = fgetcsv( $source ); // loop over it, adding the keys into the array foreach( $data AS $value ) { if ( $value ) { $keys[] = $value; } } /** * $keys should now be an array that looks like so: * * $keys = array( * 0 => 'product_id', * 1 => 'color', * 2 => 'style', * 3 => 'type', * ..etc * ); */ // now process the rest of the csv file while ($data = fgetcsv ( $source ) ) { // reset the attributes array, where we'll store the (future) JSON attributes $attributes = []; // reset product_id to ensure pick up the new / correct value $product_id = NULL; // loop over the columns in the row foreach( $data as $index => $value ) { // set the key (based on the keys array established above) $key = $keys[ $index ]; // load the values into the attributes array (unless product id, where we put that in the $product_id variable) if ( 'product_id' != $key ) { $attributes[ $key ] = $value; } else { $product_id = $value; } } if ( empty( $attributes ) || ! $product_id ) { continue; } // prepare the values for a PDO database insert $bound_values = [ 'product_id' => $product_id, 'product_attributes' => json_encode( $attributes ) ]; // insert into the database $stmt = $db->prepare( "INSERT INTO product_table SET product_id = :product_id, product_attributes = :product_attributes" ); $stmt->execute( $bound_values ); }
exporting a mysql query in php version 4 to a csv
Edit on my original post. I found the answer!!!! with help:) I now have this working by using the below code with thanks for the advice on this in the comments: <?php $f = fopen('incident_csv\test.csv', 'w'); $query = " select column1, column2, column3 from table where columns = values "; $var1 = mysql_query($query, $database connection variable); /* From Monkey Zeus */ $csv_lines = array(); // Loop your records from the DB while ($row = mysql_fetch_assoc($var1)){ $columns = array(); // Loop each column for the row foreach($row as $k=>$v){ // Surround column item in double-quotes and escape double-quotes with double-double-quotes $columns[] = '"'.str_replace('"', '""', $v).'"'; } // Join on a comma $csv_lines[] = implode(',', $columns); } // Create full CSV file by joining on a newline $csv_file_string = implode("\n", $csv_lines); /* From Monkey Zeus */ fwrite($f, $csv_file_string); ?>
You can do this: $csv_lines = array(); // Loop your records from the DB while ($row = mysql_fetch_assoc($var1)){ $columns = array(); // Loop each column for the row foreach($row as $k=>$v){ // Surround column item in double-quotes and escape double-quotes with double-double-quotes $columns[] = '"'.str_replace('"', '""', $v).'"'; } // Join on a comma $csv_lines[] = implode(',', $columns); // Write to the file right away. You are using PHP4 so I imagine system memory is not plentiful // If you use this then there is no need for the "$csv_lines[] =" from above // nor the $csv_file_string after the while loop // fwrite($f, implode(',', $columns)."\n"); } // fclose($f); // If you choose to write to the file during the while loop then close the file handle when you are finished // Create full CSV file by joining on a newline $csv_file_string = implode("\n", $csv_lines);
You can simply wite an arrayToCsv function: function arrayToCsv($array) { $string = array(); foreach ($array as $field) { $string[] = implode(';', $field); } return implode("\n", $string); } $a = array( array('First Field of First Line', 'Second Field of First Line'), array('First Field of Second Line', 'Second Field of Second Line') ); fwrite($f, arrayToCsv($a)); But, remember to have your $f setted from $f = fopen('file location', 'w'); Hope I was helpful.
Checking while loop key exists in seperate array
Having checked a variety of questions but not being able to find quite what I need, I am at a bit of a loss. I am trying to chose the columns from MySQL I want exported to CSV by parsing the column names and adding the valid column names to a $colnames array, then adding those values as headers to the CSV and then only displaying the relevant data from the database through a while loop. I have looked at the following in particular having been guided there from other questions: How to get all the key in multi-dimensional array in php Here is the code: function query_to_csv($query, $filename, $attachment = false, $headers = true, $specs_off = false) { if($attachment) { // send response headers to the browser header( 'Content-Type: text/csv; charset=UTF-8' ); header( 'Content-Disposition: attachment;filename='.$filename); $fp = fopen('php://output', 'w'); } else { $fp = fopen($filename, 'w'); } $result = mysql_query($query) or die( mysql_error() ); if($headers) { // output header row (if at least one row exists) $row = mysql_fetch_assoc($result); if($row) { // PRODUCTS TABLE SPECIFIC - get rid of specs_ and free_ columns so have nicer data set for user if($specs_off) { $columnames = array_keys($row); $colnames = array(); //$colnames = array_keys($row); foreach($columnames as $key => $value) { if((substr_count($value, "spec_") < 1) && (substr_count($value, "free_") < 1)) { array_push($colnames, $value); } } } else { $colnames = array_keys($row); } // add in additional columns if exporting client data if($table == 'clients') {array_push($colnames, "products", "last_order_date");} //write the colnames to the csv file fputcsv($fp, $colnames); // reset pointer back to beginning mysql_data_seek($result, 0); } } // done with the headers etc, now lets get on with the data // clear out and create the $row_data array $row_data = array(); // run through the row array adding values to row_data as we go while($row = mysql_fetch_assoc($result)) { // create the array_keys_multi from https://stackoverflow.com/questions/11234852/how-to-get-all-the-key-in-multi-dimensional-array-in-php/11234924#11234924 function array_keys_multi(array $array) { $keys = array(); foreach ($array as $key => $value) { $keys[] = $key; if (is_array($array[$key])) { $keys = array_merge($keys, array_keys_multi($array[$key])); } } return $keys; } // run the function on the $row array array_keys_multi($row); // now use the $keys array foreach($keys as $key => $value) { // check if the value is in the colnames array and if so push the data on to the $row_data array ready for export to CSV if(in_array($value, $colnames)) { array_push($row_data, $row[$value]); } } // now we are ready to write the CSV fputcsv($fp, $row_data); } fclose($fp); exit; } // end of query_to_csv // Write the sql statement $sql = "SELECT * FROM ".$table." "; if(isset($where_1_col)) { $sql .= " WHERE `".$where_1_col."` = '".$where_1_val."'"; } if(isset($where_2_col)) { $sql .= " AND `".$where_2_col."` = '".$where_2_val."'"; } if(isset($where_3_col)) { $sql .= " AND `".$where_3_col."` = '".$where_3_val."'"; } if(isset($where_4_col)) { $sql .= " AND `".$where_4_col."` = '".$where_4_val."'"; } if(isset($order_by_col)) { $sql .= " ORDER BY `".$order_by_col."` ". strtoupper($order_by_dir) ." "; } // output as an attachment query_to_csv($sql, $table."_export.csv", true, true, true); All I am getting is a huge export of the chosen column names repeated as many times as there are values from the initial query. I don't know how to get the values in. Any suggestions on where I am going wrong or how I can undertake this more neatly are welcomed.
It seems that you just append the new row data to $row_data but never clear that array. array_push($row_data, $row[$value]); What I did to fix it: Move // clear out and create the $row_data array $row_data = array(); into the while loop. Change // clear out and create the $row_data array $row_data = array(); while($row = mysql_fetch_assoc($result)) { ... } To while($row = mysql_fetch_assoc($result)) { // clear out and create the $row_data array $row_data = array(); ... } Note: You are using $table everywhere but never define it. Like here if($table == 'clients') If it is a global var you need to add global $table or a parameter to your function, too. Edit: As mentioned in my comment on your question you could just use array_keys() to get the keys. php.net/manual/de/function.array-keys.php And then change array_keys_multi($row); to $keys = array_keys($row); After that you can remove array_keys_multi() Further you could move that part in front of your while-loop because you only need to calculate the column names you need once and not in every iteration.