I have a number of DBF database files that I would like to convert to CSVs. Is there a way to do this in Linux, or in PHP?
I've found a few methods to convert DBFs, but they are very slow.
Try soffice (LibreOffice):
$ soffice --headless --convert-to csv FILETOCONVERT.DBF
Change the files variable to a path to your DBF files. Make sure the file extension matches the case of your files.
set_time_limit( 24192000 );
ini_set( 'memory_limit', '-1' );
$files = glob( '/path/to/*.DBF' );
foreach( $files as $file )
{
echo "Processing: $file\n";
$fileParts = explode( '/', $file );
$endPart = $fileParts[key( array_slice( $fileParts, -1, 1, true ) )];
$csvFile = preg_replace( '~\.[a-z]+$~i', '.csv', $endPart );
if( !$dbf = dbase_open( $file, 0 ) ) die( "Could not connect to: $file" );
$num_rec = dbase_numrecords( $dbf );
$num_fields = dbase_numfields( $dbf );
$fields = array();
$out = '';
for( $i = 1; $i <= $num_rec; $i++ )
{
$row = #dbase_get_record_with_names( $dbf, $i );
$firstKey = key( array_slice( $row, 0, 1, true ) );
foreach( $row as $key => $val )
{
if( $key == 'deleted' ) continue;
if( $firstKey != $key ) $out .= ';';
$out .= trim( $val );
}
$out .= "\n";
}
file_put_contents( $csvFile, $out );
}
Using #Kohjah's code, here an update of the code using a better (IMHO) fputcsv approach:
// needs dbase php extension (http://php.net/manual/en/book.dbase.php)
function dbfToCsv($file)
{
$output_path = 'output' . DIRECTORY_SEPARATOR . 'path';
$path_parts = pathinfo($file);
$csvFile = path_parts['filename'] . '.csv';
$output_path_file = $output_path . DIRECTORY_SEPARATOR . $csvFile;
if (!$dbf = dbase_open( $file, 0 )) {
return false;
}
$num_rec = dbase_numrecords( $dbf );
$fp = fopen($output_path_file, 'w');
for( $i = 1; $i <= $num_rec; $i++ ) {
$row = dbase_get_record_with_names( $dbf, $i );
if ($i == 1) {
//print header
fputcsv($fp, array_keys($row));
}
fputcsv($fp, $row);
}
fclose($fp);
}
Related
I am getting undefined offset. Please help, I can't find where's the problem at all :(
<?php
$fileTitle = $_GET['title'];
$rawData = $_GET['data'];
//fileTitle is testing
//rawData is 5000013,0.00,0.00,50.00,0+5006529,0.00,0.00,50.00,0
$lineData = explode("+",$rawData)
for( $i=0 ; $i<count($lineData) ; $i++)
{
$txtFileTitle = $fileTitle.".txt";
$txtFileLineData = $lineData[$i]."\r\n";
$txtFileStatus = file_put_contents($txtFileTitle, $txtFileLineData, FILE_APPEND);
if($txtFileStatus != false)
{
echo "SUCCESS: data written to txt file";
}
else
{
echo "FAIL: could not write to txt file";
}
}?>
The result is a .txt file with:
5000013,0.00,0.00,50.00,0+5006529,0.00,0.00,50.00,0
What i want is a .txt with:
5000013,0.00,0.00,50.00,0
5006529,0.00,0.00,50.00,0
I think the issue was due to the <= in the loop - as the counter begins at zero it would exceed the actual count by one and thus create the undefined index error.
<?php
$title = $_GET['title'];
$data = $_GET['data'];
//title is testing
//data is 5000013,0.00,0.00,50.00,0+5006529,0.00,0.00,50.00,0
$lines = explode( "+", $data );
$file = $title . ".txt";
for( $i=0 ; $i < count( $lines ); $i++ ){
$content = $lines[ $i ] . PHP_EOL;
$status = file_put_contents( $file, $content, FILE_APPEND );
echo $status ? 'SUCCESS: data written to txt file' : 'FAIL: could not write to txt file';
}
?>
edit
The use of + in the source data is ambiguous because the + is used in urls when they are encoded ( space )
<?php
$title = $_GET['title'];
$data = urlencode( $_GET['data'] );
$lines = explode( "+", $data );
$file = __DIR__ . '/' . $title . ".txt";
for( $i=0 ; $i < count( $lines ); $i++ ){
$content = urldecode( $lines[ $i ] ) . PHP_EOL;
$status = file_put_contents( $file, $content, FILE_APPEND );
echo $status ? 'SUCCESS: data written to txt file' : 'FAIL: could not write to txt file';
}
?>
You can simple to use preg_replace function to replace + whith space
$fileTitle = $_GET['title'];
$rawData = $_GET['data'];
$txtFileLineData = preg_replace('/\+/', ' ', $rawData);
$txtFileTitle = $fileTitle.".txt";
$txtFileStatus = file_put_contents($txtFileTitle, $txtFileLineData, FILE_APPEND);
if ($txtFileStatus != false) {
echo "SUCCESS: data written to txt file";
} else {
echo "FAIL: could not write to txt file";
}
I can't figure out a way to do this and I really bad with working with arrays so I hope someone can help me.
I have array $stuck that just contains names, like this
$stuck = array('Daniel','Alex','alfredo','dina');
I am using sort to put the names in alphabetical order, like this
sort($stuck);
Now the thing is I want to put this array into a csv or excel file so the first letter of name will be the title and all the names with this letter will be under this title.
This is an example of what I am trying to accomplish
Here try this. I added comments between the code.
$csvArray = array();
$nameArray = array('Daniel','Alex','alfredo','dina');
//Create an array that can be used for creating the CSV
foreach($nameArray as $name)
{
$firstLetter = strtoupper(substr($name, 0,1));
$csvArray[$firstLetter][] = $name;
}
//Determine the subarray which have the must rules
$maxRuleCount = 0;
foreach($csvArray as $firstLetter => $nameArray)
{
if(count($nameArray) > $maxRuleCount)
{
$maxRuleCount = count($nameArray);
}
}
//Create the first rule (letters)
$csvContent = implode(';', array_keys($csvArray));
$csvContent .= "\r\n";
//Loop through the CSV array and create rules of the names
for($i = 0 ; $i < $maxRuleCount ; $i++)
{
foreach($csvArray as $firstLetter => $nameArray)
{
$csvContent .= #$csvArray[$firstLetter][$i].';';
}
$csvContent .= "\r\n";
}
//The hole csv content is in the $csvContent variable now
//If you want to print it you need to add the text/plain header because of the \r\n new line characters
header("Content-Type:text/plain");
echo $csvContent;
Ouput is:
D;A
Daniel;Alex;
dina;alfredo;
<?php
$stuck = array('Daniel','Alex','Dina','durban','eigor','alfredo','dina');
// associate names with first letters in a new array
foreach ($stuck as $name) {
$first_letter = strtoupper( substr( $name, 0, 1 ) );
$new_name_array[ $first_letter ][] = $name;
}
// store the first letter list in an array and sort it
$first_letters = array_keys( $new_name_array );
sort( $first_letters );
// sort the name lists
foreach( array_keys( $new_name_array ) as $letter ) {
sort( $new_name_array[$letter] );
}
// output csv header row
echo "'".implode( "','", $first_letters )."'\n";
// output the CSV name rows
$i = 0;
while ( true ) {
$row = array();
$is_row = false;
foreach( $first_letters as $letter ) {
$row[] = $new_name_array[ $letter ][ $i ];
if( ! empty( $new_name_array[ $letter ][ $i ] ) ) $is_row = true;
}
if( ! $is_row ) exit;
echo "'" . implode( "','", $row ) . "'\n";
$i++;
}
?>
Outputs quoted CSV:
'A','D','E'
'Alex','Daniel','eigor'
'alfredo','Dina',''
'','dina',''
'','durban',''
<?php
$stuck = array('Daniel','Alex','alfredo','dina', 'eigor', 'durban');
sort($stuck);
$sortedNames = array();
$maxEl = 0;
foreach ($stuck as $name) {
$name = strtolower($name);
$sortedNames[$name{0}][] = $name;
$maxEl = count($sortedNames[$name{0}]) < $maxEl ? $maxEl : count($sortedNames[$name{0}]);
}
$headers = array_keys($sortedNames);
$finalArray = array($headers);
for($i = 0; $i < $maxEl; $i++) {
$tmpRow = array();
foreach ($sortedNames as $names) {
$tmpRow[] = isset($names[$i]) ? $names[$i] : '';
}
$finalArray[] = $tmpRow;
}
$file = fopen("names.csv","w");
foreach ($finalArray as $line)
{
fputcsv($file,$line);
}
fclose($file);
Here is my attempt at this question:
$names = array('Daniel','Alex','alfredo','dina', 'eigor', 'falfoul', 'fiona');
natcasesort($names);
$columns = array();
foreach ($names as $name) {
$first = strtoupper(substr($name, 0, 1));
$columns[$first][] = $name;
}
// pad the columns so they all have the same number of rows
$maxRows = 0;
foreach ($columns as &$col) {
$numRows = count($col);
if ( $numRows > $maxRows) {
$maxRows = $numRows;
}
}
// pad the columns
foreach ($columns as &$column) {
$column = array_pad($column, $maxRows, '');
}
$firstLetter = array_keys($columns);
$numCols = count($firstLetter);
// transpose the columns array
$rows = array();
foreach ($columns as $csvCol) {
foreach ($csvCol as $k=>$n) {
$rows[$k][] = $n;
}
}
// pad the rows to ensure they all have the same number of
// columns
foreach ($rows as &$row) {
$row = array_pad($row, $numCols, '');
}
// add the title row to the rows
array_unshift($rows, $firstLetter);
$handle = fopen("names.csv", 'w');
foreach ($rows as $fRow) {
fputcsv($handle, $fRow);
}
I have to read config files in PHP which contain entries with semicolons, e.g.
[section]
key=value;othervalue
I notices that parse_ini_file() removes all semicolons and what follows, even when set to INI_SCANNER_RAW.
The INI files come from legacy systems and I can't change the format. I only have to read them.
What's the best tool to use when I have to preserve the entries with semicolons?
I would recommend reading the file into an array first, convert the semicolons to pipes |, then spit that out to a temporary file and use parse_ini_file() with that new temporary file.
Like so...
$string = file_get_contents('your_file');
$newstring = str_replace(";","|",$string);
$tempfile = 'your_temp_filename';
file_put_contents($tempfile, $newstring);
$arrIni = parse_ini_file($tempfile);
Then after that you could always replace the pipes with semicolons as you enumerate your new INI based array.
For ini files, the ; is the comment symbol.
So it's actually a good idea to not use it for something else.
However, you can use this slightly modified functions from the solutions found here :
<?php
//Credits to goulven.ch AT gmail DOT com
function parse_ini ( $filepath )
{
$ini = file( $filepath );
if ( count( $ini ) == 0 ) { return array(); }
$sections = array();
$values = array();
$globals = array();
$i = 0;
foreach( $ini as $line ){
$line = trim( $line );
// Comments
if ( $line == '' || $line{0} == ';' ) { continue; }
// Sections
if ( $line{0} == '[' )
{
$sections[] = substr( $line, 1, -1 );
$i++;
continue;
}
// Key-value pair
list( $key, $value ) = explode( '=', $line, 2 );
$key = trim( $key );
$value = trim( $value );
if (strpos($value, ";") !== false)
$value = explode(";", $value);
if ( $i == 0 ) {
// Array values
if ( substr( $line, -1, 2 ) == '[]' ) {
$globals[ $key ][] = $value;
} else {
$globals[ $key ] = $value;
}
} else {
// Array values
if ( substr( $line, -1, 2 ) == '[]' ) {
$values[ $i - 1 ][ $key ][] = $value;
} else {
$values[ $i - 1 ][ $key ] = $value;
}
}
}
for( $j=0; $j<$i; $j++ ) {
$result[ $sections[ $j ] ] = $values[ $j ];
}
return $result + $globals;
}
You can see examples of usage following the link.
Pls, let someone help me with this code, When I export, it export the file but start with row2, the first row is excluded.it reads like , header, then row2,3,4 till the end of the row.
<?php
require_once("/includes/session.php");
require_once("/includes/db_connection.php");
require_once("/includes/functions.php");
// Table Name that you want
// to export in csv
$ShowTable = "staff_tab";
$today=date("dmY");
$FileName = "StaffRecord".$today.". csv";
$file = fopen($FileName,"w");
$sql = mysqli_query($connection,("SELECT * FROM $ShowTable LIMIT 500"));
$row = mysqli_fetch_assoc($sql);
// Save headings alon
$HeadingsArray=array();
foreach($row as $name => $value){
$HeadingsArray[]=$name;
}
fputcsv($file,$HeadingsArray);
// Save all records without headings
while($row = mysqli_fetch_assoc($sql)){
$valuesArray=array();
foreach($row as $name => $value){
$valuesArray[]=$value;
}
fputcsv($file,$valuesArray);
}
fclose($file);
header("Location: $FileName");
echo "Complete Record saves as CSV in file: <b style=\"color:red;\">$FileName</b>";
?>
Your call to $row = mysqli_fetch_assoc($sql); is pushing the internal pointer forward to row 2. Use mysqli_data_seek($sql, 0); to push it back to the start. http://php.net/manual/en/function.mysql-data-seek.php
...
$sql = mysqli_query($connection,("SELECT * FROM $ShowTable LIMIT 500"));
$row = mysqli_fetch_assoc($sql);
mysqli_data_seek($sql, 0); // return pointer to first row
// Save headings alon
$HeadingsArray=array();
...
maybe this could help as this will include all rows and set headers from the db headers. This would also force download the file without leaving the page.
$export = mysqli_query ($query) or die ( "Sql error : " . mysqli_error( ) );
// extract the field names for header
$fields = mysqli_num_fields ( $export );
for ( $i = 0; $i < $fields; $i++ )
{
$headers = mysqli_fetch_field($export);
$header .= $headers->name. "\t";
}
// export data
while( $row = mysqli_fetch_row( $export ) )
{
$line = '';
foreach( $row as $value )
{
if ( ( !isset( $value ) ) || ( $value == "" ) )
{
$value = "\t";
}
else
{
$value = str_replace( '"' , '""' , $value );
$value = '"' . $value . '"' . "\t";
}
$line .= $value;
}
$data .= trim( $line ) . "\n";
}
$data = str_replace( "\r" , "" , $data );
if ( $data == "" )
{
$data = "\nNo Record(s) Found!\n";
}
// allow exported file to download forcefully
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=Something.xls");
header("Pragma: no-cache");
header("Expires: 0");
print "$header\n$data";
I want to get table from database and then break output data by files (50 entries in each file): list01.txt, list02.txt... But somehow I got stacked at the question how to break data more effectively.
if ( $result = $mysqli->query($query) ) {
icount = 0;
while ( $row = mysqli_fetch_array($result) ) {
if ( icount % 50 == 0 ) {
$snum = int( icount / 50 );
$filename = 'scripts/spisok'.$snum.'.txt';
$handle = fopen( $filename, 'w' );
}
fwrite( $filename, $row['uname'].';'.$row['email'].'<br />' );
icount++;
}
echo 'ok';
$result->free();
}
Can I just break $result into 50-entry arrays first and then write them all? Sorry, im novice to PHP
You can also use array_chunk
<?php
if ( $result = $mysqli->query( $query ) ) {
$data = array();
while( $row = mysqli_fetch_array( $result ) ) {
$data[] = $row['uname'].';'.$row['email'];
}
$result->free();
// divide an array into a desired number of split lists
$chunks = array_chunk( $data, 50 );
// loop through chunks
foreach( $chunks as $index => $chunk ) {
$file = 'scripts/spisok'.( $index + 1 ).'.txt';
$chunk = implode( "<br />", $chunk );
file_put_contents( $file, $chunk );
// or
/*
$handle = fopen( $file, 'w' );
fwrite( $handle, $chunk );
fclose( $handle );
*/
}
unset( $data, $chunks );
}
?>
There is nothing particularly wrong with what you are doing except for some syntax errors here and there.
Try this :-
if ( $result = $mysqli->query($query) ) {
$icount = 0;
$handle = NULL;
while ( $row = mysqli_fetch_array($result) ) {
if ( $icount % 50 == 0 ) {
if ( $handle !== NULL ) {
fclose($handle);
}
$snum = int( $icount / 50 );
$filename = 'scripts/spisok'.$snum.'.txt';
$handle = fopen( $filename, 'w' );
}
fwrite( $handle, $row['uname'].';'.$row['email'].'<br />' );
$icount++;
}
fclose($handle);
echo 'ok';
$result->free();
}