Convert CSV to XML using PHP - php

I have multiple CSV files in a folder with different data. I want to read in 1 CSV file at a time, convert the data to XML and then output the file as .xml before reading in the next CSV file.
Currently, the CSV files that I have all have a header row in it. The code I ave right now runs fine for all the CSV files that have a header row in it.
However, when it reads in a file that does not have a header row in it, it throws an error. I want it to be able to detect if there is no headings in the header row and then make it input strings which have been preset in variables in the code. Is this possible?
Current Code
function converttoxml( $input_filename ) {
echo $input_filename;
//set the delimiter
$delimiter = "," ;
//set count to 0
$row_count++ ;
//strip the input filename of the extension
$stripped = pathinfo($input_filename, PATHINFO_FILENAME);
//set output filename
$outputFilename = UPLOAD_DIR."/".$stripped.".xml";
//open inputfilename
$inputFile = fopen( $input_filename, 'rt' );
//get input file pointers for csv parsing
$headers = fgetcsv( $inputFile );
//create new DOM document
$doc = new DomDocument();
//set the output to clean
$doc->formatOutput = true;
//create element
$root = $doc->createElement( 'Shipping_Details' );
$root = $doc->appendChild( $root );
//while there are rows in the csv file
while ( ( $row = fgetcsv( $inputFile ) ) !== FALSE ) {
//create container row
$container = $doc->createElement('Job_Header_Details');
//for loop
foreach ( $headers as $i => $header ) {
//explode with the delimiter the header
$arr = explode( $delimiter, $header );
//print_r($arr);
//for loop
foreach ( $arr as $key => $ar ) {
//only accept regualar expressions matching this
$child = $doc->createElement(preg_replace( "/[^A-Za-z0-9]/","",$ar ) );
//add the previous child to $child
$child = $container->appendChild( $child );
//explode row with delimiter
$whole = explode( $delimiter, $row[$i] );
//left and right trim anything with speechmarks
$value = $doc->createTextNode( ltrim( rtrim( $whole[$key], '"') ,'"') );
//append previous value to $value
$value = $child->appendChild( $value );
}//for
}//for
//append to root - container
$root->appendChild($container);
}//while
echo "Saving the XML file\n" ;
$result = $doc->saveXML();
echo "Writing to the XML file\n" ;
$handle = fopen( $outputFilename, "w" );
fwrite( $handle, $result );
fclose( $handle );
return $outputFilename;
Examples of CSV files that will/willnot process
CSV File example that will work with above code
CSV File example that will NOT work with the above code
With the example that will not work, I believe it is because the header row is missing. In this case, I want to somehow define what each column heading should be and input it in.
for example
$column_1 = "<heading1>";
$column_2 = "<heading2>";
$column_3 = "<heading3>";
$column_4 = "<heading4>";
$column_5 = "<heading5>";
$column_6 = "<heading6>";
etc..
So when the script runs and it detects headings in the CSV file, it will use them. But when it detects that they are not there, the it will use the above $column examples to input the in.
The error I get when a CSV file does not have a header description
Any ideas?

Just to try this out, added ...
//get input file pointers for csv parsing
$headers = fgetcsv( $inputFile );
$row = 1;
foreach ( $headers as &$header ) {
if (preg_match('/\A(?!XML)[a-z][\w0-9-]*/i', $header) === 0 ) {
$header = 'heading'.$row;
}
$row++;
}
unset($header);
Which when passed in a header of a,1223 produces headers of a,heading2. Whereas a,b gives a,b.
Update: Although if any element fails, you'll want to process this row as data by the look of it.

Related

Unable to insert data in CSV file PHP

I am in process of inserting data in the desired CSV file from another CSV file.
CSV file is creating fine with out any problem but its is not insert array data in file.
It only inserts header on the first row.
Below is code I am trying:
date_default_timezone_set('America/New_York');
set_time_limit(0);
ini_set("memory_limit", -1);
$realPath = realpath( dirname(__FILE__) );
$path = $realPath.'/3pltracking/';
$files = scandir($path);
$FilePath = $path.$files[2];
$result = array();
$date = date('m-d-Y_his');
if (file_exists($FilePath))
{
if (($handle = fopen($FilePath, "r")) !== FALSE)
{
$i=0;
while (($data = fgetcsv($handle, 10000, ",")) !== FALSE)
{
$i++;
if($i==1) continue;
//$list = array('$data[2],$data[25],$data[4],$data[30],$data[41],$data[27]');
echo $data[2].",".$data[25].",".$data[4].",".$data[30].",".$data[41].",".$data[27];
echo "<br>";
$list = array($data[2].",".$data[25].",".$data[4].",".$data[30].",".$data[41].",".$data[27]);
// the problem is here I believe as it is empty array if I check it outside while loop
}
fclose($handle);
$headers = array('ReferenceNumber', 'TotalCartons', 'ShipCarrier', 'TrackingNum', 'FreightPP', 'TotalWeight');
$fp = fopen($realPath.'\3pltracking\TrackingFiles\Tracking_File_'.$date.'.csv', 'w');
fputcsv($fp, $headers);
foreach ($list as $line) {
$val = explode(",", $line);
fputcsv($fp, $val);
}
fclose($fp);
} else {
$body = "File Not Found";
}
}
Here is my CSV file data:
TransactionNumber,CustomerName,ReferenceNumber,PurchaseOrderNumber,ShipCarrier,ShipService,ShipBilling,ShipAccount,EarliestShipDate,CancelDate,Notes,ShipToName,ShipToCompany,ShipToAddress1,ShipToAddress2,ShipToCity,ShipToState,ShipToZip,ShipToCountry,ShipToPhone,ShipToFax,ShipToEmail,ShipToCustomerName,ShipToDeptNumber,ShipToVendorID,TotalCartons,TotalPallets,TotalWeight,TotalVolume,BOLNum,TrackingNum,TrailerNum,SealNum,ShipDate,ItemNumber,ItemQuantityOrdered,ItemQuantityShipped,ItemLength,ItemWidth,ItemHeight,ItemWeight,FreightPP,WarehouseID,LotNumber,SerialNumber,ExpirationDate,Supplier,Cost,FulfillInvShippingAndHandling,FulfillInvTax,FulfillInvDiscountCode,FulfillInvDiscountAmount,FulfillInvGiftMessage,SoldToName,SoldToCompany,SoldToAddress1,SoldToAddress2,SoldToCity,SoldToState,SoldToZip,SoldToCountry,SoldToPhone,SoldToFax,SoldToEmail,SoldToCustomerID,SoldToDeptNumber,FulfillInvSalePrice,FulfillInvDiscountPct,FulfillInvDiscountAmt
242328,PARADIGM TRENDS,123810,40-402849,CUSTOMER PICK UP,LTL,FreightCollect,,,,,HG BUYING- JEFFERSON DC 884,HG BUYING- JEFFERSON DC 884,125 LOGISTICS CENTER PKWY,,JEFFERSON,AL,30549,US,,,,,,,30,0,30,0.0174,,,,,,DOV3S,64,64,4,1,1,4,0,1,,,,,,0,0,,0,,,,,,,,,,,,,,,0,0,0
33,d,123810,40-402849,CUSTOMER PICK UP,LTL,FreightCollect,,,,,HG BUYING- JEFFERSON DC 884,HG BUYING- JEFFERSON DC 884,125 LOGISTICS CENTER PKWY,,JEFFERSON,AL,30549,US,,,,,,,30,0,30,0.0174,,,,,,DOV3S,64,64,4,1,1,4,0,1,,,,,,0,0,,0,,,,,,,,,,,,,,,0,0,0
44,PARAdgdfDIGM TRENDS,123810,40-402849,CUSTOMER PICK UP,LTL,FreightCollect,,,,,HG BUYING- JEFFERSON DC 884,HG BUYING- JEFFERSON DC 884,125 LOGISTICS CENTER PKWY,,JEFFERSON,AL,30549,US,,,,,,,30,0,30,0.0174,,,,,,DOV3S,64,64,4,1,1,4,0,1,,,,,,0,0,,0,,,,,,,,,,,,,,,0,0,0
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,BY3M,176,176,11,1,1,11,,,,,,,,,,,,,,,,,,,,,,,,,,0,0,0
There are so many ways of going about this... including str_getcsv($csvData). However here we'd go for something old-school & a bit twisted;-). We would create a Function that uses Regex and a Looping Construct to build-up the relevant CSV Data Structure. The Function below illustrates how. Also note that although we mentioned that this is a somewhat twisted, old-school approach: don't be fooled... because it does its thing still ;-).
<?php
$csvSourceFile = __DIR__ . "/1.csv";
$csvPreferredColumns = array('ReferenceNumber', 'TotalCartons', 'ShipCarrier', 'TrackingNum', 'FreightPP', 'TotalWeight');
$newCsvStrut = processCSVData($csvSourceFile, $csvPreferredColumns, __DIR__ . "/test.csv");
/**
* #param $csvSource // PATH TO THE MAIN CSV FILE
* #param array $csvPreferredColumns // ARRAY OF HEADER COLUMN-NAMES TO BE EXTRACTED FROM MAIN CSV
* #param null $newCSVFileName // NAME OF THE NEW CSV FILE TO BE CREATED.
* #return string
*/
function processCSVData($csvSource, array $csvPreferredColumns, $newCSVFileName=null){
// GET THE CONTENTS OF THE CSV FILE & STORE IT IN A VARIABLE
$csvData = file_get_contents($csvSource);
// SPLIT THE CONTENTS OF THE CSV FILE LINE BY LINE: THAT IS; AT THE END OF EACH LINE
// THUS CONVERTING THE DATA TO AN ARRAY...
$arrCsvLines = preg_split("#\n#", $csvData);
//FILTER OUT UNWANTED EMPTY VALUES FROM THE ARRAY
$arrCsvLines = array_filter($arrCsvLines);
// CREATE SOME VARIABLES TO BE USED WITHIN THE LOOP...
$strDataFinal = "";
$arrDataMain = $arrDataFinal = array();
// IF THERE IS MORE THAN ONE LINE IN THE ARRAY WE CREATED ABOVE,
// THEN CONTINUE PROCESSING THE DATA...
if($arrCsvLines && count($arrCsvLines)>0){
// SINCE THE HEADER IS ALWAYS THE FIRST LINE IN THE CHAIN,
// WE EXPLICITLY EXTRACT IT AND STORE IT IN A VARIABLE FOR LATER USE
$arrCsvHeaders = preg_split("#\,([\s\t]+)?#", $arrCsvLines[0]);
// NOW WE LOOP THROUGH ALL THE LINES WE CREATED BY SPLITTING THE CONTENTS
// OF THE CSV FILE AT THE END-OF-LINE BOUNDARY
foreach($arrCsvLines as $key=>$arrCsvLine){
// WE DON'T WANT ANYTHING AT INDEX "0" SINCE IT IS THE HEADER
// AND WE ALREADY DEALT WITH IT ABOVE....
// SO IF THE INDEX $key IS NOT 0, WE CAN CONTINUE PROCESSING
if($key != 0){
$arrDataTemp = array();
$arrTempCsvData = preg_split("#\,([\s\t]+)?#", $arrCsvLine);
foreach($arrTempCsvData as $iKey=>$sData){
$arrDataTemp[$arrCsvHeaders[$iKey]] = $sData;
}
$arrDataMain[] = $arrDataTemp;
}
}
foreach($arrDataMain as $iKey=>$subData){
$arrTempFinal = array();
foreach($subData as $key=>$data){
if(in_array($key, $csvPreferredColumns)){
$arrTempFinal[$key] = $data;
}
}
$arrDataFinal[] = implode(",\t", $arrTempFinal);
}
$arrDataFinal = array_merge( array(implode(",\t", $csvPreferredColumns)), $arrDataFinal);
$strDataFinal = implode("\n", $arrDataFinal);
if($newCSVFileName){
file_put_contents($newCSVFileName, $strDataFinal);
}
}
return $strDataFinal;
}
var_dump($newCsvStrut);
// PRODUCES SOMETHING SIMILAR TO THE LINES BELOW:
string 'ReferenceNumber, TotalCartons, ShipCarrier, TrackingNum, FreightPP, TotalWeight
123810, CUSTOMER PICK UP, 30, 30, , 0
123810, CUSTOMER PICK UP, 30, 30, , 0
123810, CUSTOMER PICK UP, 30, 30, , 0
, , , , , ' (length=204)

Copying Images from URl list to my server all at once by php

I have big list of Urls in html file for images something like this :
image1
image2
image3
image4
image5
image6
image7
Around 50,000 Image
I want to make small script that can copy all images to my server so i can have them in :
http://Mywebsite.com/images/image1.jpg
http://Mywebsite.com/images/image1.jpg
http://Mywebsite.com/images/image1.jpg
...
I want to make loop and each Url in the list must be deleted after the image is copied successfully because sometimes if page crush on loading or something i can continue my loop without overwriting or reading again , if there is a better solution to not overwrite and read the url again please tell me.
I would create a script that reads your html file line per line. You can do that using fopen and fgets.
fopen("path/to/some/file", "r");
while ( ( $line = fgets( $handle ) ) !== false )
{
// do somehting with $line
}
This way the file gets not simply parsed into memory, so you don't have to worry about size
Then after parsing every line I would write down a lock file containing the current line number / index. So if your script crashes and you restart it the iteration simply skips every line until it's current index is higher than the index from the lock file.
the script
It might work but, in the end should not simply copy paste everything. But i hope it helps you finding your solution.
#!/usr/bin/env php
<?php
// I DID NOT TEST THIS!
// but it should work.
$handle = fopen("path/to/the/html/file/containing/the/urls.html", "r");
$storage = "path/where/you/want/your/images/";
$lockFile = __DIR__.'/index.lock';
$index = 0;
// get the lock index
if ( !file_exists( $lockFile ) )
{
file_put_contents( $lockFile, 0 );
}
// load the current index
$start = file_get_contents( $lockFile );
if ( $handle )
{
// line by line step by step
while ( ( $line = fgets( $handle ) ) !== false )
{
// update the
$index++;
if ( $start > $index )
{
continue;
}
// match the url from the element
preg_match( '/<a href="(.+)">/', $line, $url ); $url = $url[1];
$file = basename( $url );
// check if the file already exists
if ( !file_exists( $storage.$file )) //edited
{
file_put_contents( $storage.$file, file_get_contents( $url ) );
}
// update the lock file
file_put_contents( $lockFile, $index );
}
fclose($handle);
}
else
{
throw new Exception( 'Could not open file.' );
}
you can do something like this, of course you should also add here some error checking things :)
define("SITE_DIR", '/home/www/temp');
$file = file('in.txt');
foreach ($file AS $row){
preg_match('/(?<=\")(.*?)(?=\")/', $row, $url);
$path = parse_url($url[0], PHP_URL_PATH);
$dirname = pathinfo($path, PATHINFO_DIRNAME);
if (!is_dir(SITE_DIR . $dirname)){
mkdir(SITE_DIR . $dirname, 0777, true);
}
file_put_contents(SITE_DIR. $path, file_get_contents($url[0]));
}

Edit specific record from a line in text file and PHP

I'm trying to make a simple news hit counter with PHP and text file. i wrote a simple code to check and read the file:
Text File:
//Data in Source File
//Info: News-ID|Hits|Date
1|32|2013-9-25
2|241|2013-9-26
3|57|2013-9-27
PHP File:
//Get Source
$Source = ENGINE_DIR . '/data/top.txt';
$Read = file($Source);
//Add New Record
foreach($Read as $News){
//Match News ID
if($News[0] == "2"){
//Add New Record and Update the Text File
}
}
Problem is i can't change the news hits! For example, i need change hits from second line from 241 to 242 and write it again in to the txt file.
I searched in this site and Google and tried some ways but i couldn't fix that.
At the least, you're forgetting to write the increment back to the file. Also, you're going to want to parse each row into columns you can work with (delimited by a pipe |).
Untested code, but the idea is:
$Source = ENGINE_DIR . '/data/top.txt'; // you already have this line
$Read = file($Source); // and this one
foreach ( $Read as $LineNum => $News ) { // iterate through each line
$NewsParts = explode('|',$News); // expand the line into pieces to work with
if ( $NewsParts[0] == 2 ) { // if the first column is 2
$NewsParts[1]++; // increment the second column
$Read[$LineNum] = implode('|',$NewsParts); // glue the line back together, we're updating the Read array directly, rather than the copied variable $News
break; // we're done so exit the loop, saving cycles
}
}
$UpdatedContents = implode(PHP_EOL,$Read); // put the read lines back together (remember $Read as been updated) using "\n" or "\r\n" whichever is best for the OS you're running on
file_put_contents($Source,$UpdatedContents); // overwrite the file
You could read the file and do something like this:
//Get Source
$Source = ENGINE_DIR . '/data/top.txt';
$Read = file($Source);
$News = array();
foreach ($Read as $line) {
list($id, $views, $date) = explode('|', $line);
$News[$id] = array(
'id' => $id,
'views' => $views,
'date' => $date,
);
}
At this point you have the array $News which contains every news item and you can change them as you wish (example: $News[2]['views'] = 242;).
The only thing you're missing now is the writing back to the file part, which is also easy.
$fh = fopen(ENGINE_DIR . '/data/top.txt', 'w'); //'w' mode opens the file for write and truncates it
foreach ($News as $item) {
fwrite($fh, $item['id'] . '|' . $item['views'] . '|' . $item['date'] . "\n");
}
fclose($fh);
And that's it! :)

new line is not created when trying to update a text in a text file

Im trying to update a text file using the code below:
$filename = "flat-file-data.txt"; // File which holds all data
$rowToUpdate = $_REQUEST['number']; // This is line need to be updated
$newString = $_REQUEST['line'].'\r\n'; // This is what you want to replace it with
$arrFp = file( $filename ); // Open the data file as an array
// Replace the current element in array which needs to be updated with new string
$arrFp[$rowToUpdate-1] = $newString;
$numLines = count( $arrFp ); // Count the elements in the array
$fp = fopen( $filename, "w" ); // Open the file for writing
for($i=0; $i < $numLines; $i++)
{
fwrite($fp, $arrFp[$i]);
}
fclose( $fp ); // Close the file
as you can see from the above codes, it will update the line 1 with the data from $newString and it should be able to create a new line after each line with this '\r\n'.
Unfortunately, it doesnt create a new line, it just goes along with the same line or in the current line like for example 'this is the firstline\r\nthis is the second line'. Is there any accurate way that I could make it in a new line like
this is the firstline
this is the secondline
and so on and so forth?
You have to put \r\n in double quotes:
$newString = $_REQUEST['line']."\r\n";

Reading a file and returning the line containing

I have a file that contains 100 highscores for a game I'm scripting.
1.2345, name1
1.3456, name2
1.4567, name3
for example.
With php, I need to get the content of the line nameX appears on so I can overwrite it if the new score is higher than the old score. Also I need to find out which line number nameX appears on so they know what place (ranking) they are in.
Which php functions should I look into so that I can make this work?
You can either use fopen and fread or file for this one. Personally, I would opt for file as it sounds like this is a fairly small file to begin with.
$row = -1;
$fl = file( '/path/to/file' );
if( $fl )
{
foreach( $fl as $i => $line )
{
// break the line in two. This can also be done through subst, but when
// the contents of the string are this simple, explode works just fine.
$pieces = explode( ", ", $line );
if( $pieces[ 1 ] == $name )
{
$row = $i;
break;
}
}
// $row is now the index of the row that the user is on.
// or it is -1.
}
else
{
// do something to handle inability to read file.
}
For good measure, the fopen approach:
// create the file resource (or return false)
$fl = fopen( '/path/to/file', 'r' );
if( !$fl ) echo 'error'; /* handle error */
$row = -1;
// reads the file line by line.
while( $line = fread( $fl ) )
{
// recognize this?
$pieces = explode( ", ", $line );
if( $pieces[ 1 ] == $name )
{
// ftell returns the current line number.
$row = ftell( $fl );
break;
}
}
// yada yada yada
Here is a link that i always recommend, and it never failed so far.
Files in php
From Link:
<?php
// set file to read
$file = '/usr/local/stuff/that/should/be/elsewhere/recipes/omelette.txt' or die('Could not read file!');
// read file into array
$data = file($file) or die('Could not read file!');
// loop through array and print each line
foreach ($data as $line) {
echo $line;
}
?>
First you need to read all the file content out. Modifiy the line you desire, then put them all together back the file. This however will have performance and constitancy prolbems if you running scripts simultaneous.

Categories