I have a CSV file in which I want the first 11 lines to be removed. The file looks something like:
"MacroTrends Data Download"
"GOOGL - Historical Price and Volume Data"
"Historical prices are adjusted for both splits and dividends"
"Disclaimer and Terms of Use: Historical stock data is provided 'as is' and solely for informational purposes, not for trading purposes or advice."
"MacroTrends LLC expressly disclaims the accuracy, adequacy, or completeness of any data and shall not be liable for any errors, omissions or other defects in, "
"delays or interruptions in such data, or for any actions taken in reliance thereon. Neither MacroTrends LLC nor any of our information providers will be liable"
"for any damages relating to your use of the data provided."
date,open,high,low,close,volume
2004-08-19,50.1598,52.1911,48.1286,50.3228,44659000
2004-08-20,50.6614,54.7089,50.4056,54.3227,22834300
2004-08-23,55.5515,56.9157,54.6938,54.8694,18256100
2004-08-24,55.7922,55.9728,51.9454,52.5974,15247300
2004-08-25,52.5422,54.1672,52.1008,53.1641,9188600
I want only the stocks data and not anything else. So I wish to remove the first 11 lines. Also, there will be several text files for different tickers. So str_replace doesn't seem to be a viable option. The function I've been using to get CSV file and putting the required contents to a text file is
function getCSVFile($url, $outputFile)
{
$content = file_get_contents($url);
$content = str_replace("date,open,high,low,close,volume", "", $content);
$content = trim($content);
file_put_contents($outputFile, $content);
}
I want a general solution which can remove the first 11 lines from the CSV file and put the remaining contents to a text file. How do I do this?
Every example here won't work for large/huge files. People don't care about the memory nowadays. You, as a great programmer, want your code to be efficient with low memory footprint.
Instead parse file line by line:
function saveStrippedCsvFile($inputFile, $outputFile, $lineCountToRemove)
{
$inputHandle = fopen($inputFile, 'r');
$outputHandle = fopen($outputFile, 'w');
// make sure you handle errors as well
// files may be unreadable, unwritable etc…
$counter = 0;
while (!feof($inputHandle)) {
if ($counter < $lineCountToRemove) {
fgets($inputHandle);
++$counter;
continue;
}
fwrite($outputHandle, fgets($inputHandle) . PHP_EOL);
}
fclose($inputHandle);
fclose($outputHandle);
}
I have a CSV file in which I want the first 11 lines to be removed.
I always prefer to use explode to do that.
$string = file_get_contents($file);
$lines = explode('\n', $string);
for($i = 0; $i < 11; $i++) { //First key = 0 - 0,1,2,3,4,5,6,7,8,9,10 = 11 lines
unset($lines[$i]);
}
This will remove it and with implode you can create a new 'file' out of it
$new = implode('\n',$lines);
$new will contain the new file
Did'nt test it, but I'm pretty sure that this will work
Be carefull! I will quote #emix his comment.
This will fail spectacularly if the file content exceeds available PHP memory.
Be sure that the file isn't to 'huge'
Use file() to read it as array and simply trim first 11 lines:
$content = file($url);
$newContent = array_slice($content, 12);
file_put_contents($outputFile, implode(PHP_EOL, $newContent));
But answer these questions:
Why there is additional content in this CSV?
How will you know how much lines to cut off? What if it's more than 11 lines to cut?
Related
I am working on a program that parses text files uploaded by a user and then saves the parsed XML file on the server. However, when I write the XML file I get some the text
at the end of each line. This text is not in my original text file. I didn't even notice it until I opened the new XML file to verify that it was righting all of the content. Has anyone ran into this before and if so can you tell me if it's due to the way I'm creating and writing my file?
fileUpload.php - These 3 lines occur when the user uploads the file.
$fileName = basename($_FILES['fileaddress']['name']);
$fileContents = file_get_contents($_FILES['fileaddress']['tmp_name']);
$xml = $parser->parseUnformattedText($fileContents);
$parsedFileName = pathinfo($fileName, PATHINFO_FILENAME) . ".xml";
file_put_contents($parsedFileName, $xml);
parser.php
function parseUnformattedText($inputText, $bookName = "")
{
//create book, clause, text nodes
$book = new SimpleXmlElement("<book></book>");
$book->addAttribute("bookName", $bookName);
$conj = $book->addChild("conj", "X");
$clause = $book->addChild("clause");
$trimmedText = $this->trimNewLines($inputText);
$trimmedText = $this->trimSpaces($inputText);
$text = $clause->addChild("text", $trimmedText);
$this->addChapterVerse($text, "", "");
//make list of pconj's for beginning of file
$pconjs = $this->getPconjList();
//convert the xml to string
$xml = $book->asXml();
//combine the list of pconj's and xml string
$xml = "$pconjs\n$xml";
return $xml;
}
Input text file
1:1 X
it seemed good to me also,
X
having had perfect understanding of all things from the very first
to write you an orderly account, [most] excellent Theophilius
and
1:4
that
you may know the certainty of those things in which you were instructed
1:5 X
There was in the days of Herod, the king of Judea and a certain priest named Zacharias
X
his wife[was] of the daughters of Aaron
and
her name [was] Elizabeth.
1:8 So
it was,
that
while he was serving as priest 1:9 before God in the order of his division,
1:10 and
the whole multitude of the people was praying outside at the hour of incense
but
therefore
it was done.
Going off of Seroczynski's answer I was able to create a function that trimmed removed any carriage returns from the text. The XML output looked fine after that. Here's the function I used to fix the issue:
function trimCarriageReturns($text)
{
$textOut = str_replace("\r", "\n", $text);
$textOut = str_replace("\n\n", "\n", $textOut);
return $textOut;
}
is the ASCII character for \r\n which doesn't seem to come out correctly from parseUnformattedText().
Try $xml = nl2br($parser->parseUnformattedText($fileContents));
My colleague and I are working on a chat application for a small Flash based game. We would like to keep our chat file as small as possible by automatically deleting old text after the file has reached a certain limit. Say the file exceeds 50 lines, we would like to delete the existing information and begin again at line 1. Is this possible?
<?php
$file = "saved.txt";
$edited_text = $_POST['new_text'];
$open = fopen($file, "a+");
fwrite($open, "\n" . $edited_text);
fclose($open);
?>
Basically something like this:
$lines = file('saved.txt');
$lines[] = 'new line of text';
array_unshift($lines); // remove first array element
file_put_contents('saved.txt', implode(PHP_EOL, $lines));
Read the file into an array, one line per array element
Append your new line(s) of text
Remove as many lines from the start of the array as necessary
dump array back out to file as text.
This would work:
// Read entire file
$lines = file_get_contents('saved.txt');
// Add what you want to the beginning of array
array_unshift($lines, 'new line of text');
// Keep first 50 items
$lines = array_splice($lines, 0, 50);
// Write them back
file_put_contents('saved.txt', implode(PHP_EOL, $lines));
Will always keep the first 50 elements intact (which includes messages from new to old).
This question already has answers here:
Need to write at beginning of file with PHP
(10 answers)
Closed 9 years ago.
Hi I want to append a row at the beginning of the file using php.
Lets say for example the file is containing the following contnet:
Hello Stack Overflow, you are really helping me a lot.
And now i Want to add a row on top of the repvious one like this:
www.stackoverflow.com
Hello Stack Overflow, you are really helping me a lot.
This is the code that I am having at the moment in a script.
$fp = fopen($file, 'a+') or die("can't open file");
$theOldData = fread($fp, filesize($file));
fclose($fp);
$fp = fopen($file, 'w+') or die("can't open file");
$toBeWriteToFile = $insertNewRow.$theOldData;
fwrite($fp, $toBeWriteToFile);
fclose($fp);
I want some optimal solution for it, as I am using it in a php script. Here are some solutions i found on here:
Need to write at beginning of file with PHP
which says the following to append at the beginning:
<?php
$file_data = "Stuff you want to add\n";
$file_data .= file_get_contents('database.txt');
file_put_contents('database.txt', $file_data);
?>
And other one here:
Using php, how to insert text without overwriting to the beginning of a text file
says the following:
$old_content = file_get_contents($file);
fwrite($file, $new_content."\n".$old_content);
So my final question is, which is the best method to use (I mean optimal) among all the above methods. Is there any better possibly than above?
Looking for your thoughts on this!!!.
function file_prepend ($string, $filename) {
$fileContent = file_get_contents ($filename);
file_put_contents ($filename, $string . "\n" . $fileContent);
}
usage :
file_prepend("couldn't connect to the database", 'database.logs');
My personal preference when writing to a file is to use file_put_contents
From the manual:
This function is identical to calling fopen(), fwrite() and fclose()
successively to write data to a file.
Because the function automatically handles those three functions for me I do not have to remember to close the resource after I'm done with it.
There is no really efficient way to write before the first line in a file. Both solutions mentioned in your questions create a new file from copying everything from the old one then write new data (and there is no much difference between the two methods).
If you are really after efficiency, ie avoiding the whole copy of the existing file, and you need to have the last inserted line being the first in the file, it all depends how you plan on using the file after it is created.
three files
Per you comment, you could create three files header, content and footer and output each of them in sequence ; that would avoid the copy even if header is created after content.
work reverse in one file
This method puts the file in memory (array).
Since you know you create the content before the header, always write lines in reverse order, footer, content, then header:
function write_reverse($lines, $file) { // $lines is an array
for($i=count($lines)-1 ; $i>=0 ; $i--) fwrite($file, $lines[$i]);
}
then you call write_reverse() first with footer, then content and finally header. Each time you want to add something at the beginning of the file, just write at the end...
Then to read the file for output
$lines = array();
while (($line = fgets($file)) !== false) $lines[] = $line;
// then print from last one
for ($i=count($lines)-1 ; $i>=0 ; $i--) echo $lines[$i];
Then there is another consideration: could you avoid using files at all - eg via PHP APC
You mean prepending. I suggest you read the line and replace it with next line without losing data.
<?php
$dataToBeAdded = "www.stackoverflow.com";
$file = "database.txt";
$handle = fopen($file, "r+");
$final_length = filesize($file) + strlen($dataToBeAdded );
$existingData = fread($handle, strlen($dataToBeAdded ));
rewind($handle);
$i = 1;
while (ftell($handle) < $final_length)
{
fwrite($handle, $dataToBeAdded );
$dataToBeAdded = $existingData ;
$existingData = fread($handle, strlen($dataToBeAdded ));
fseek($handle, $i * strlen($dataToBeAdded ));
$i++;
}
?>
I am trying to use a php call through AJAX to replace a single line of a .txt file, in which I store user-specific information. The problem is that if I use fwrite once getting to the correct line, it leaves any previous information which is longer than the replacement information untouched at the end. Is there an easy way to clear a single line in a .txt file with php that I can call first?
Example of what is happening - let's say I'm storing favorite composer, and a user has "Beethoven" in their .txt file, and want's to change it to "Mozart", when I used fwrite over "Beethoven" with "Mozart", I am getting "Mozartven" as the new line. I am using "r+" in the fopen call, as I only want to replace a single line at a time.
If this configuration data doesn't need to be made available to non-PHP apps, consider using var_export() instead. It's basically var_dump/print_r, but outputs the variable as parseable PHP code. This'd reduce your code to:
include('config.php');
$CONFIG['musician'] = 'Mozart';
file_put_contents('config.php', '<?php $CONFIG = ' . var_export($CONFIG, true));
This is a code I've wrote some time ago to delete line from the file, it have to be modified. Also, it will work correctly if the new line is shorter than the old one, for longer lines heavy modification will be required.
The key is the second while loop, in which all contents of the file after the change is being rewritten in the correct position in the file.
<?php
$size = filesize('test.txt');
$file = fopen('test.txt', 'r+');
$lineToDelete = 3;
$counter = 1;
while ($counter < $lineToDelete) {
fgets($file); // skip
$counter++;
}
$position = ftell($file);
$lineToRemove = fgets($file);
$bufferSize = strlen($lineToRemove);
while ($newLine = fread($file, $bufferSize)) {
fseek($file, $position, SEEK_SET);
fwrite($file, $newLine);
$position = ftell($file);
fseek($file, $bufferSize, SEEK_CUR);
}
ftruncate($file, $size - $bufferSize);
echo 'Done';
fclose($file);
?>
I have been struggling to create a Simple ( really simple ) chat system for my website as my knowledge on Javascripting/AJAX are Limited after gather resources and help from many kind people I was able to create my simple chat system but left with one problem.
The messages are posted to a file called "msg.html" in this format :
<p><span id="name">$name</span><span id="Msg">$message</span></p>
And then using PHP and AJAX I will retrieve the messages instantly from the file using the
file(); function and a foreach(){} loop withing PHP here is the code :
<?php
$file = 'msg.html';
$data = file($file);
$max_lines = 20;
if(count($data) > $max_lines){
// here i want the data to be deleted from oldest until i only have 20 messages left.
}
foreach($data as $line_num => $line){
echo $line_num . " . " . $line;
}
?>
My Question is how can i delete the oldest messages so that i am only left with the latest 20 Messages ?
How does something like this seem to you:
$file = 'msg.html';
$data = file($file);
$max_lines = 20;
foreach($data as $line_num => $line)
{
if ($line_num < $max_lines)
{
echo $line_num . " . " . $line;
}
else
{
unset($data[$line_num]);
}
}
file_put_contents('msg.html', $data);
?>
http://www.php.net/manual/en/function.file-put-contents.php for more info :)
I suppose you can read the file, explode it into an array, chop off everything but last 20 fields and write it back to file, overwriting the old one... Perhaps not the best solution but one that comes to mind if you really cant use database as Delan suggested
That's called round-robin if I recall correctly.
As far as I know, you can't remove arbitrary portions of a file. You need to overwrite the file with the new contents (or create a new file and remove the old one). You could also store messages in individual files but of course that implies up to $max_lines files to read.
You should also use flock() to avoid data corruption. Depending on the platform it's not 100% reliable but it's better than nothing.