My text file sample.txt. I want to exclude the first row from the text file and store the other rows into mysql database.
ID Name EMail
1 Siva xyz#gmail.com
2 vinoth xxx#gmail.com
3 ashwin yyy#gmail.com
Now I want to read this data from the text file except the first row(ID,name,email) and store into the MYsql db.Because already I have created a filed in database with the same name.
I have tried
$handle = #fopen($filename, "r"); //read line one by one
while (!feof($handle)) // Loop till end of file.
{
$buffer = fgets($handle, 4096); // Read a line.
}
print_r($buffer); // It shows all the text.
Please let me know how to do this?
Thanks.
Regards,
Siva R
It's easier if you use file() since it will get all rows in an array instead:
// Get all rows in an array (and tell file not to include the trailing new lines
$rows = file($filename, FILE_IGNORE_NEW_LINES);
// Remove the first element (first row) from the array
array_shift($rows);
// Now do what you want with the rest
foreach ($rows as $lineNumber => $row) {
// do something cool with the row data
}
If you want to get it all as a string again, without the first row, just implode it with a new line as glue:
// The rows still contain the line break, since we only trimmed the copy
$content = implode("\n", $rows);
Note: As #Don'tPanic pointed out in his comment, using file() is simple and easy but not advisable if the original file is large, since it will read the whole thing into memory as an array (and arrays take more memory than strings). He also correctly recommended the FILE_IGNORE_NEW_LINES-flag, just so you know :-)
You can just call fgets once before your while loop to get the header row out of the way.
$firstline = fgets($handle, 4096);
while (!feof($handle)) // Loop till end of file.
{ ...
Related
I need help processing files holding about 46k lines or more than 30MB of data.
My original idea was to open the file and turn each line into an array element. This worked the first time as the array held about 32k values total.
The second time, the process was repeated, the array only held 1011 elements, and finally, the third time it could only hold 100.
I'm confused and don't know much about the backend array processes. Can someone explain what is happening and fix the code?
function file_to_array($cvsFile){
$handle = fopen($cvsFile, "r");
$path = fread($handle, filesize($cvsFile));
fclose($handle);
//Turn the file into an array and separate lines to elements
$csv = explode(",", $path);
//Remove common double spaces
foreach ($csv as $key => $line){
$csv[$key] = str_replace(' ', '', str_getcsv($line));
}
array_filter($csv);
//get the row count for the file and array
$rows = count($csv);
$filerows = count(file($cvsFile)); //this no longer works
echo "File has $filerows and array has $rows";
return $csv;
}
The approach here can be split in 2.
Optimized file reading and processing
Proper storage solution
Optimized file processing can be done like so:
$handle = fopen($cvsFile, "r");
$rowsSucceed = 0;
$rowsFailed = 0;
if ($handle) {
while (($line = fgets($handle)) !== false) { // Reading file by line
// Process CSV line and check if it was parsed correctly
// And count as you go
if (!empty($parsedLine)) {
$csv[$key] = ... ;
$rowsSucceed++;
} else {
$rowsFailed++;
}
}
fclose($handle);
} else {
// Error handling
}
$totalLines = $rowsSucceed + $rowsFailed;
Also you can avoid array_filter() simply by not adding processed line if its empty.
It will allow to optimize memory usage during script execution.
Proper storage
Proper storage here is needed for performing operations on certain amount of data. File reading are ineffective and expensive. Using simple file based database like sqlite can help you a lot and increase overall performance of your script.
For this purpose you probably should process your CSV directly to database and than perform count operation on parsed data avoiding excessive file line counts etc.
Also it gives you further advantage on working with data not keeping it all in memory.
Your question says you want to "turn each line into an array element" but that is definitely not what you are doing. The code is quite clear; it reads the entire file into $path and then uses explode() to make one massive flat array of every element on every line. Then later you're trying to run str_getcsv() on each item, which of course isn't going to work; you've already exploded all the commas away.
Looping over the file using fgetcsv() makes more sense:
function file_to_array($cvsFile) {
$filerows = 0;
$handle = fopen($cvsFile, "r");
while ($line = fgetcsv($handle)) {
$filerows++;
// skip empty lines
if ($line[0] === null) {
continue;
}
//Remove common double spaces
$csv[] = str_replace(' ', '', $line);
}
//get the row count for the file and array
$rows = count($csv);
echo "File has $filerows and array has $rows";
fclose($handle);
return $csv;
}
Hellooo,
I wrote myself a little PHP experiment. This script counts how many times the user clicked a button labeled with a specific class (id="link_1", class="heart")
During each click, the script reads a txt file, finds the right id, then adds +1 to that id's number, like so:
#counte_me.php
$file = 'count_me.txt'; // stores the numbers for each id
$fh = fopen($file, 'r+');
$id = $_REQUEST['id']; // posted from page
$lines = '';
while(!feof($fh)){
$line = explode('||', fgets($fh));
$item = trim($line[0]);
$num = trim($line[1]);
if(!empty($item)){
if($item == $id){
$num++; // increment count by 1
echo $num;
}
$lines .= "$item||$num\r\n";
}
}
fclose($fh);
file_put_contents($file, $lines, LOCK_EX);
The result
# count_me.txt
hello_darling||12
This works wonderfully well. The problem happens when, from time to time, I find myself staring at an empty count_me.txt!
Not really know when or how it happens, but it does. I start making increments and happens, sometimes sooner, sometimes way later. It may be on my way to 10 or to 200 or to 320 or anything in between. Its totally random.
Driving me crazy. I'm not experienced enough, but that's why I'm playing with this thing.
Someone knows what I am doing wrong here for the file to get dumped like that?
UPDATE 1
So far, Oluwafemi Sule's suggestion is working, but I have to remove LOCK_EX from the file_put_contents for it to work, otherwise it just doesn't.
// NEW LINE ADDED
if (!empty($lines)) {
file_put_contents($file, $lines);
}
$lines is initially set to an empty string and only updated on the following condition.
if(!empty($item)) {
# and so on and so on
}
And finally at the end,
file_put_contents($file, $lines, LOCK_EX);
The reason that $lines still remains set to the initial empty string happens when item is empty. Remember the newline added from "$item||$num\r\n", there could be more than a single line added there(I won't put it past a text editor to add a new line to end that file .)
I suggest to only write to the file when $lines isn't empty.
if (!empty($lines)) {
file_put_contents($file, $lines, LOCK_EX);
}
I'm trying to display only the rows that contain a specific word in a specific column. Basically I would like to show only the rows that have "yes" in the Display column.
First_Name, Last_Name, Display
Kevin, Smith, yes
Jack, White, yes
Joe, Schmo, no
I've been trying various things with fgetcsv & str_getcsv from other answers and from php.net but nothing is working so far.
It doesn't do anything but this is my current code:
$csv = fopen('file.csv', 'r');
$array = fgetcsv($csv);
foreach ($array as $result) {
if ($array[2] == "yes") {
print ($result);
}
}
Let's have a look at the documentation for fgetcsv():
Gets line from file pointer and parse for CSV fields
fgetcsv reads a single line, not the whole file. You can keep reading lines until you reach the end of the file by putting it in a while loop, e.g.
<?php
$csv = fopen('file.csv', 'r');
// Keep looping as long as we get a new $row
while ($row = fgetcsv($csv)) {
if ($row[2] == "yes") {
// We can't just echo $row because it's an array
//
// Instead, let's join the fields with a comma
echo implode(',', $row);
echo "\n";
}
}
// Don't forget to close the file!
fclose($csv);
You should use data tables.
https://datatables.net/examples/basic_init/zero_configuration.html
That's how I deal with my textfiles. But be carefull, with a large amount of Data (> 10000 rows) you should have a loog at the deferRender option.
https://datatables.net/reference/option/deferRender <-- JSON DATA required.
My colleague and I are working on a chat application for a small Flash based game. We would like to keep our chat file as small as possible by automatically deleting old text after the file has reached a certain limit. Say the file exceeds 50 lines, we would like to delete the existing information and begin again at line 1. Is this possible?
<?php
$file = "saved.txt";
$edited_text = $_POST['new_text'];
$open = fopen($file, "a+");
fwrite($open, "\n" . $edited_text);
fclose($open);
?>
Basically something like this:
$lines = file('saved.txt');
$lines[] = 'new line of text';
array_unshift($lines); // remove first array element
file_put_contents('saved.txt', implode(PHP_EOL, $lines));
Read the file into an array, one line per array element
Append your new line(s) of text
Remove as many lines from the start of the array as necessary
dump array back out to file as text.
This would work:
// Read entire file
$lines = file_get_contents('saved.txt');
// Add what you want to the beginning of array
array_unshift($lines, 'new line of text');
// Keep first 50 items
$lines = array_splice($lines, 0, 50);
// Write them back
file_put_contents('saved.txt', implode(PHP_EOL, $lines));
Will always keep the first 50 elements intact (which includes messages from new to old).
I am attempting to insert the data from an uploaded file into a single dimension array.
The file is as such that there is one student number to a line like so:
392232,392231,etc
this is the most common way I've found online:
while (($line = fgetcsv($file, 25, ',')) !== FALSE) {
//$line is an array of the csv elements
print_r($line);
}
However form what I understand this will create an array ($line) for each row. Which is not what I want.
that aside I tried this to see if it is working and my code is not printing out the array after using ftgetcsv(). The file is successfully uploading.
here is my code:
if(isset($_FILES['csv_file']) && is_uploaded_file($_FILES['csv_file']['tmp_name'])){
//create file name
$file_path = "csv_files/" . $_FILES['csv_file']['name'];
//move uploaded file to upload dir
if (!move_uploaded_file($_FILES['csv_file']['tmp_name'], $file_path)) {
//error moving upload file
echo "Error moving uploaded file";
}
print_r($_FILES['csv_file']);
$file = fopen('$file_path', 'r');
while (($line = fgetcsv($file, 25, ',')) !== FALSE) {
//$line is an array of the csv elements
print_r($line);
}
//delete csv file
unlink($file_path);
}
First off, can anyone obviously see why it wouldnt work to at least print them as seperate arrays of data (each row).
Second, is it possible to set it so that it creates a 1d array of all rows in the file?
Many thanks,
Question 1 is because of
print_r($_FILES['csv_file']);
$file = fopen('$file_path', 'r');
should be:
$file = fopen($file_path, 'r');
and for Question 2, check out the array_push
1st Question:
This line will actually try to open a file called '$file_path' because you're using single quotes (so it doesn't expand to the value of the variable). You can just remove the quotes.
$file = fopen('$file_path', 'r');
$file is null after this.
2nd Question:
If all you want to do is convert a file into an array by lines you can use one of these instead:
file() - get whole file into a 1D array of lines of the file (closest to what you want)
fgets() - get a string per line per call; keep calling this until it returns false to get each line one at a time
file_get_contents() - get the whole file into a string and process as you like
According to PHP.net $line has to return as array.
"returns an array containing the fields read."
But if you are sure it's contains only one student number you can use $line[0] to get the first line value (Ignoring the ",")
Here are some general comments on your code:
You are passing the file path into the fopen() function incorrectly. The variable should not be surrounded with single quotes.
Since you are deleting the CSV file after processing it, moving it is unnecessary. Simply use $_FILES['csv_file']['tmp_name'] as the path to the file.
Since there is only one entry per row in your CSV file, simply access the first element of the array that is returned from fgetcsv(): $numbers[] = $line[0];