Read, sort and split text file into blocks - php
I am trying to re-activate my php knowledge for the following task:
I have a larger textfile containing unsorted lines of comma separated informations, each value enclosed by a '"'.
Each line can be understood as a single dataset, the first value of the line tells me in which table the row belongs.
Now I need to read the file, sort the lines (so that the lines belonging to the same table are together), detect the different blocks and save them in seperate text files. After that, I can do a fast import into a mysql database using load data from infile..
So, I can open the file and sort the lines via this:
<?php
$lines = file("importfile_unsorted.txt");
natsort($lines);
file_put_contents("importfile_sorted.txt", implode($lines));
?>
This works. But now I get stucked. importfile_sorted.txt looks like this:
"AV1","0","0","0","0","0","0","0","0","0","0","0:0","0:0","0:0"
"AV2","0","0","0","0","0","0","0","0","0","0","0:0","0:0","0:0"
.... [this would be the first block, all these lines should be saved in "av.txt"
In the next line the new block "F" begins with several lines:
"F1","D","D","Deutsch",,,"0","W"
"F4","E","E","Englisch",,,"0","W"
"F7","K","K","Kath.Religionslehre",,,"0","W"
"F8","Ev","Ev","Evang.Religionslehre",,,"0","W"
"F9","Eth","Eth","Ethik",,,"0","W"
... [save all these lines beginning with Fxx into file f.txt and go to the next blocks]
"G1","PhL","PÜG"
"G2","ChL","ChÜ"
..
"K1","5a","5a",,"304","Ma","Wei","0","16","16","5",,,,,"1","1","0",,"0","0","0","0"
"K2","5b","5b",,"303","Wo","Hm","0","32","16","5",,,,,"1","1","0",,"0","0","0","0"
"K3","5c","5c",,"302","Gr","Ro","0","32","16","5",,,,,"1","1","0",,"0","0","0","0"
... and so on. Later, there are blocks with a fixed first column like this:
"PL","Di 1","Ba","Q12","Inf1","CoR1"
"PL","Di 1","Bb","Q12","F","Ü2"
"PL","Di 1","Eg","Q12","L","M23"
...
and
"PLS","Di 1","Am"," frei "
"PLS","Di 1","Bad"," ----"
"PLS","Di 1","Bk"," frei "
...
followed by several other blocks (L1... L97, M, R1... R40, U1... U560).
I know all possible "identifiers" (AVx, Fx, Gx, .. PL, PLS..) of the blocks, but it is also possible that a block is omitted and the input file does not a single line of it at all.
The input file contains about 4000 lines all together, so performance should not be too low (although it's not time-critical, the import is done maybe 10 times a year..).
So, is there a way of getting this done in a "smart" and fast way or should I read the input file line by line, detect and remember the first value, add the current line to a result string and loop until a new first value occurs?
Thanks for your help!
Heiko
Use the built in CSV parser, don't split this manually
http://php.net/manual/en/function.str-getcsv.php
Related
Vim: sort al use statements inside a php file
I'll like to learn how to select all lines that starts with use, inside current php file, and then sort them. Until now: I've worked with :sort command. Also, ... I've remapped sort command: map <Leader>s :sort<CR> But I'll like to create a complex function that select all use statements inside a php file, and after that, that sort all that lines. I know how to sort. But I have no idea of what is the way to select some arbitrary group of strings with VimL function! SortUseStatements() let g:select='/^use .*$<CR>V' !?!?!?!?!?!?!?!?!? exec g:select endfunction map <F2> :call SortUseStatements()<CR> I know that I want to execute commands like: go to the beginning of current file sarch first ^use .*;$ statement start selection go to the end of current file sarch last ^use .*;$ statement and finally, run :sort
This can be neatly done with :help cmdline-ranges. You want the search to start at the beginning of the file, so your initial address is 0 (to start before the first line). Then, search for the first line beginning with use. When separated with ; the cursor position will be set to that line before interpreting the next line specifier. Then, you need to navigate to the last of the use statements (starting from the first use line, so again separated by ;). I first choose an easy way that searches for an empty line, and then takes the previous one. With :print, you can check that the range is fine before replacing the command with :sort: :0;/^use /;/^$/-1 print If there's no empty line after the use block, you have to use a more complex pattern for any line not starting with use: :0;/^use /;/^\(use \)\#!/-1 print Alternatives For anything more complex, a single :range probably won't do. I would then determine the start and end line numbers separately, using let lnum = search(...) with appropriate patterns. Then you can insert the numbers into an Ex command via :execute startLnum . ',' . endLnum . 'sort'
New Lines inside DB dont get displayed in textarea
Some strange behaviour with the same script on different Websites here. Here you go:
I have an textarea where users enter text. They also do newlines using enter. That is stored inside the mysql db like this:
Line One\r\nLine Two\r\n\r\nLine Three
The problem occures when displaying that text inside a textarea again.
In most cases everything works fine. The following is displayed inside the textarea:
Line One
Line Two
Line Three
On some websites however only the following is displayed inside the textarea:
Line One
Line Two
Line Three
Inside the db the two newlines (\r\n\r\n) are still there. Only one is displayed however.
This happens like i said only on some websites, in most cases multiple newlines like this \r\n\r\n are displayed fine.
UPDATE:
Seams like JSON has something to do with it. It is an JSON string containing a string like "Line One\r\nLine Two\r\n\r\nLine Three".
{"mystring":"Line One\r\nLine Two\r\n\r\nLine Three"}
$row = $db->load(); // get the json string
var_dump($row); // \r\n\r\n still inside!
$jrow = json_decode($row);
var_dump($jrow->mystring); // on some servers multiple newlines are chopped to one
Try using n2lbr when displaying the DB value in the textarea. Does that help at all? It's worked for me in the past.
How to add an array into a comma delimited text file without deleting previous array values
I'm trying to make a form where the user can add their own 'questions + answers' to the quiz.
I loaded the original questions from a text file. The added questions will then be processed by process_editadd.php
<?php
session_start();
$file = fopen('data.txt', 'r');
$array=$_SESSION['questions_array'];
//make array out of values
$q=array($_POST['question'],$_POST['one'],$_POST['two'],$_POST['three'],$_POST['four']);
//add to file
$file=fopen("data.txt","w+");
fwrite($file, implode(',', $q)).
header('Location:module.php');
?>
The array adds onto the text file, but the problem is that it replaces the whole thing. I don't want the questions to replace the previous ones, I just want them added. Do you guys know what's wrong with the code?
Note: I'm not allowed using mySQL or Javascript
You could switch to using an actual database and make your life a lot easier... Failing that, look into fputcsv and fgetcsv to make it a slightly less tedious problem. Your implode version right now is also vulnerable to CSV injection... you don't handle the case where any of the text you're writing MIGHT contain a comma. If it does, you'll suddenly find you'll have extra "columns" when you read the data back in later on.
How would I go about by creating a parser which has the ability to import txt into mysql
http://pastebin.com/raw.php?i=7NTGXU5R I have about a hundred of those listened in the same file I tried working on a php solution, but I wasn't sure how to parse the space, I could only find fgetcsv which does commas What direction should I head to in order to make sense I remember some C++ from years ago, I was thinking I do something like a getline, then store the line (or row in our case) into an array Once that is done, just write a bunch of if statements to go through each line and classify the first element (column) in each array to be the designated 'header'.
Tasks like that always boil down to a large amount of custom string-munching code. Your best weapons of choice will be regular expressions. Forget about fgetcsv if files look like your file does.
The basic logic might look something like this:
Fetch all rows of the file via the file function.
Save each table area to an own array containing the rows:
foreach ($filelines as $line)
{
$lefreportlines[] = trim(substr($line, 0, 93));
$middlereportlines[] = trim(substr($line, 67, 135));
...
}
When you're done, start processing each report as it deserves. For example, the leftmost report might simply be parsed with preg_split('/\s+/', $line);
Either way, you'll have a lot of work to do. Good luck!
php text file news updates
I am trying to make a news feed type thing in php. I have a text file - news.txt and a php file index.php. I have done the surrounding code and opening/closing the text file. Now I am stuck how to insert the new news item $newsnew to the top of the news.txt file and how to delete the old bottom news file in the news.txt file. Is there any way to do this without deleting the whole file and writing it all again? EDIT: Each news item is just a small string, say 500 characters, a single line.
Use a database. If you really must use text files, use a different file for every news-item and name them sequentially like: news001.txt news002.txt etc. Then you can just add and delete files, read the directory and display what´s there.
Use the file() function to import the items in news.txt as an array, and use array_unshift() to add the new first item, and array_pop() to remove the last item. Join the array back into a single string and write it to news.txt:
$items = file('news.txt');
array_unshift($items, 'New item 1');
array_pop($items);
$newstext = implode(PHP_EOL, $items);
// write $newstext to the external file
If this is a XML file you could read it, parse it and delete the last child in the DOM. But if you have all your data in a DB it could be much easier to rewrite the file every time. EDIT: after your edit: yes, you can do it like this: write your new line to a new file read the old file line by line and write it to the new one skip the last line (detected by counting or EOF) delete the old file and rename the new
No, there is not. But you might consider storing the messages in revers order. That way you only need to append to news.txt when new news arrive.
You are not going to be able to prepend to the beginning of the file without writing the whole thing out again. You could append to the end of it with the "a" mode flag to fopen(), but still to delete the oldest item you'll need to write out the entire file again. Really, a database is solution here instead of a single text file.
There are many ways you can do it using the flat text file, but I'm not really sure it it's worth it. You can use some lightweight structured file or embedded database. For example SQLite, which would store it in normal file, no additional setup needed.