I was using the PHP function parse_ini_file() to build an array structure from a INI File I have, but I was somewhat forced to create a custom parser because the native function has no support for nested files nor it parses the file comments.
Although a real INI files do not support complex hierarchy, with such feature I could easily manipulate other informations I have (or so I hope).
And the possibility to have the comments in the file is not really a requirement, but if I allow the INI file to be edited through a GUI, if I don't have access to them the resulting file would not make much sense.
And this "not required requirement" is the problem I can't solve.
Right... I'm publishing the class (and a supporting interface) in this Gist because it's a little big.
Do not judge the method Reader::process() too hard because this is not the real class. The real one involves an entire Stream Reader that would just complicate things if posted here. That fragment, however, is a valid replacement.
The INI File I'm using for testing purposes is this one:
[info]
; Comment Section 1: author
author="Bruno Augusto"
; Comment Section 1: copyright
copyright="MIT"
[descriptor]
; Comment Section 2: name
name="Item Name"
; Comment Section 2: description
description="Item Description"
; Comment Section 2: Line 1
; Comment Section 2, Line 2 (version)
version=1.0
[contents]
; Comment Section 3: Line 1
; Comment Section 3, Line 2 (files)
files[] = config.ini
files[] = folder/file1.php
files[] = folder/folder2/file2.php
files[] = folder/folder2/folder3/file3.php
files[] = folder/folder2/folder3/file4.php
[hierarchyTest]
; Comment Section 4: levels
levels[first\second\third] = "foo"
levels[first\second\third] = "anotherone"
levels[first\second\third] = "onemore"
levels[first\second\third\fourth] = "bar"
levels[first\second\third\fourth] = "baaz"
[hierarchyTest2]
; Comment Section 5: Line 1
; Comment Section 5, Line 2 (levels2)
levels2[first.second.third] = "baaz"
The usage is as simple as instantiate the Reader class with the path of the INI file and invoke the Reader::read() method. We're all big boys here, I think I can suppress this. :p
The buggy comments are proccesed starting in line 68 and added to the final structure in line 81*.
The problem is that currently all the values are being accumulated after each iteration. The one to blame is the array_merge() used in line 69. By removing it:
$comments = $line;
Almost everything works fine, except that multi-line comments holds only the last line as expected by a direct assignment.
Initially, I thought I could simply empty the array after use it so I've added such instruction in the line 84:
$comments = array();
And here is the weirdness. It works well for almost all entries, but the sections named contents and hierarchyTest have no comments added.
I've isolated the problem and it's because of the hierarchy parsing, starting at line 92 but I have no idea how to solve this.
This isolation is so true that if I add one more entry to the last section (hierarchyTest2) with the format:
levels2[first.second.third] = "baazaaa"
Its comments vanishes too.
Related
Quick update: The reason I need this solution is that this one php file is used to expand the flat file for about hundred users (that all use the same php file, but have their own flat files)
SOLUTION:
I worked with this one more day, rephrased the question and got a really great answer. I add it here for future help for others:
$content = file_get_contents("newstest.db");
$content = preg_replace('/(^ID:.*\S)/im', '$1value4:::', $content);
$content = preg_replace('/(^\d+.*\S)/im', '$1:::', $content);
file_put_contents("newstest.db", $content);
The original content of the flat file used when testing the code was:
ID:::value1:::value2:::value3:::
1:::My:::first:::line:::
2:::My:::second:::line:::
3:::Your:::third:::line:::
ORIGINAL QUESTION:
I have a PHP script I am trying to modify. Being a PHP newbie, and have searched both here and on Google without finding a solution, I ask here.
I need to add more values (columns) in the flat file, automatically if the "column" does not exist from before.
Because this one PHP file is shared with many users (each with their own flat file), I need a way to automatically add new "columns" in their flat files if the column does not exist. Doing it manually is very time consuming, and I bet there is an easy way.
INFO:
The flat file is named "newstest.db"
The flat file has this layout:
id:::title:::short:::long:::author:::email:::value1:::value2:::value3:::
So the divider is :::
I understand the basics, that I need to add for instance "value4:::" after "value3:::" in the first line of the news.db, then add ::: to the other existing lines to update all lines and prepare for the new "value4"
Today the php uses this to connect to the flat file:
($filesource is the path to the flat file including it's name. Unique for each user.)
$connect_to_file = connect_pb ($filesource);
And to write to the file I use:
insert_pb($filesource,"$new_id:::$title:::$short:::$long:::$author:::$email:::$value1:::::::::");
(As you see in this case value 2 and 3 is not used in this case, but are in others.)
QUESTION:
Is there a quick/ existing php code to use to add a new column if it doesn't already exist? Or do I need to make the php code for this specific task?
I understand that the code must do something along:
If "value4" does not exist in line 0 in $filesource
then add "value4:::" at the end of line 0,
and for each of the other lines add ":::" at the end.
I don't know where to start, but I have tried for some hours.
I understand this:
update_pb(pathtofiletosaveto,"id","x == (ID of news)","value in first line","value to add");
But I don't know how to make an if statement as in 1) above, neither how to update the line 0 in the flat file to add "value4:::" at the end etc.
MY CODE (does not work as intended):
OR, may be I need to read only line 1 in the file (newstest.db), and then exchange that with a new line if "value4" is not in line 1?
A suggestion, but I don't know how do all:
(It's probably full of errors, as I have tried to read up and find examples and combining code.)
<?php
// specify the file
$file_source="newstest.db";
// get the content of the file
$newscontent = file($file_source, true);
$lines ='';
// handle the content, add "value4:::" and ":::" to the lines
foreach ($newscontent as $line_num => $linehere) {
// add "value4:::" at the end of first line only, and put it in memory
if($line_num[0]) {$lines .= $linehere.'value4:::';}
else {
// then add ":::" to the other lines and add them to memory
$lines .= $linehere.':::';
}
// echo results just to see what is going on
//echo 'Line nr'.$line_num.':<br />'.$lines.'<br /><br />';
}
// add
// to show the result
echo "Here is the result:<br /><br />".$lines."<br /><br />";
//Write new content to $file_source
$f = fopen($file_source, 'w');
fwrite($f,$lines);
fclose($f);
echo "done updating database flat file";
?>
This ALMOST works...
But it does NOT add "value4:::" to the end of the first line,
and it does not add ":::" to the end of the next lines, but to the beginning...
So a couple of questions remains:
1) How can I search in line 0 after "value4", and then write "value4:::" at the end of the line?
2) How can I add ":::" at the end of each line, and not in the beginning?
I kindly ask you to either help me with this.
Do you absolutely have to use PHP for this task? It seems like something you only need to do once, and is much easier to do in a different way.
For example, if you have a *nix shell with sed, sed -i 's/$/:::/' <file> will do that task for you.
In Python you can parse an .ini file and access the single values like this:
myini.ini
[STRINGS]
mystring = fooooo
value = foo_bar
script.py
import configparser
config = configparser.ConfigParser()
# ----------------------------------------------------------------------
config.read("myini.ini")
test = config["STRINGS"]["mystring"]
print (test) #-> OUTPUT: fooooo
How can I do the same in PHP? Unfortunately, I was not able to find any examples.
Not to fear, parsing an .ini file is a standard method. (See parse_ini_file in the php docs).
Using your file as the base of this example:
myini.ini
[STRINGS]
mystring = fooooo
value = foo_bar
test.php
$ini_array = parse_ini_file("myini.ini");
print_r($ini_array); # prints the entire parsed .ini file
print($ini_array['mystring']); #prints "fooooo"
Note that by default parse_ini_file ignores sections and gloms all ini settings into the same object. If you'd like to have things scoped sectionally as in your python example, pass true for the process_sections parameter (second parameter).
test2.php
$ini_array = parse_ini_file("myini.ini", true /* will scope sectionally */);
print($ini_array['mystring']); #prints nothing
print($ini_array['STRINGS']['mystring']); #prints fooooo
From the intermediate approach, to get match section, it needs to read any INI file from line to line, in this case you can use fgets function or stream_get_line function to read for each line until the section founded or end of file (not founded).
Inside every line we can use preg_match (using regex pattern) that match with the actual section name been searched.
The essential of this approach is to reduce memory usage meanwhile many concurrent request occurs at the same time, so in this case we use string object with any sufficient length as a buffer.
Good luck Buddy.
I have the following code:
$l1 = file($file1['tmp_name']);// get file 1 contents
$l2 = file($file2['tmp_name']);// get file 2 contents
$l3 = array_diff($l1, $l2);// create diff array
Here are the files:
File 1:
6974527983
6974527984
6974527985
File 2:
6974527983
$l3 should be:
6974527984
6974527985
But, instead it is just spitting out the values from File 1:
6974527983
6974527984
6974527985
Am I setting this up right?
UPdate -
Using print_r(), I have verified that the files being loaded are being properly parsed into arrays:
File 1 -
Array ( [0] => 6974527983 [1] => 6974527984 [2] => 6974527985 ) 1
File 2 -
Array ( [0] => 6974527983 ) 1
So I don't believe there are any issues with the newlines in the text files.
If each number is on a new line, you could try splitting each file by line breaks and comparing the arrays that way.
$l1 = explode("\n", file($file1['tmp_name']));
$l2 = explode("\n", file($file2['tmp_name']));
$l3 = array_diff($l1, $l2);
Using the following example you can see that array_diff() works as expected:
$a = array(
6974527983,
6974527984,
6974527985
);
$b = array(
6974527983
);
var_dump(array_diff($a, $b));
Output:
array(2) {
[1] =>
int(6974527984)
[2] =>
int(6974527985)
}
This shows that file($file2['tmp_name']) is the problem in your case. Try:
var_dump(file($file2['tmp_name']));
to check the file's contents.
Okay, I will post an answer as I think this will solve your issue.
Without knowing more about the structure of your files, we can only assume that there is a possible issue with line endings. There are three possible line endings:
Unix: \n
Windows: \r\n
Classic mac: \r
I see two possible scenarios here:
The line endings in each file are different to each other
The line endings in both files are \r (classic mac)
As Mark Baker pointed out, you should use the FILE_IGNORE_NEW_LINES flag as the second argument for each of your file() calls. This, as far as I can make out from quickly experimenting here, should resolve the issue if one file had Unix and the other had Windows line endings.
However, it does not seem to deal well in cases where at least one file has '\r' line endings. In this case, there's an ini setting that might help:
ini_set('auto_detect_line_endings', true);
Consulting the docs for auto_detect_line_endings:
When turned on, PHP will examine the data read by fgets() and file() to see if it is using Unix, MS-Dos or Macintosh line-ending conventions.
This enables PHP to interoperate with Macintosh systems, but defaults to Off, as there is a very small performance penalty when detecting the EOL conventions for the first line, and also because people using carriage-returns as item separators under Unix systems would experience non-backwards-compatible behaviour.
So, TL;DR: debug your line endings to make sure you know what's going on (with file or hexdump or similar), and use a combination of auto_detect_line_endings and FILE_IGNORE_NEW_LINES.
Hope this helps :)
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
I have to work on a project involving huge amount of data stored in a raw text file. Each field is delimited by its size, ie, field 1 is from position 0 to 3, etc.. (not CSV file)
The file contains over a million lines.
I need to store it into a database. I checked several posts about what would be the best way to go about it, and it seems like the technology choice matters less than the algorithm. I'm open to Php, Perl or Python. Feel free to suggest anything.
Now, the file structure in itself is a bit tricky. Here is an example:
A880780093vvd47aa8db20d4133e6f587cf046054e8316000212093659D11001
C880780093d47aa8db20d4133e6f587cf046054e831600021209365907000 0711012012C
A880780093vvcaacb22bfb091127f9c9e14175d858ee25000212093681O11001
C880780093caacb22bfb091127f9c9e14175d858ee2500021209368107000 0611012012ADI
D880780093caacb22bfb091127f9c9e14175d858ee250002120936810700011012012HK00210Z
A880780093vvb92f937a3fd1268c1478deb174a1bfca86000212093750S11041
C880780093b92f937a3fd1268c1478deb174a1bfca8600021209375007000 3911012012PB
C880780093b92f937a3fd1268c1478deb174a1bfca8600021209375007000 3911012012B 1002
E880780093212093750b92f937a3fd1268c1478deb174a1bfca8600007000110120120100000127000000000000
C880780093b92f937a3fd1268c1478deb174a1bfca8600021209375007000 3911012012B
Basically, there are 6 types of lines, from A to F; line A is the header of the block. Lines B and C have the exact same length and fields. Line D is a possible complement to line C, meaning that it is attached to a line C but not required; also meaning there cannot be a line D without a line C. Lines E and F are independent lines, only attached to line A. (all lines are part of a block, so they could all be "attached" to a line A, or a virtual block ID)
How would I go about to create a model that would allow me to:
- modify some data on some lines based on some criteria (ie, if 5th char of line C is 4, then 10th becomes 7)
- keep track of the modified ones (ie, I want to be able to link them to their original selves)
- Be able to rebuild the original text file, deleting the original lines and replacing them by their modified version
- Be able to insert new lines in the block: if line C has 7th char = 0 then I add below it a D line.
- keep the line order intact. (if one line is inserted, it moves the order for the following line by 1 rank ahead)
I thought about using a parent_id foreign key in all 5 line tables (one per each line type, since they do not have the same fields); thus resolving the line ordering issue, but I am stuck at rebuilding the modified file version. I also thought about dividing the file into blocks (starting by a line A), then linking lines to block ID...
Any suggestion would be greatly appreciated!
Thanks a lot in advance!
Go through the file line by line and use a stack. Something along the lines:
<?php
// You'd have to implement the database yourself!
$db = new Database();
$db->startTransaction();
$stack = array();
$fh = fopen("my-file", "r");
$i = 0;
while (($buffer = fgets($handle, 4096)) !== false) {
if (!isset($buffer[0])) {
continue;
}
switch ($buffer[0]) {
case "A":
// Do something ...
break;
case "C":
// Do something ...
break;
case "C":
if ($stack[$i] != "C") {
trigger_error("Line D without preceding line C");
}
// Do something ...
break;
// More stuff ...
}
$stack[$i++] = $buffer;
$db->insert("INSERT INTO table (line) VALUES ($buffer)");
}
$db->commitTransaction();
?>
Of course there are better solutions than the ugly switch, but it's quick'n'dirty. Your database design answer is impossible to answer because we have no clue about the requirements. All in all consider posting your work and ask specific questions regarding a small piece of a big problem and not asking to solve big problems.
Using the fgets() function (or any other way) in PHP is there a way to read the LAST line of a file then work backwards?
What I am doing: (as per request) I am appending to a .txt file lines from an input (don't worry about that) and of course append adds to the END of a file so I need to display the contents of a file with one line being one entry to be displayed
So if the contents of the file are:
Hello, World!
Foo likes Bar!
then it needs to display as
1. Foo likes Bar!
2. Hello, World!
As far as I know PHP does not know the gets() function. But..
If you use 'file()' and inverse the array you can work it through from bottom to top. That should do the trick for you!
If it is a really big file, it might be better to use the following...
exec("tail -1 input_file > output_file") ;
$string = file_get_contents("output_file") ;
unlink("output_file") ;
This is based on UNIX/Linux system commands, but I'm sure Windows has a command similar to tail.