How can we do further comparision or computation on the output of a console application. As I believe it is in ASCII form. I have called the console application in php which gives a huge amount of numeric datas. I want that data to be saved on an array and not in a txt file as the process of reading and writing a file takes time I want it to directly save it in an array. I used exec($command,$result) but I cannot get the result to be saved in proper form. My output is as shown below:
Columns 1 through 7
0.1373 0.0414 0.0541 0.1342 0.5606 0.5293 0.1652
Columns 8 through 14
0.0341 0.0396 0.0633 0.0778 0.0289 0.0654 0.0752
Columns 15 through 21
0.3055 0.4602 0.0631 0.0360 0.0188 0.0497 0.0228...........
I dont want the columns line to be saved in the array and I want each element in column to be saved in different index of array. Eg array[1]=0.1373 and array [2]=0.414.
My suggestion would be a simple regular expression that you run on each line of the output:
if (preg_match_all('/\d+\.\d+/', $line, $matches)) {
print_r($matches[0]);
}
If the expression matches, you will have an array of results that you can use for further processing. The output for the first line that matches from your post:
Array
(
[0] => 0.1373
[1] => 0.0414
[2] => 0.0541
[3] => 0.1342
[4] => 0.5606
[5] => 0.5293
[6] => 0.1652
)
You can catch the output of an external command using [s,t]=system(...).
The second output shall be the string for what you have in the file.
Then you can parse the string (with regexp for example or str split utils) to extract the data as you want.
Related
I am creating a web page on my CentOS server, where I want to traverse all my photos and videos and then show them on my page. However then it seems that when the files have our Danish national characters included, like æøåÆØÅ, then my exec command cannot access the file - and I need the exec command as I need mediainfo to show-and-tell the format and other details of the file (video, audio or image).
Let's assume I have this data and array that I am traversing (3 files):
$folder = "!My Folder"; // parent folder has a special char
Array
(
[0] => Array
(
[filename] => !file #2.jpg
[descr] => File with special chars, but no national chars
)
[1] => Array
(
[filename] => file with Danish æøå.jpg
[descr] => File with Danish chars, but no special chars
)
[2] => Array
(
[filename] => file with no special.jpg
[descr] => File with nothing special
)
)
I am then reading the mediainfo info from PHP like this:
$param = escapeshellarg("$folder/$filename"); // escape file argument
exec("mediainfo $param", $outputArray); // store line-by-line output in an array
This works fine for file [0] and [2] (I get a populated array), but [1] just returns an empty array from the output:
Array
(
[0] =>
)
As a note then I am able to use mediainfo directly on the server and doing this will work fine and return detailed data:
[usr#srv !My Folder]# mediainfo file\ with\ Danish\ æøå.jpg
So it seems to be the exec that has some problems with this?
I am using PHP 8.1 and I have no problem accessing or storing files on my server, via Samba, with these Danish characters.
An alternative solution would be to rename the files with these characters, but ideally I hope to avoid doing that as it is kind of "destructive" (messing with the original files).
Does anyone have a good idea how to access those files via exec?
### UPDATE 1 - BUT STILL NO SOLUTION ###
Just to make it crystal clear and to "prove" this is something related to exec then I refer to the answer from #CBroe below and adding one additional line to set the locale character encoding in PHP, setlocale + outputting the exec command:
setlocale(LC_CTYPE, "en_US.UTF-8");
$param = escapeshellarg("$folder/$filename");
exec("mediainfo $param", $outputArray);
echo "mediainfo $param";
The echo will output this (and an empty array):
mediainfo '/server/original/!My Folder/file with Danish æøå.jpg'
But if I run this exact same command directly on my server, mediainfo '/server/original/!My Folder/file with Danish æøå.jpg' then it will show me the media info for the file:
I also believe that this is some character encoding issue, but yet I do not know how to solve it ;-)
### UPDATE 2 - BUT STILL NO SOLUTION ###
As suggested below, then I also did try using putenv('LANG=en_US.UTF-8'), but for me that didn't help. I also tried using shell_exec() instead of exec() - same result and it did not help.
putenv("LANG=en_US.UTF-8");
$folder = "!My Folder"; // parent folder has a special char
$filename = "file with Danish æøå.jpg";
$param = escapeshellarg("$folder/$filename");
$output = shell_exec("mediainfo $param");
$outputArray = explode("\n",$output);
print_r($outputArray);
This will result in an array with two empty values:
Array
(
[0] =>
[1] =>
)
Somewhat late, but i stumbled across this today (pretty much the same, filename from glob() function and working further). This is not a problem of exec() but of the environment that mediainfo uses.
If you use UTF-8 filenames, it is essential to set the LANG environment correctly.
putenv('LANG=en_US.UTF-8');
worked for me.
Testing echo escapeshellarg("file with Danish æøå.jpg"); on https://3v4l.org/FEWfI only gives me 'file with Danish .jpg' as result.
Checking the user comments for the function, there is https://www.php.net/manual/en/function.escapeshellarg.php#99213:
When escapeshellarg() was stripping my non-ASCII characters from a UTF-8 string, adding the following fixed the problem:
<?php
setlocale(LC_CTYPE, "en_US.UTF-8");
?>
That indeed appears to fix the problem, https://3v4l.org/1DfpF - result now is 'file with Danish æøå.jpg'
I am using an associative array which I initialized like this:
$img_captions = array();
Then, later in the code I am filling it in a while loop with keys and values coming in from a .txt file (every line in that .txt file contains a pair - a string - separated by '|') looking like this:
f1.jpg|This is a caption for this specific file
f2.jpg|Yea, also this one
f3.jpg|And this too for sure
...
I am filling the associative array with those data like this:
if (file_exists($currentdir ."/captions.txt"))
{
$file_handle = fopen($currentdir ."/captions.txt", "rb");
while (!feof($file_handle) )
{
$line_of_text = fgets($file_handle);
$parts = explode('/n', $line_of_text);
foreach($parts as $img_capts)
{
list($img_filename, $img_caption) = explode('|', $img_capts);
$img_captions[$img_filename] = $img_caption;
}
}
fclose($file_handle);
}
When I test that associative array if it actually contains keys and values like:
print_r(array_keys($img_captions));
print_r(array_values($img_captions));
...I see it contains them as expected, BUT when I try to actually use them with direct calling like, let's say for instance:
echo $img_captions['f1.jpg'];
I get PHP error saying:
Notice: Undefined index: f1.jpg in...
I am clueless what is going on here - can anyone tell, please?
BTW I am using USBWebserver with PHP 5.3.
UPDATE 1: so by better exploring the output of the 'print_r(array_keys($img_captions));' inside Chrome (F12 key) I noticed something strange - THE FIRST LINE OF '[0] => f1.jpg' LOOKS VISUALLY VERY WEIRD tho it looks normal when displayed as print_r() output on the site, I noticed it actually in fact is coded like this in webpage source (F12):
Array
(
[0] => f1.jpg
[1] => f2.jpg
[2] => f3.jpg
[3] => f4.jpg
[4] => f5.jpg
[5] => f6.jpg
[6] => f7.jpg
[7] => f8.jpg
[8] => f9.jpg
[9] => f10.jpg
)
So when I tested anything else than the 1. line it works OK. I tryed to delete completely the file and re-write it once again but still the same occurs...
DISCLAIMER Guys, just to clarify things more properly: THIS IS NOT MY ORIGINAL CODE (that is 'done completely by me'), it is
actually a MiniGal Nano PHP photogalery I had just make to suit my
needs but those specific parts we are talking about are FROM THE
ORIGINAL AUTHOR
I will recommend you to use file() along wth trim().
Your code becomes short, readable and easy to understand.
$parts= file('your text file url', FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
$img_captions = [];
foreach($parts as $img_capts){
list($img_filename, $img_caption) = explode('|', $img_capts);
$img_captions[trim(preg_replace("/&#?[a-z0-9]+;/i","",$img_filename))] = trim(preg_replace("/&#?[a-z0-9]+;/i","",$img_caption));
}
print_r($img_captions);
So after a while I realize there is something wrong with my .txt file itself as it:-
ALWAYS PUT SOME STRANGE SIGNS IN FRONT OF THE 1st LINE WHATEVER I DO IT WAS ALWAYS THERE EVEN WITH THE NEW FILE CREATED FROM SCRATCH (although those are UNVISIBLE unless seen as a source code on a webpage!!!)
So I decided to test it in another format, this time .log file and all of a sudden everything works just fine.
I do not know if it is just my local problem of some sort (most probably is) or something else I am not aware of.
But my solution to this was changing the file type holding the string pairs (.txt => .log) which solved this 'problem' for me.
Some other possible solution to this as #AbraCadaver said:
(Those strange signs: [0] => f1.jpg) That's the HTML entity for a BYTE ORDER MARK or BOM, save your file
with no BOM in whatever editor you're using.
I'm working on an application that uses some saved data to work with it later.
I can't use database so this have to be file based.
The thing works like this:
A file name called hosts.txt that contains host:service on every line (diff hosts and services).
My PHP file reads the txt line by line, splits by the delimiter ":" and makes a request with the data that has just recived.
Untill here, cool. I also have another file (a HTML form) that allows me to add that at the end of the file. But my problem is the next:
I want to be able to add and remove that from the file so when the next check is done (the check is made every 30 sec) the data should be updated.
Example:
host.txt contains this:
host1:service1
host1:service2
host2:service2
host3:service1
Now I want to be able to add a new host:service to the list (I've already done this by adding the new data at the end of the text file then that will be included on the next check when php will read again line by line.
Now now, how can I remove a host:service from the file ?
I mean, after reading the file, the PHP will make something like this:
Host: Host1 | Service: Service1 | Status: Warning (this will depend on the HTTP request I will make) (X)
Host: Host1 | Service: Service2 | Status: OK (X)
Host: Host3 | Service: Service1 | Status: Critical (X)
I want to be able to remove a host:service from the list (and the file) just by clicking that (X). Is this possible guys? I'm a bit lost ( It will be easy for me to work with database but I can't on this project).
I hope I will get something clear.
Thanks in advance,
Regards.
In PHP you want to do something like this:
Code
$string = 'host1:service1
host1:service2
host2:service2
host3:service1';
$splitted = explode("\n", $string);
$data = array();
foreach ($splitted as $key => $value) {
$data[$key] = explode(':', $value);
}
echo '<pre>';
print_r($data);
echo '<pre>';
You now have structured data you can do checks with if certain values match or are specific categories. The control over the output is now fully in your hands.
Output
Array
(
[0] => Array
(
[0] => host1
[1] => service1
)
[1] => Array
(
[0] => host1
[1] => service2
)
[2] => Array
(
[0] => host2
[1] => service2
)
[3] => Array
(
[0] => host3
[1] => service1
)
)
If you still have issues outputting your data, feel free to ask in a comment and I'll update the answer.
Roughly:
1) Do an explode on the enters in the textfile so that you have a numbered array.
2) set an id for each line in the html (integer for each line number)
3) make a jquery function that has a "click" listener
4) create an AJAX script that has the integer as input
5) read the hosts.txt again, remove the line which corresponds with the number (unset($array['linenumber']))
6) implode with enters, write to file (overwriting the old one)
7) reload page (or hide the row)
I have a file, "serialized.txt", which contains a serialized array (created by doing serialize($array)).
s:133:"a:7:{i:0;i:640;i:1;i:480;i:2;i:2;i:3;s:24:"width="640" height="480"";s:4:"bits";i:8;s:8:"channels";i:3;s:4:"mime";s:10:"image/jpeg";}";
To fetch the contents I do:
$string = file_get_contents("serialized.txt");
Then I do:
print_r(unserialize($string));
The output that I get:
a:7:{i:0;i:640;i:1;i:480;i:2;i:2;i:3;s:24:"width="640" height="480"";s:4:"bits";i:8;s:8:"channels";i:3;s:4:"mime";s:10:"image/jpeg";}
This is the unserialized version of the string (contents of the file) when it should be printing the unserialized array. If I copy the string and do the following:
print_r(unserialize('a:7:{i:0;i:640;i:1;i:480;i:2;i:2;i:3;s:24:"width="640" height="480"";s:4:"bits";i:8;s:8:"channels";i:3;s:4:"mime";s:10:"image/jpeg";}'));
I get the correct output:
Array
(
[0] => 640
[1] => 480
[2] => 2
[3] => width="640" height="480"
[bits] => 8
[channels] => 3
[mime] => image/jpeg
)
So the problem seems to be isolated to the serialized array when pulling from the file.
According to the unserialize docs the function should be returning false if there is a problem; not the contents of the string.
The serialized data is taken from getimagesize and I have verified that if I serialize another array and place it into the file:
serialize(array("hi"));
I can successfully generate the output:
Array
(
[0] => hi
)
Are there any ideas why this may be happening? A bug with the serialization process relating to a getimagesize array, or potentially a "hidden" character in the file that my copy and paste removes? I have millions of these files already generated so it's not possible for me to change the storage method. I guess the solution may just be to write my own parser to serialize the array? The input is always the same format so that's plausible, but I would like to know of this a bug or my error with something somewhere.
As far as I can see your data is double serialized so the following code should print your array:
$string = file_get_contents("serialized.txt");
print_r(unserialize(unserialize($string)));
Although you should think about how you save to file. You may want to remove one serialization.
Does that solve your problem?
I'm trying to help my dad out -- he gave me an export from a scheduling application at his work. We are trying to see if we can import it into a mysql database so he/co-workers can collaborate online with it.
I've tried a number of different methods but none seem to work right -- and this is not my area of specialties.
Export can be seen here: http://roikingon.com/export.txt
Any help / advice on how to go about parsing this would be greatly appreciated!
Thanks !!
I've made an attempt to write a (somewhat dynamic) fixed-with-column parser. Take a look: http://codepad.org/oAiKD0e7 (it's too long for SO, but it's mostly just "data").
What I've noticed
Text-Data is left aligned with padding on the right like "hello___" (_ = space)
Numerical data is right aligned with padding on the left "___42"
If you want to use my code there's yet stuff to do:
The record types 12.x have variable column count (after some static columns), you'd have to implement another "handler" for it
Some of my width's are most probably wrong. I think there is a system (like numbers are 4 characters long and text 8 characters long, with some variations for special cases). Someone with domain knowledge and more than one sample file could figure out the columns.
Getting the raw-data out is only the first step, you have to map the raw-data to some useful model and write that model to the database.
With that file structure you're basically in need of reverse engineering a proprietary format. Yes, it is space delimited but the format does not follow any kind of standard like CSV, YAML etc. It is completely proprietary with what seems to be a header and separate section with headers of their own.
I think your best bet is to try and see if there's some other type of export that can be done such as Excel or XML and working from there. If there isn't then see if there's an html output of some kind that can be screen scraped, and pasted into Excel and seeing what you get.
Due to everything I mentioned above it will be VERY difficult to massage the file in its current form into something that can be sensibly imported into a database. (Note that from the file structure a number of tables would be needed.)
you can use split with a regular expression (zero or more spaces).
I will try and let you know.
There doesnt seem to be a structure with you data.
$data = "12.1 0 1144713 751 17 Y 8 517 526 537 542 550 556 561 567 17 ";
$arr = preg_split("/ +/", $data);
print_r($arr);
Array
(
[0] => 12.1
[1] => 0
[2] => 1144713
[3] => 751
[4] => 17
[5] => Y
[6] => 8
[7] => 517
[8] => 526
[9] => 537
[10] => 542
[11] => 550
[12] => 556
[13] => 561
[14] => 567
[15] => 17
[16] =>
)
Try this preg_split("/ +/", $data); which splits the line by zero or more spaces, then you will have a nice array, that you can process. But looking at your data, there is no structure, so you will have to know which array element corresponds to what data.
Good luck.
Open it with excel and save it as comma-delimited. Treat consecutive delimiters as one, or not. Then resave it with excel as a csv, which will be comma-separated and easier to import to mysql.
EDIT:
The guy who says to use preg_split on "[ +]" is giving you essentially the same answer as I just did above.
The question is what to do after that, then.
Have you determined yet how many "row types" there are? Once you've determined that and defined their characteristics it will be a lot easier to write some code to go through it.
If you save it in csv, you can use the PHP fgetcsv function and related functions. For each row, you would check it's type and perform operations depending on the type.
I noticed that your data rows could possibly be divided on whether or not the first column's data contains a "." so here's an example of how you might loop through the file.
while($row = fgetcsv($file_handle)) {
if(strpos($row[0],'.') === false) {
// do something
} else {
// do something else
}
}
"do something" would be something like "CREATE TABLE table_$row[0]" or "INSERT INTO table" etc.
Ok, and here's some more observation:
Your file is really like multiple files glued together. It contains multiple formats. Notice all the rows starting with "4" next have a 4-letter company abbreviation followed by full company name. One of them is "caco". If you search for "caco", you find it in multiple "tables" within the file.
I also notice "smuwtfa" (days of the week) sprinkled around.
Use clues like that to determine the logic of how to treat each row.