I have a long string of values separated by tabs and I wish to cut the data as if using unix cut -f. If I use cut -f5 it cuts all my data into a single column of the value which is in the 5th position. Is there a PHP function that can do the same?
Below is an example of the raw file with each word in the row separated by a tab
The result would be as follows if I ran cut -f2:
I guess the answer really is "no", as far as I know. But you can combine a few PHP functions to achieve the same result.
You can use file to read the lines from the file into an array
$rows = file($path_to_your_file);
Then convert that array of strings to a multidimensional array
$rows = array_map(function($row){
return str_getcsv($row, "\t");
}, $rows);
Then get the column you want from that array.
$column_5 = array_column($rows, 4);
Not as concise as cut -f5, but PHP rarely is for things like this.
Incidentally, if you don't care that your PHP program will only work on systems that have unix cut, you can actually just use the cut -f in shell_exec.
Related
I am attempting to load many CSV files of historical data to a PGSQL server with PHP. I noticed PHP provides a pg_copy_from() function for loading local data into a remote server, but the exact format of the input is vague.
The documentation says it expects an array of rows, each row being a delimited string of the values in that row. PHP's file() function returns an array in a format as described, but feeding it into pg_copy_from() returns an error of:
$rows = file('path/to/data.csv', FILE_IGNORE_NEW_LINES); // Same result without ignore_new_lines
pg_copy_from($db, 'dest_table', $rows, ',');
'PHP Warning: pg_copy_from(): Copy command failed: ERROR: invalid input syntax for integer: ""col_2""'
I checked the array, $rows, and it's of a format like this:
[index]=>""foo",3,7,"bar",500,"z""
Where all string values and the string itself are in double quotes. I assume it's trying to load the entire string into the 1st column of my table, hence the error that col_2 is unable to load (cause there's nothing to load).
Are my assumptions about the array wrong or does the delimited string need a different format (if so, how)?
Thanks for any help/tips!
As noted in comments by #Wolph - just remove first element of your $rows array, because it contains headers. To remove first element of array array_shift can be used without reversing the array:
$rows = file('path/to/data.csv', FILE_IGNORE_NEW_LINES);
array_shift($rows);
pg_copy_from($db, 'dest_table', $rows, ',');
I have a 15,000 row PHP array. I need to iterate through each row to generate a 15,000 row Javascript array. Each row of the PHP array has a 5% chance of containing one or more HTML special characters like ó that I need to replace with the equivalent javascript hex. There are about 50 HTML special characters I have to look out for and replace, so I'd use str_replace(array_of_HTML_targets, array_of_hex_replacements, haystack). Is it more efficient to:
Go through each line of the PHP array, search for an ampersand, and if one exists do the search and replace (considering this will apply for only 5% of the rows)
Execute the search and replace on the entire array
Concatenate the array into one giant string and execute the search and replace on the giant string
Other idea? Please specify
Btw, reason for 15,000 PHP array is this is a data visualization app.
Since you already need to dump your PHP data into a string (probably JSON), you might as well work on the final string, like so:
$json = json_encode($your_php_array);
$unhtmlref = preg_replace_callback("/&#(x[0-9a-f]+|\d+);/",function($m) {
if( $m[1][0] == "x") $m[1] = substr($m[1],1);
else $m[1] = dechex($m[1]);
return sprintf("\\u%04s",$m[1]);
},$json);
This is safe, because HTML character codes don't have any special meaning in a JSON string.
That said, I have a function in my JavaScript "utility belt" that does something similar:
function unHTMLref(str) {
// take a string and return it, with all HTML character codes parsed
var div = document.createElement('div');
div.innerHTML = str.replace(/</g,"<");
return div.firstChild.nodeValue;
}
So basically you can either parse before, or after. Personally, I'd prefer "after" because it shifts some of the "grunt" work to the browser, allowing the server to do more important things.
Let's say I have text file Data.txt with:
26||jim||1990
31||Tanya||1942
19||Bruce||1612
8||Jim||1994
12||Brian||1988
56||Susan||2201
and it keeps going.
It has many different names in column 2.
Please tell me, how do I get the count of unique names, and how many times each name appears in the file using PHP?
I have tried:
$counts = array_count_values($item[1]);
echo $counts;
after exploding ||, but it does not work.
The result should be like:
jim-2,
tanya-1,
and so on.
Thanks for any help...
Read in each line, explode using the delimiter (in this case ||), and add it to an array if it does not already exist. If it does, increment the count.
I won't write the code for you, but here a few pointers:
fread reads in a line
explode will split the line based on a delimiter
use in_array to check if the name has been found before, and to determine whether you need to add the name to the array or just increment the count.
Edit:
Following Jon's advice, you can make it even easier for you.
Read in line-by-line, explode by delimiter and dump all the names into an array (don't worry about checking if it already exists). After you're done, use array_count_values to get every unique name and its frequency.
Here's my take on this:
Use file to read the data file, producing an array where each element corresponds to a line in the input.
Use array_filter with trim as the filter function to remove blank lines from this array. This takes advantage that trim returns a string having removed whitespace from both ends of its argument, leaving the empty string if the argument was all whitespace to begin with. The empty string converts to boolean false -- thus making array_filter disregard lines that are all whitespace.
Use array_map with a callback that involves calling explode to split each array element (line of text) into three parts and returning the second of these. This will produce an array where each element is just a name.
Use array_map again with strtoupper as the callback to convert all names to uppercase so that "jim" and "JIM" will count as the same in the next step.
Finally, use array_count_values to get the count of occurrences for each name.
Code, taking things slowly:
function extract_name($line) {
// The -1 parameter (available as of PHP 5.1.0) makes explode return all elements
// but the last one. We want to do this so that the element we are interested in
// (the second) is actually the last in the returned array, enabling us to pull it
// out with end(). This might seem strange here, but see below.
$parts = explode('||', $line, -1);
return end($parts);
}
$lines = file('data.txt'); // #1
$lines = array_filter($lines, 'trim'); // #2
$names = array_map('extract_name', $lines); // #3
$names = array_map('strtoupper', $names); // #4
$counts = array_count_values($names); // #5
print_r($counts); // to see the results
There is a reason I chose to do this in steps where each steps involves a function call on the result of the previous step -- that it's actually possible to do it in just one line:
$counts = array_count_values(
array_map(function($line){return strtoupper(end(explode('||', $line, -1)));},
array_filter(file('data.txt'), 'trim')));
print_r($counts);
See it in action.
I should mention that this might not be the "best" way to solve the problem in the sense that if your input file is huge (in the ballpark of a few million lines) this approach will consume a lot of memory because it's reading all the input in memory at once. However, it's certainly convenient and unless you know that the input is going to be that large there's no point in making life harder.
Note: Senior-level PHP developers might have noticed that I 'm violating strict standards here by feeding the result of explode to a function that accepts its argument by reference. That's valid criticism, but in my defense I am trying to keep the code as short as possible. In production it would be indeed better to use $a = explode(...); return $a[1]; although there will be no difference as regards the result.
While I do feel that this website's purpose is to answer questions and not do homework assignments, I don't acknowledge the assumption that you are doing your homework, since that fact has not been provided. I personally learned how to program by example. We all learn our own ways, so here is what I would do if I were to attempt to answer your question as accurately as possible, based on the information you have provided.
<?php
$unique_name_count = 0;
$names = array();
$filename = 'Data.txt';
$pointer = fopen($filename,'r');
$contents = fread($pointer,filesize($filename));
fclose($pointer);
$lines = explode("\n",$contents);
foreach($lines as $line)
{
$split_str = explode('|',$line);
if(isset($split_str[2]))
{
$name = strtolower($split_str[2]);
if(!in_array($name,$names))
{
$names[] = $name;
$unique_name_count++;
}
}
}
echo $unique_name_count.' unique name'.(count($unique_name_count) == 1 ? '' : 's').' found in '.$filename."\n";
?>
I have a list of 50,000 ID's in a flat file and need to remove any duplicate ID's. Is there any efficient/recommended algorithm for my problem?
Thanks.
You can use the command line sort program to order and filter the list of ids. This is a very efficient program and scales well too.
sort -u ids.txt > filteredIds.txt
Read into a dictionary line by line, discarding duplicates. When all read, write out to a new file.
I've did some experiments once and the fastest solution I could get in PHP was by sorting the items and manually remove all the duplicate items.
If performance isn't that much of an issue for you (which I suspect, 50,000 is not that much) than you can use array_unique(): http://php.net/array_unique
i guess if you have large enough memory allowance, you can put all these ids in array
$array[$id] = $id;
this would automatically weed out the dupes.
You can do:
file_put_contents($file,implode("\n",array_unique(file($file)));
How it works?
Read the file using function file
which returns an array.
Get rid of the duplicate lines using
array_unique
implode those unique lines with "\n"
to get a string
write the string back to the file
using file_put_contents
This solution assumes that you've got one ID per line in the flat file.
You can do it via array / array_unique, in this example i guess your ids are separated by line braks, if thats not the case just change it
$file = file_get_contents('/path/to/file.txt');
$array = explode("\n",$file);
$array = array_unique($array);
$file = implode("\n",$array);
file_put_contents('/path/to/file.txt',$file);
If you can just explode the contents of the file on a comma (or any delimiter), then array_unique will produce the least (and cleanest) code, otherwise if your are parsing the file going with the $array[$id] = $id is the fastest and cleanest solution.
If you can use a terminal (or native unix execution), the easiest way: (assuming that there is nothing else in the file):
sort < ids.txt | uniq > filteredIds.txt
I am fetching an array of floats from my database but the array I get has converted the values to strings.
How can I convert them into floats again without looping through the array?
Alternatively, how can I fetch the values from the database without converting them to strings?
EDIT:
I am using the Zend Framework and I am using PDO_mysql. The values are stored one per column and that is a requirement so I can't serialize them.
array_map('floatval', $array) only works on single dimensional arrays.
I can't floatval the single elements when I use them because I have to pass an array to my flash chart.
The momentary, non-generic solution is to extract the rows and do array_map('floatval',$array) with each row.
You could use
$floats = array_map('floatval', $nonFloats);
There is the option PDO::ATTR_STRINGIFY_FETCHES but from what I remember, MySQL always has it as true
Edit: see Bug 44341 which confirms MySQL doesn't support turning off stringify.
Edit: you can also map a custom function like this:
function toFloats($array)
{
return array_map('floatval', $array);
}
$data = array_map('toFloats', $my2DArray);
How are you getting your data? mysql, mysqli or PDO, some other way or even some other database?
you could use array_map with floatval like so:
$data = array_map('floatval', $data);
but that still executes a loop and i think it assumes you only have one column in your data.
you're probably best of casting to float when you use your value, if you have to. php is likely to do a good job of interpreting it right anyway.
LOL... are you working on the same project I am tharkun?
I just finished (last night) creating something, in a ZF based project, that uses pdo_mysql to retrieve and format data and then output it as xml for use in a flash piece. The values were going in as strings but needed to be floats.
Since I'm also the one who wrote the part that gets the data and the one who created the database I just made sure the data was converted to float before it went into the database.
I simply cast the values as float as part of some other formatting, for what it is worth.
protected function _c2f($input)
{
$input = (float)$input;
$output = round(($input * 1.8) + 32, 2);
return $output;
}
I found an easy way for this operation.
You can do this just by adding foreach loop to your code. foreach loop is used to fetch your array string data one by one. and then, you can simply convert this by function number_format. i used 2 place after convert to float value. i.e it used to print value after dot value 2 place.
$example= array("12.20", "15.05", "55.70");
foreach($example as $float)
{
$update_value = number_format($float,2);
echo $update_value."<br>";
}
Not sure what you're asking here? You can cast a string to a float, using (float) $string, but since PHP is dynamically typed, that will happen anyway, when needed. There is no reason to do an explicit cast.
What are you using floating point values for?