I have array of objects, for each objects i have name, category, price and couple other things. and from that array i want to remove objects that has specific category.
public function getCSV()
{
$contents = Storage::disk('dropbox')->get('InventoryReport.csv');
$lines = explode(PHP_EOL, $contents);
$items = array();
// categories to ignore when importing
$ignore = ['Misc', 'Soft Drinks', 'Juices', 'Water', 'Snack', 'Energy Drink', 'Tobacco', 'Gum', 'Account Payment'];
foreach ($lines as $line)
{
$items[] = str_getcsv($line);
}
array_shift($items);
array_pop($items);
// foreach ($items as $item)
// {
// $i = Item::where('upc', $item[7])->first();
// if($i == null)
// {
// $name = str_slug($item[8], '-');
// // $inventory = Item::create(
// // ['upc' => $item[7],
// // 'name' => $item[8],
// // 'price' => $item[9],
// // 'stock' => $item[10],
// // 'cost' => $item[11],
// // 'category' => $item[2],
// // 'slug' => $name
// // ]
// // );
// }
// }
}
above is my code. I want to remove all the items in array that has category thats inside $ignore array. before i store it into database.
Replace this:
foreach ($lines as $line)
{
$items[] = str_getcsv($line);
}
with:
foreach ($lines as $line) {
$item = str_getcsv($line);
if (count($item) > 1 && !in_array($item[2], $ignore)) {
$items[] = $item;
}
}
Now $items will only have the items you are looking for.
NB: Review whether the calls of array_shift and array_pop are still needed, depending on what the category was of those two items.
Related
I have this partially working. I need to grab the data of each player, and present a variable for each "cricket" and "x01" games. I am able to grab the data from the top table, however the 2nd one is not showing any data in my code. I am probably missing something simple, but I can't figure it out.
I want the output to show like this. The part under the line break is what I am missing.
"Howard Hill": {
"name": "Howard Hill",
"team": "Team 2",
"ppd_01": "34.54",
"games_01": "153",
"wins_01": "999",
"assists_01": "69",
"sspre_01": "7.876",
"mpr_crk": "9.99",
"games_crk": "999",
"wins_crk": "999",
"assists_crk": "99",
"sspre_crk": "9.999"
}
Here is my code
<?php
ini_set('default_socket_timeout', 180); // 900 Seconds = 15 Minutes
libxml_use_internal_errors(true);
$doc = new DOMDocument();
$doc->loadHTML(file_get_contents('http://freerdarts.com/past_stats/tues-2018-player-standings.html'));
$doc->strictErrorChecking = false;
$pre = [];
foreach ($doc->getElementsByTagName('table') as $table) {
foreach ($table->getElementsByTagName('tr') as $i => $tr) {
$y = 0;
foreach ($tr->childNodes as $td) {
$text = trim($td->nodeValue);
if ($y > 7) {
unset($pre[$i]);
continue;
}
if (empty($text)) {
continue;
}
$pre[$i][] = $text;
$y++;
}
}
}
// normalise
$pstats = [];
foreach ($pre as $row) {
$pstats[$row[0]] = [
'name' => $row[0],
'team' => $row[1],
'ppd_01' => $row[2],
'games_01' => $row[3],
'wins_01' => $row[4],
'sspre_01' => $row[5],
];
}
echo '<pre>'.json_encode($pstats, JSON_PRETTY_PRINT).'</pre>';
//echo $pstats['Scott Sandberg']['01'];
?>
One problem you're facing is that you're not getting the proper table that needs parsing.
Take note there are multiple tables inside that page.
You need to point out inside the loop that you're skipping other tables in the HTML page and only choose to process the score report table, nothing else:
if (strpos($table->getAttribute('class'), 'report') === false) {
continue;
}
So after getting other tables out of the way, you can start processing the data inside the specific table results that you want to store.
Another thing to point out is you need to skip the headers inside the table. You don't need to anyways.
if ($tr->parentNode->nodeName === 'thead') continue; // skip headers
After that, its just a matter of looping on each <td>.
One gotcha on the tables is that one table has six 6 columns. Another one has 7 so first gather all <td> values. After gathering just unset it from the gathered data so that you have a uniform column layout structure. (I assume you're trying to skip out assists)
Here's the full code:
$pre = []; // initialize container
$keys = ['name', 'team', 'ppd', 'games', 'wins', 'sspre']; // keys needed to be used in the json
foreach ($doc->getElementsByTagName('table') as $table) { // loop all found tables
if (strpos($table->getAttribute('class'), 'report') === false) {
continue; // if its not the report table, skip
}
foreach ($table->getElementsByTagName('tr') as $i => $tr) { // loop each row of report table
if ($tr->parentNode->nodeName === 'thead') continue; // skip headers
$row_values = []; // initialize container for each row
foreach ($tr->childNodes as $td) { // loop each cell
$text = trim($td->nodeValue); //
if ($text === '') continue;
$row_values[] = $text;
}
// unset assist if this table has 7 columns
if (count($row_values) === 7) unset($row_values[5]);
$row_values = array_combine($keys, $row_values); // combine the keys and values
$pre[$row_values['name']] = $row_values; // push them inside
}
}
// finally encode in the end
echo json_encode($pre);
Here's the sample output
I have modified #Ghost code. Try below code.
<?php
libxml_use_internal_errors(true);
$doc = new DOMDocument();
$doc->loadHTML(file_get_contents('http://freerdarts.com/past_stats/tues-2018-player-standings.html'));
$doc->strictErrorChecking = false;
$pre = [];
$keys = ['name', 'team', 'ppd', 'games', 'wins', 'sspre'];
$keys2 = ['name', 'mpr', 'games', 'wins','assists', 'sspre'];
foreach ($doc->getElementsByTagName('table') as $k => $table) {
if (strpos($table->getAttribute('class'), 'report') === false) {
continue;
}
foreach ($table->getElementsByTagName('tr') as $i => $tr) {
if ($tr->parentNode->nodeName === 'thead') continue; // skip headers
$row_values = [];
foreach ($tr->childNodes as $td) {
$text = trim($td->nodeValue);
if ($text === '') continue;
$row_values[] = $text;
}
if($k == 1 ){
$row_values = array_combine($keys, $row_values);
}elseif($k == 2 ){
unset($row_values[1]);
$row_values = array_combine($keys2, $row_values);
}
$pre[$row_values['name']][] = $row_values;
}
}
$new_arr = [];
foreach($pre as $name => $row){
$new_arr[$name] = [
"name"=> $name,
"team"=> $row[0]['team'],
"ppd_01" => $row[0]['ppd'],
"games_01" => $row[0]['games'],
"wins_01" => $row[0]['wins'],
"sspre_01" => $row[0]['sspre'],
"mpr_crk" => $row[1]['mpr'],
"games_crk" => $row[1]['games'],
"wins_crk" => $row[1]['wins'],
"assists_crk" => $row[1]['assists'],
"sspre_crk" => $row[1]['sspre']
];
}
echo '<pre>'.json_encode($new_arr, JSON_PRETTY_PRINT).'</pre>';
Here is sample output
https://www.tehplayground.com/Du5rId3iRx3NH6UL
It seems to me that you want to combine the x01 table values with the crk table values under the same name. Here is the code that I think you are looking for with an example.
$x01 = [];
$crk = [];
$keys_01 = ['name', 'team', 'ppd_01', 'games_01', 'wins_01', 'sspre_01'];
$keys_crk = ['name', 'team', 'mpr_crk', 'games_crk', 'wins_crk', 'assists_crk', 'sspre_crk'];
$table_num = 1;
foreach ($doc->getElementsByTagName('table') as $table) {
if (strpos($table->getAttribute('class'), 'report') === false) {
continue;
}
foreach ($table->getElementsByTagName('tr') as $i => $tr) {
if ($tr->parentNode->nodeName === 'thead') continue; // skip headers
$row_values = [];
foreach ($tr->childNodes as $td) {
$text = trim($td->nodeValue);
if ($text === '') continue;
$row_values[] = $text;
}
// build x01 array
if ($table_num === 1) {
$row_values = array_combine($keys_01, $row_values);
$x01[$row_values['name']] = $row_values;
// build crk array
} else {
$row_values = array_combine($keys_crk, $row_values);
$crk[$row_values['name']] = $row_values;
}
}
$table_num++;
}
$combined = array_merge_recursive($x01, $crk);
// after arrays are merged, remove duplicate values
foreach ($combined as $name => $value) {
if ($value['name']) {
$combined[$name]['name'] = $name;
}
if ($value['team']) {
$combined[$name]['team'] = $value['team'][0];
}
}
echo json_encode($combined, JSON_PRETTY_PRINT);
i am using csv importer library to load bulk of products into mysql database with codeigniter. csv file has some column names from one table like name, price and some column names from another table like categories. it inserts name and price to first table and categories will be inserted after we got id from first table, so category will go with id of the product. but this is taking too long to process the csv file (1000 entries = 2 minutes). the code i have put is
$csv_file_data = $this->csvimport->get_array($_FILES['csv']['tmp_name']);
foreach ($csv_file_data as $csv_data) {
$required = array(
'name' => $csv_data['name'],
'price' => $csv_data['price']
);
foreach ($required as $key => $value) {
$this->db->set($key, $value);
}
$categories= array();
foreach ($csv_data as $key => $value) {
if(strstr($key, 'categories')) {
$category = explode('|', $value);
for($i = 0; $i < sizeof($category); $i++) {
array_push($categories, $category[$i]);
}
}
}
$this->db->insert('first_table');
$id = $this->db->insert_id();
foreach ($categories as $metatag) {
$this->db->set('productid', $id);
$this->db->set('category', $metatag);
$this->db->insert('second_table');
}
}
but it is taking too long to process only 1000 entries, how to optimize this to process 5000 entries in less than 20-30 seconds ?
I believe this is what you're after ...
$csv_file_data = $this->csvimport->get_array($_FILES['csv']['tmp_name']);
foreach ($csv_file_data as $csv_data) {
$required = array(
'name' => $csv_data['name'],
'price' => $csv_data['price']
);
$this->db->set($required);
$categories = array();
foreach ($csv_data as $key => $value) {
if (strstr($key, 'categories')) {
$category = explode('|', $value);
for ($i = 0; $i < sizeof($category); $i++) {
$categories[] = $category[$i];
}
}
}
$this->db->insert('first_table');
$bulk = array();
$id = $this->db->insert_id();
foreach ($categories as $metatag) {
$bulk[] = array(
'category' => $metatag,
'productid' => $id,
);
}
$this->db->insert_batch('table2', $bulk);
}
Note the use of insert_batch which will dramatically improve your speed and also you can give set() an array.
Can you try a version like this and tell me if this is working and return me the time for 100 rows ?
$bulk = [];
$csv_file_data = $this->csvimport->get_array($_FILES['csv']['tmp_name']);
$required = [];
foreach ($csv_file_data as $csv_data) {
$bulk = [];
$id = $this->db->insert_id();
$required[] = [
'name' => $csv_data['name'],
'price' => $csv_data['price'],
];
foreach ($csv_data as $key => $value) {
if (false !== strpos($key, 'categories')) {
$category = explode('|', $value);
foreach ($category as $metatag) {
$bulk[] = [
'category' => $metatag,
'productid' => $id,
];
}
}
}
}
$this->db->insert_batch('first_table', $required);
$this->db->insert_batch('second_table', $bulk);
I have an array which has different items example shirts, shoes, ties, jackets etc. And each of these items can have multiple designs denoted by their ids.
$variations = array(
'shirts' => array('2'),
'shoes' => array('7', '3'),
'jackets' => array('1', '5')
);
Now we are looking for an efficient way to create different variations of all of these.
## Required result ##
$result = array(
array('2','7','1'),
array('2', '7', '5'),
array('2','3','1'),
array('2','3','5')
);
Any help would be appreciated :)
EDIT: Our current function
function cartesian($input) {
$result = array();
while (list($key, $values) = each($input)) {
// If a sub-array is empty, it doesn't affect the cartesian product
if (empty($values)) {
continue;
}
// Seeding the product array with the values from the first sub-array
if (empty($result)) {
foreach($values as $value) {
$result[] = array($key => $value);
}
}
else {
// Second and subsequent input sub-arrays work like this:
// 1. In each existing array inside $product, add an item with
// key == $key and value == first item in input sub-array
// 2. Then, for each remaining item in current input sub-array,
// add a copy of each existing array inside $product with
// key == $key and value == first item of input sub-array
// Store all items to be added to $product here; adding them
// inside the foreach will result in an infinite loop
$append = array();
foreach($result as &$product) {
// Do step 1 above. array_shift is not the most efficient, but
// it allows us to iterate over the rest of the items with a
// simple foreach, making the code short and easy to read.
$product[$key] = array_shift($values);
// $product is by reference (that's why the key we added above
// will appear in the end result), so make a copy of it here
$copy = $product;
// Do step 2 above.
foreach($values as $item) {
$copy[$key] = $item;
$append[] = $copy;
}
// Undo the side effecst of array_shift
array_unshift($values, $product[$key]);
}
// Out of the foreach, we can add to $results now
$result = array_merge($result, $append);
}
}
return $result;
}
While I agree with comments beneath your question I implemented generator-based solution for fun so I can share it anyway:
$variations = array(
'shirts' => array('2'),
'shoes' => array('7', '3'),
'jackets' => array('1', '5')
);
var_dump(iterator_to_array(cartesian($variations), false));
function cartesian($lists, $product = [])
{
if (empty($product)) {
// first run, reverse array for array_pop and remove empty lists
$lists = array_reverse(array_filter($lists, 'count'));
}
$curr = array_pop($lists);
foreach ($curr as $c) {
if (empty($lists)) {
yield array_merge($product, [$c]);
} else {
yield from cartesian($lists, array_merge($product, [$c]));
}
}
}
This is how I did it
$parameters = array(
'shirts' => array('2'),
'shoes' => array('7', '3'),
'jackets' => array('1', '5')
);
$arPhrases = $parameters[0];
for ($i = 1; $i < count($parameters); $i++) {
$notFullCount = count($arPhrases);
foreach ($arPhrases as $phrase) {
foreach ($parameters[$i] as $newPart) {
$arPhrases[] = $phrase." ".$newPart;
}
}
$arPhrases = array_slice($arPhrases, $notFullCount);
}
I have this general data structure:
$levels = array('country', 'state', 'city', 'location');
I have data that looks like this:
$locations = array(
1 => array('country'=>'USA', 'state'=>'New York', 'city'=>'NYC', 'location'=>'Central Park', 'count'=>123),
2 => array('country'=>'Germany', ... )
);
I want to create hierarchical arrays such as
$hierarchy = array(
'USA' => array(
'New York' => array(
'NYC' => array(
'Central Park' => 123,
),
),
),
'Germany' => array(...),
);
Generally I would just create it like this:
$final = array();
foreach ($locations as $L) {
$final[$L['country']][$L['state']][$L['city']][$L['location']] = $L['count'];
}
However, it turns out that the initial array $levels is dynamic and can change in values and length So I cannot hard-code the levels into that last line, and I do not know how many elements there are. So the $levels array might look like this:
$levels = array('country', 'state');
Or
$levels = array('country', 'state', 'location');
The values will always exist in the data to be processed, but there might be more elements in the processed data than in the levels array. I want the final array to only contain the values that are in the $levels array, no matter what additional values are in the original data.
How can I use the array $levels as a guidance to dynamically create the $final array?
I thought I could just build the string $final[$L['country']][$L['state']][$L['city']][$L['location']] with implode() and then run eval() on it, but is there are a better way?
Here's my implementation. You can try it out here:
$locations = array(
1 => array('country'=>'USA', 'state'=>'New York', 'city'=>'NYC', 'location'=>'Central Park', 'count'=>123),
2 => array('country'=>'Germany', 'state'=>'Blah', 'city'=>'NY', 'location'=>'Testing', 'count'=>54),
);
$hierarchy = array();
$levels = array_reverse(
array('country', 'state', 'city', 'location')
);
$lastLevel = 'count';
foreach ( $locations as $L )
{
$array = $L[$lastLevel];
foreach ( $levels as $level )
{
$array = array($L[$level] => $array);
}
$hierarchy = array_merge_recursive($hierarchy, $array);
}
print_r($hierarchy);
Cool question. A simple approach:
$output = []; //will hold what you want
foreach($locations as $loc){
$str_to_eval='$output';
for($i=0;$i<count($levels);$i++) $str_to_eval .= "[\$loc[\$levels[$i]]]";
$str_to_eval .= "=\$loc['count'];";
eval($str_to_eval); //will build the array for this location
}
Live demo
If your dataset always in fixed structure, you might just loop it
$data[] = [country=>usa, state=>ny, city=>...]
to
foreach ($data as $row) {
$result[][$row[country]][$row[state]][$row[city]] = ...
}
In case your data is dynamic and the levels of nested array is also dynamic, then the following is an idea:
/* convert from [a, b, c, d, ...] to [a][b][...] = ... */
function nested_array($rows, $level = 1) {
$data = array();
$keys = array_slice(array_keys($rows[0]), 0, $level);
foreach ($rows as $r) {
$ref = &$data[$r[$keys[0]]];
foreach ($keys as $j => $k) {
if ($j) {
$ref = &$ref[$r[$k]];
}
unset($r[$k]);
}
$ref = count($r) > 1 ? $r : reset($r);
}
return $data;
}
try this:
<?php
$locations = [
['country'=>'USA', 'state'=>'New York', 'city'=>'NYC', 'location'=>'Central Park', 'street'=>'7th Ave', 'count'=>123],
['country'=>'USA', 'state'=>'Maryland', 'city'=>'Baltimore', 'location'=>'Harbor', 'count'=>24],
['country'=>'USA', 'state'=>'Michigan', 'city'=>'Lansing', 'location'=>'Midtown', 'building'=>'H2B', 'count'=>7],
['country'=>'France', 'state'=>'Sud', 'city'=>'Marseille', 'location'=>'Centre Ville', 'count'=>12],
];
$nk = array();
foreach($locations as $l) {
$jsonstr = json_encode($l);
preg_match_all('/"[a-z]+?":/',$jsonstr,$e);
$narr = array();
foreach($e[0] as $k => $v) {
if($k == 0 ) {
$narr[] = '';
} else {
$narr[] = ":{";
}
}
$narr[count($e[0]) -1] = ":" ;
$narr[] = "";
$e[0][] = ",";
$jsonstr = str_replace($e[0],$narr,$jsonstr).str_repeat("}",count($narr)-3);
$nk [] = $ko =json_decode($jsonstr,TRUE);
}
print_r($nk);
Database have three field:
here Name conatin contry state and city name
id,name,parentid
Pass the contry result to array to below function:
$data['contry']=$this->db->get('contry')->result_array();
$return['result']=$this->ordered_menu( $data['contry'],0);
echo "<pre>";
print_r ($return['result']);
echo "</pre>";
Create Function as below:
function ordered_menu($array,$parent_id = 0)
{
$temp_array = array();
foreach($array as $element)
{
if($element['parent_id']==$parent_id)
{
$element['subs'] = $this->ordered_menu($array,$element['id']);
$temp_array[] = $element;
}
}
return $temp_array;
}
I am developing a search engine with vector space Model. I successfully computed tf-idf with associative array data already define in code. Now I want that data should be come from directory where I have a folders and in each folder there is a number of text files with dummy data. I have tried alot but stuck at 1 point using glob function because I want all .txt files as key and its contents as value in foreach loop of glob function.... Below is my code.
Tf-idf With Associative Array Data
$collection = array(
1 => 'this string is a short string but a good string',
2 => 'this one isn\'t quite like the rest but is here',
3 => 'this is a different short string that\' not as short'
);
$dictionary = array();
$docCount = array();
foreach($collection as $docID => $doc) {
$terms = explode(' ', $doc);
$docCount[$docID] = count($terms);
foreach($terms as $term) {
if(!isset($dictionary[$term])) {
$dictionary[$term] = array('df' => 0, 'postings' => array());
}
if(!isset($dictionary[$term]['postings'][$docID])) {
$dictionary[$term]['df']++;
$dictionary[$term]['postings'][$docID] = array('tf' => 0);
}
$dictionary[$term]['postings'][$docID]['tf']++;
}
}
$temp = ('docCount' => $docCount, 'dictionary' => $dictionary);
As you see in 1st foreach loop is that $DocID is key and $doc is its contents(value) of collection array. But I don't know how to implement exact same thing when files read from directory. See code below..
Tf-idf With .txt Files and its contents read from directory
foreach (glob("C:\\wamp\\www\\Web-info\\documents\\awd_1990_00\\*.txt") as $file) {
$file_handle = fopen($file, "r");
//echo $file;
$dictionary = array();
$docCount = array();
foreach($file as $docID=> $value) {
echo $value;
$terms = explode(' ', $doc);
$docCount[$docID] = count($terms);
foreach($terms as $term) {
if(!isset($dictionary[$term])) {
$dictionary[$term] = array('df' => 0, 'postings' => array());
}
if(!isset($dictionary[$term]['postings'][$docID])) {
$dictionary[$term]['df']++;
$dictionary[$term]['postings'][$docID] = array('tf' => 0);
}
$dictionary[$term]['postings'][$docID]['tf']++;
}
}
}
$temp = array('docCount' => $docCount, 'dictionary' => $dictionary);
This gives me error on 1st foreach loop that invalid arugument supplied for foreach loop. As I mentioned earlier I want .txt files as a key and its contents as a value in 1st foreach loop. But I got this error Can anybody please Tell me how to do this.. Thanks in advance..
If you want to treat the entire file as one value, you can use file_get_contents() to read the file into a string:
$dictionary = array();
$docCount = array();
foreach (glob("C:\\wamp\\www\\Web-info\\documents\\awd_1990_00\\*.txt") as $docID) {
$value = file_get_contents($docID);
...
}