so basically my objective here is pretty simple. I have an big CSV inventory file with about 400k lines/items and I am receiving a csvstock file every couple minutes. I am trying to find an efficent and fast way to transfer the new stock count from the gathered stock-feed and update the stock on my inventory file. I created an array from the stock-feed, looping trough my inventory csv file and if the sku from the stock-feed matches with a sku of my inventory file, I am replacing the stock count from the stock-feed to my inventory file.
But the efficency here is really bad, takes years to update those 400k lines.
Any idea for a good efficent way to update my inventory csv with the data from my stock-feed?
Maybe PHP is not the right to work here with, if so, any other tips on how to handle it fast and efficient ?
That is my stinky code so far:
<?php
$rows = array_map('str_getcsv', file('stock-feed.csv') , [","]);
$header = array_shift($rows);
$csv = array();
foreach ($rows as $row)
{
$csv[] = array_combine($header, $row);
}
$row = 1;
$filedone = fopen("updatedStockInventory.csv", "w");
$fileName = "Inventory.csv";
$file = fopen($fileName, "r");
while (($column = fgetcsv($file, 10000, "|")) !== false)
{
if ($row == 0)
{
}
else
{
$key = array_search($column[2], array_column($csv, 'sku'));
$stock = $csv[$key]['stock'];
if ($key != "")
{
//Write line with new stock count
}
else
{
//skip
}
fputs($filedone, $line);
}
$row++;
}
?>
Related
I'm looking to read a CSV export file with PHP. I have access to the File Path Variable called $file_path.
How would I sort the CSV export file by a specific column and then sort it again by a second column? and then save the CSV file to the same file name and file path.
UPDATE:
I got it to read the CSV, then sort it and also save it to the CSV. However, it's also sorting the headers. I am trying to use array_shift and array_unshift but when I use array_shift with a multi-layer array, I am getting an error. (unshift works fine though).
function wp_all_export_after_export($export_id, $exportObj)
{
// Check whether "Secure Mode" is enabled in All Export -> Settings
$is_secure_export = PMXE_Plugin::getInstance()->getOption('secure');
if (!$is_secure_export) {
$filepath = get_attached_file($exportObj->attch_id);
} else {
$filepath = wp_all_export_get_absolute_path($exportObj->options['filepath']);
}
$handle = fopen($filepath, 'r') or die('cannot read file');
$binNumber = array();
$category = array();
$rows = array();
$header = array();
//build array of binNumbers, Categories, and array of rows
while (false != ( $row = fgetcsv($handle, 0, ',') )) {
$binNumber[] = $row[3];
$category[] = $row[1];
$rows[] = $row;
}
fclose($handle);
$header = $rows[0];
array_shift($rows);
//sort array of rows by BinNumber & then Category using our arrays
array_multisort($binNumber,SORT_ASC, $category, SORT_ASC, $rows);
array_unshift($rows,$header);
$file = fopen($filepath,"w");
foreach ($rows as $line) {
fputcsv($file, $line);
}
fclose($file);
}
add_action('pmxe_after_export', 'wp_all_export_after_export', 10, 2);
In a script I'm having I'm pulling a csv from a remote server using ftp. I save this file locally and then open the file. I loop through all the contents of the file matching a certain value against it. If it matches, the script can continue.
Enough talking. Lets show some code...
$filename = 'ftp://.....';
$localCsv = '/tmp/'.date('Ymd').'.csv';
if (!file_exists($localCsv)) {
$content = file_get_contents($filename);
file_put_contents($localCsv, $content);
}
Now that we have the file created. We can continue to loop.
$handle = fopen($localCsv, "r");
while(!feof($handle)) {
$rows[] = fgets($handle);
}
fclose($handle);
$results = array();
foreach ($rows as $rid => $row) {
$columns = explode("\t", $row);
$results[$columns[2]] = $columns;
}
if (array_key_exists($searchValue, $results)) {
... Continue script ...
}
There is just one tiny little problem with this method. It's so slow it's almost going backwards.
Heres all baked together, maybe thats faster?
$handle = fopen($localCsv, "r");
$results = array();
while(!feof($handle)) {
$columns = explode("\t", fgets($handle));
$results[$columns[2]] = $columns;
if ($columns[2] == $searchValue) {
//SEARCH HIT
}
}
fclose($handle);
If thats not working you could try the csv-specific methods that are in PHP
I am writing a script to create Magento attributes programatically, pulling the data from a CSV. Not sure I have the actual loop correct that pulls the data from the CSV - was hoping for some expert guidance on the logic?
<?php
$fh = fopen("attributes.csv", "r");
$i = 0;
while (($l = fgetcsv($fh, 1024, ",")) !== FALSE) {
$i++;
if($i == 1) continue; //ignoring the headers, so skip row 0
$data['label'] = trim($l[2]);
$data['input'] = trim($l[3]);
$data['type'] = trim($l[2]);
//Create the attribute
$data=array(
'type'=>$data['type'],
'input'=>'text',
'label'=>$data['label'],
'global'=>Mage_Catalog_Model_Resource_Eav_Attribute::SCOPE_GLOBAL,
'is_required'=>'0',
'is_comparable'=>'0',
'is_searchable'=>'0',
'is_unique'=>'1',
'is_configurable'=>'1',
'use_defined'=>'1'
);
$model->addAttribute('catalog_product','test_attribute',$data);
}
?>
I basically just want it to grab the attribute data from the CSV, and for each row in the CSV run the code to create it (using the label and name as specified in the CSV - im guessing I am missing something obvious in the loop? (just really learning what I'm doing!)
You reset the $data array in each loop, after inserting the values from CSV, so the CSV-content gets lost. Try this
$fh = fopen("attributes.csv", "r");
$i = 0;
$attributes=array(); //!!
while (($l = fgetcsv($fh, 1024, ",")) !== FALSE) {
$i++;
if($i == 1) continue; //ignoring the headers, so skip row 0
$data=array();
$data['label'] = trim($l[2]);
$data['input'] = trim($l[3]);
$data['type'] = trim($l[2]);
$data['global']=Mage_Catalog_Model_Resource_Eav_Attribute::SCOPE_GLOBAL;
$data['is_required']='0';
$data['is_comparable']='0';
$data['is_searchable']='0';
$data['is_unique']='1';
$data['is_configurable']='1';
$data['use_defined']='1';
//insert $data to the attributes array
$attributes[]=$data;
//or
$model->addAttribute('catalog_product','test_attribute',$data);
}
I am new to PHP and would like to be able to read in a csv file which has two columns, one is the number (kind of like a ID) then the other holds a integer value. I have looked up the fgetcsv function but I have not been able to find a way to read a specific column from the csv file.
I would like to get all the values from the second column only, without the heading.
Any way of doing this?
This is what I have so far:
$file = fopen('data.csv', 'r');
$line = fgetcsv($file);
And this is some sample data from the csv file:
ID,Value
1,243.00
2,243.00
3,243.00
4,243.00
5,123.11
6,243.00
7,180.00
8,55.00
9,243.00
Any help would be appreciated.
Thanks.
fgetcsv() only reads a single line of the file at a time. You'll have to read the file in a loop to get it all:
$data = array();
while($row = fgetcsv($file)) {
$data[] = $row;
}
The heading you can skip by doing an fgetcsv once outside the loop, to read/trash the header values. And if you only want the second column, you can do:
$data[] = $row[1];
However, since you've got data in there, maybe it might be useful to keep it, plus key your new array with the ID values in the csv, so you could also have:
$data[$row[0]] = $row[1];
and then your nice shiny new array will pretty much exactly match what's in the csv, but as an array keyed by the ID field.
$csv = array_map("str_getcsv", file("data.csv", "r"));
$header = array_shift($csv);
// Seperate the header from data
$col = array_search("Value", $header);
foreach ($csv as $row) {
$array[] = $row[$col];
}
// Iterate through data set, creating array from Value column
$header = fgetcsv($h);
$rows = array();
while ($row = fgetcsv($h)) {
$rows []= array_combine($header, $row);
}
$fp = fopen($filePath, "r+");
$header = fgetcsv($fp);
while ($members = fgetcsv($fp)) {
$i = 0;
foreach ($members as $mem) {
$membersArray[ $i ][ ] = $mem;
$i++;
}
}
$newArray = array_combine($header, array_map("array_filter",$membersArray));
You can also use this class http://code.google.com/p/php-csv-parser/
<?php
require_once 'File/CSV/DataSource.php';
$csv = new File_CSV_DataSource;
$csv->load('data.csv');
var_export($csv->connect());
?>
How can I get the contents of a CSV file into a MySQL database row by row? Ive tried a few methods but can never get more than one row returned, using fgetcsv. One method I've tried that seemed to come so close to working:
$fileName = $_FILES['SpecialFile']['name'];
$tmpName = $_FILES['SpecialFile']['tmp_name'];
$fileSize = $_FILES['SpecialFile']['size'];
if(!$fileSize)
{
echo "File is empty.\n";
exit;
}
$fileType = $_FILES['SpecialFile']['type'];
$file = fopen($tmpName, 'r');
if(!$file)
{
echo "Error opening data file.\n";
exit;
}
while(!feof($file))
{
$data = str_replace('"','/"',fgetcsv($file, filesize($tmpName), ","));
$linemysql = implode("','",$data);
$query = "INSERT INTO $databasetable VALUES ('$linemysql')";
return mysql_query($query);
}
fclose($file);
only enters one row, but if I print_r $data it returns all the rows. How do I get it to insert all th rows?
Another method:
$data = str_getcsv($csvcontent,"\r\n","'","");
foreach($data as &$Row)
{
$linearray = str_getcsv($Row,',',''); //parse the items in rows
$linemysql = implode("','",$linearray);
echo $query = "INSERT INTO $databasetable VALUES ('$linemysql')";
}
This almost works too, but there is text within the csv that also contains new lines, so I dont know howto split the actual rows and not the new lines in the text as well.??
this function return an array from csv file
function CSVImport($file) {
$handle = fopen($file, 'r');
if (!$handle)
die('Cannot open file.');
$rows = array();
//Read the file as csv
while (($data = fgetcsv($handle, 1000, ";")) !== FALSE) {
$rows[] = $data
}
fclose($handle);
return $rows;
}
// this will return an array
// make some logic to read the array and save it
$csvArray = CSVImport($tmpName);
if (count($csvArray)) {
foreach ($csvArray as $key => $value) {
// $value is a row of adata
}
}
I think this is what you are looking for.
function getCSVcontent($filePath) {
$csv_content = fopen($filePath, 'r');
while (!feof($csv_content)) {
$rows[] = fgetcsv($csv_content,1000,";");
}
fclose($csv_content);
return $rows;
}
Make sure that you new line separator is ";" or give the correct one to fgetcsv(). Regards.