I had set up the catalog to sort on these values, which I saved through the admin last night. For most of a full day they were there. Now, like ghosts they have disappeared.
I suspect the problem could be tied to values saved through the admin rather than imported.
Can anyone point me to the source of this problem and the solution in the code?
Between the last time I saw the values working on the frontend (a few hours ago) and now, I did several things:
1.) Added 2 new attributes and some test values (unrelated to problem attribute), reindexed.
2.) Tried to import unrelated values for 60,000 products. Import hanged, so
3.) Imported values for 20,000 products at a time, no errors.
4.) reindexed.
Now all values saved manually are gone. Again, how can this happen? If they were saved to the DB, then shouldn't it take an delete call from the code to the DB to do this? How/why would such a call be made when I did not execute any such command in code or through admin? How can I fix this and avaoid in future?
TO AXEL (clarifying):
Thanks for your reply, #Axel.
1.) I created a text attribute called "sort_order" and entered some integer values through the admin.
2.) Then I did a full db backup with mysqldump.
3.) Then, I created two new attributes, "random_order" (price type) and "random_order_1" (text type). The purpose was to experiment with two solutions for shuffling the products in the catalog pages.
4.) Through phpMyAdmin I did a simple query to give me all products in random order:
SELECT `sku`
FROM `catalog_product_entity`
WHERE 1
ORDER BY RAND( )
and exported the result to csv. I simply used excel to number the items from 1-60,000, creating an import csv file with columns: sku, random_order (price type), random_order_1 (text type), with both attributes having same integer values.
5.) I used standard import method (replace existing complex data) in admin, 20k products at a time. After import, values for previously set and seemingly totally unrelated "random_order" have been deleted.
Before reindexing, every item's sort_order is now reset to default (=1), but it still appears in proper order on the frontend (so value still exists in product flat table), while the random_order and random_order_1 attributes have their imported values.
After reindex, all trace of sort_order is wiped out. That would make sense if I was actually importing that attribute, but I'm not. No other attribute appears to have been affected.
I restored db from mysqldump, tried whole process again. Same result.
Related
I use PHP for solving this task. But that's not the point.
Every morning I get four files each of them contains nearly 6000-8000 records having the following form:
Product name
Package
Producer
Price with tax
Expire date
Rest
Series
Parsing these records I get the table of products. Later clients make orders so I need to keep id of item in the order table. (clients would like to see history of purchases)
All is fine. The arising problem is that one day any of suppliers can send completely different price list. I.e. some products will be removed and others will be added. So it would be completely wrong to rely upon they order in the price list.
What I've come to is to parse catalog blindly adding all items one time. Every next time when I get catalog I need to add only new items and remove the old one from DB. (though not actually removing but just marking as deleted so that no new purchases would be possible )
For deciding whether item is new or not I will retrieve record by record from Excel file and check "Product name", "package", "producer", "series" fields in conjunction against the table with products.
If no such item is found I assume that this is a new item and I'll add it to DB.
What to do with deleted items? I'm not warned about deletion of them. So I can't find out what items are missing in the new Excel file. The solution is to scan DB item by item and see whether all items in DB are present in Excel file. If some item is missing then I will mark it as deleted.
Once deleted items can be back for selling. So I will need to select all deleted items and check one by one against Excel file. If item appears in Excel file - I add it back.
It's worth to note that some suppliers give their catalogs as Excel files and others as DBF files as for now. Who knows what formats will come in the future. Also the number of suppliers is supposed to grow up (next month we get 2 more into play).
My question. Is there any better way to do it more efficient? I'm afraid that my method is too straightforward.
Having 8000 records and doing 3 checks I will get O^2 complexity for every price list making whole search through MySQL Db. Perhaps it will work for 8000 records but I'm sure it's going to fail when one day I get price list with let's say 10^5 record.
Is there a better way to organise it?
Thanks.
i have a slight problem here.
i am using stock in the channel with magento, and there plugin uses magento's default tax_class_id.
but the problem we are facing is we have deleted the default magento tax class id.
we have tried to change the plugin to make it use our custom tax_class_id. but as soon we do that plugin dont work.
we have tried to create the same name tax_class but it dont work as it use the new tax_class_id which is now 11.
can we make magento retrieve the deleted tax_class_id?
I don't use Magento, but it sounds like the problem is you deleted a row of data from the tax_class_id table, and when you try to recreate it you get a different value for the auto incrementing primary key tax_class_id, which if I'm correctly reading what you wrote, is confusingly the name of both the table and one of the columns in that table. Not that it matters what they're called, you can go in the tax_class_id table, create a new row, and then double-click on the tax_class_id column to edit the value there to make it match the original value (provided you know what the original value was, of course, but looking in whatever tables reference that value should give you that answer if you don't know). So basically just create it as you've done, it will automatically assign a new counter ID but you can edit that through phpMyAdmin.
Note that, in general, messing with auto incrementing ID values and changing existing data is a good way to mess up your referential integrity, but as long as you have a good backup and understand what you're doing this should work fine for you.
I'm in the process of creating an application where we are fed several external product feeds daily, and we populate our products database with the feeds.
However the trick is we need to keep the product db in sync with the latest feed(s).
Previously I had toyed with the theory of populating the current product list from db in an array, and doing array comparison with the latest feed, that got gunned down once the product count was in the thousands. (Ran out of memory when trying to get a 5000 records).
So after abit of research, it seems the solution would probably lie on the SQL side, using TRIGGERS perhaps. Though I'm not quite sure how to go about it, hence my question.
So the 2 objectives I need to accomplish with the syncing process:
1) Insert new products that do not already exist in our db. We can accomplish this with the INSERT IGNORE method.
2) Find products on our DB that do NOT exist on the latest feed, and do something to them. (flag as deleted, or move to a deleted products table, etc.)
Step 2 is where I'm having trouble. I'm thinking now maybe for every sync operation, we insert the products from the latest feed into a 'Temp-Products' table, and somehow compare 'Products Table' with 'Temp-Products' table in finding the records that need to be flagged as deleted.
Any advice please?
Thanks
Obviously over-thought this one. The solution as suspected and further enforced by Anigel is to create a temporary table, 'products_temp' to store new feeds. We then run a simple join to find out what products are in the Products table, but not in 'products_temp', hence suggesting that the products have been sold out or deleted on the retailer.
We can then either flag the results of the query as deleted/sold out/do whatever.
The query I used is this:
SELECT products.sku_number, products_temp.sku_number FROM products LEFT OUTER JOIN products_temp ON products.`sku_number` = products_temp.`sku_number` WHERE products_temp.sku_number IS null
I recently made a module that calculates the real gross margin for every order and order_item based on shipping cost data I import. I did this by adding 2 columns to the sales_flat_order table and the sales_flat_order_item table. This seemed to work great, until I realized that when I saved the imported data, it also updated the updated_at value. Since this was the first import of all orders, they all now report having been updated today. This is throwing off reports and other shipping software that syncs with it.
This brings me to 2 questions:
Is adding a column to an existing table (in this case, the sales tables) a major NO-NO?
If not, is there a way to set the data that doesn't increase the updated_at value?
If it helps, the code that actually writes the data is inside my IndexController.php file. It loops through the collection of orders and the items within those orders and sets the necessary values using something like $order->setGrossMargin($orderGM)->save();. I imagine it is the call to save() that is doing it, but I'm not sure the right way around this problem.
In the mean time, I'm working on a solution in which I import the data to custom tables and only read from the sales tables when necessary. Either way, it's a good exercise :)
Brian
Instead of calling save(), did you try:
$order->getResource()->saveAttribute('gross_margin')
i have customised oscommerce to pull in a csv file of products, delete anything thats not with an image/proper description/proper title gets removed.
The import runs on a cron job basis pulling information from a supplier, it hasnt run since yesterday but a product has disappeared-
Anyone who has used oscommerce will know that, product information is stored over multiple tables. example is-
products
product_description
and so on. the thing that has got me that the information is deleted from the product table but not from the product_description table.
The product that is being deleted is a manually input one which carries a special tag/prefix on the model item of the product table. Therefore shouldn't be touched at all. Am clueles as what is going on.
Is there mysql integrity checks deleting records? could there be another plugin working on oscommerce?
If you can identify 'magic' rows in your table you could create a BEFORE DELETE trigger on that table to check the row someone is planning to delete isn't one of them, and raising an error if it is. I haven't created a trigger for quite a while so I forget the syntax, but there's more about triggers here... http://dev.mysql.com/doc/refman/5.0/en/create-trigger.html