i have customised oscommerce to pull in a csv file of products, delete anything thats not with an image/proper description/proper title gets removed.
The import runs on a cron job basis pulling information from a supplier, it hasnt run since yesterday but a product has disappeared-
Anyone who has used oscommerce will know that, product information is stored over multiple tables. example is-
products
product_description
and so on. the thing that has got me that the information is deleted from the product table but not from the product_description table.
The product that is being deleted is a manually input one which carries a special tag/prefix on the model item of the product table. Therefore shouldn't be touched at all. Am clueles as what is going on.
Is there mysql integrity checks deleting records? could there be another plugin working on oscommerce?
If you can identify 'magic' rows in your table you could create a BEFORE DELETE trigger on that table to check the row someone is planning to delete isn't one of them, and raising an error if it is. I haven't created a trigger for quite a while so I forget the syntax, but there's more about triggers here... http://dev.mysql.com/doc/refman/5.0/en/create-trigger.html
Related
I'm in the process of creating an application where we are fed several external product feeds daily, and we populate our products database with the feeds.
However the trick is we need to keep the product db in sync with the latest feed(s).
Previously I had toyed with the theory of populating the current product list from db in an array, and doing array comparison with the latest feed, that got gunned down once the product count was in the thousands. (Ran out of memory when trying to get a 5000 records).
So after abit of research, it seems the solution would probably lie on the SQL side, using TRIGGERS perhaps. Though I'm not quite sure how to go about it, hence my question.
So the 2 objectives I need to accomplish with the syncing process:
1) Insert new products that do not already exist in our db. We can accomplish this with the INSERT IGNORE method.
2) Find products on our DB that do NOT exist on the latest feed, and do something to them. (flag as deleted, or move to a deleted products table, etc.)
Step 2 is where I'm having trouble. I'm thinking now maybe for every sync operation, we insert the products from the latest feed into a 'Temp-Products' table, and somehow compare 'Products Table' with 'Temp-Products' table in finding the records that need to be flagged as deleted.
Any advice please?
Thanks
Obviously over-thought this one. The solution as suspected and further enforced by Anigel is to create a temporary table, 'products_temp' to store new feeds. We then run a simple join to find out what products are in the Products table, but not in 'products_temp', hence suggesting that the products have been sold out or deleted on the retailer.
We can then either flag the results of the query as deleted/sold out/do whatever.
The query I used is this:
SELECT products.sku_number, products_temp.sku_number FROM products LEFT OUTER JOIN products_temp ON products.`sku_number` = products_temp.`sku_number` WHERE products_temp.sku_number IS null
Hi I am using PHP to manipulate information in my MySQL database. However I am looking for a way to update a table (all records need to be updated) based on information within another table.
For example I have a list of products lets say 10 each with a unique id stored in a products table. I have a purchases table which has the same product ID and the amount of purchases done for each product. I want to update each product in the products table to reflect the total purchases made for each product and store it in a column called instock which is part of the products table.
How can this be done?
If I understand your situation correctly, you're dealing with a stock-count. When an item is purchased (represented by a entry in the Products table) then the stock count figure should be decreased. This should happen within the same transaction as the new entry to the Products table to keep your data consistent. I would recommend using a Trigger on the table to implement this. You'll find lots of information about implementing triggers in MySQL on this site. A trigger you could use might look something like this:
CREATE TRIGGER update_stock_count
BEFORE INSERT ON Purchases
FOR EACH ROW
BEGIN
UPDATE Products SET stock_count = stock_count - NEW.quantity_ordered
WHERE product_id = NEW.product_id;
END;
This trigger doesn't take into account that there might not be enough stock of a product, nor does it handle updates or deletes on the Purchases table but it could be modified to do so.
I have about 800,000 products in my magento database, and I need to delete about half of those products (approx 400,000).
Running on magento 1.7.0.2
It's just taking me forever to do it from the "Manage Products" page. The process takes to long and the server keeps timing out.
Is there a SQL statement that can deleted selected products from within the database?
All the products that need to be deleted, the title starts with *NLA
I know magneto stores the product data in several different tables, so I'm trying to figure out how to delete all the data associated with the selected products from all tables.
I know MySQL statements, but I'm not an expert, and can't figure this one out.
For example if it was only one table I would do something like this:
DELETE FROM product_table WHERE title_value LIKE '*NLA%'
I would appreciate it if someone could point me in the right direction.
You can use this query
DELETE FROM product_table WHERE title_value LIKE '*NLA%
Magento used InnoDB engine Storage for MySQL with foreign keys. All data which has FK keys will be also deleted.
Your SQL statement should do the job IF you're deleting from a single table.
If stored on different tables, it depends on the way they are stored. If, for example, products are stored according to their key being a foreign key in the different tables, then all related tables will update when the products are deleted from the main table (depending on whether the foreign keys are set to CASCADE ON DELETE or not).
But be careful about the regex given in LIKE condition, make sure there are no other products that have 'NLA' somewhere in their title.
Edit: you said starting with 'NAL'? In that case your regex should be something like 'NAL*%'
DELETE FROM product_table WHERE title LIKE 'NAL*%'
In Magento product details are stored in data base using EAV concept so there are more then 30 tables where a product data will go.
I suggest you to use Magento import export facility for bulk delete.
refer this link :
magento: bulk delete products via import/export facility?
There is an extension Magmi which can be used as well.
I recently made a module that calculates the real gross margin for every order and order_item based on shipping cost data I import. I did this by adding 2 columns to the sales_flat_order table and the sales_flat_order_item table. This seemed to work great, until I realized that when I saved the imported data, it also updated the updated_at value. Since this was the first import of all orders, they all now report having been updated today. This is throwing off reports and other shipping software that syncs with it.
This brings me to 2 questions:
Is adding a column to an existing table (in this case, the sales tables) a major NO-NO?
If not, is there a way to set the data that doesn't increase the updated_at value?
If it helps, the code that actually writes the data is inside my IndexController.php file. It loops through the collection of orders and the items within those orders and sets the necessary values using something like $order->setGrossMargin($orderGM)->save();. I imagine it is the call to save() that is doing it, but I'm not sure the right way around this problem.
In the mean time, I'm working on a solution in which I import the data to custom tables and only read from the sales tables when necessary. Either way, it's a good exercise :)
Brian
Instead of calling save(), did you try:
$order->getResource()->saveAttribute('gross_margin')
On my website several vendors can import their articles.
I want to give them the option to cleanup their articles (e.g. remove articles not in importfile).
The first thing I thought of is to just first delete all the articles of the vendor before import, however this might bring up a situation that a customer is visiting the site and don't see any products (of the specific vendor).
So after some more thinking I've came up with two other solutions:
import the products and remember what products were imported / updated (if they already existed) and delete the products of the vendor which weren't in the import afterwards
import the products in another temp table, remove the current products of the vendor and then copy the products in the temp table to the 'real' table.
However there might be some issues with both options.
is let's say remembering all imported / updated products really a good idea since sometimes it can be > 1 million products?
will it be possible to delete the current products and copy the products from the temp table to the real table be so fast that there is only a tiny chance that a visiting customer won't see any products?
Perhaps there are any more issues with the options I figured out.
Or perhaps there are some other options to do this?
PS
'Locking' the site / locking out customers while importing is not an option.
When I read your questions, two solutions came to my mind:
Fire up a "maintenance mode" while importing, but thats probably not what you want. Sorry didn't read your last statement.
Import (or delete) the items one by one, because that way the user would be missing at most one product at any given instant. There are a couple of caveats to consider here, though:
What happens if a product is deleted while it is inside the users shopping basket?
The references in already made purchases need to stay intact, in order to be able to reconstruct a bill after the product bought has been deleted.
Also, you can rename a table in PostreSQL like this:
ALTER TABLE TableB RENAME TO TableC;
In case you want to go with your "temporary table" solution, which I too have used, though under much less critical circumstances, admittedly. But note that, at least in MySQL InnoDB, you have to worry about foreign keys, which will still point to the old table if renamed, e.g. if there is a foreign key from TableA to TableB, and you rename TableB to TableB_old and TableB_new to TableB, than the foreign key will point to TableB_old. I don't know if this is the case with PostgreSQL though.
I have a variation on on of your suggested methods:
Have a table called vendortables setup that has columns for vendorname or id and the name of the table that lists their products.
When a vendor wants to import, create a new table for the input (possibly vendorname + upload start time). Upload all of the data into it. Once that's done, grab the name of the current vendor's table (in vendortables) and update so the name of the new table is associated with the uploading vendor. Then delete the old table (that you grabbed before the update).
This way, no table data is every copied.
Example:
SELECT * FROM vendortables
+--+-----+
|id|table|
+--+-----+
|01|test |
+--+-----+
SELECT * FROM test
+--+----+
|id|name|
+--+----+
|01|car |
+--+----+
//Import & upload new data
CREATE TABLE test1 USING('?', 'new car')
SELECT * FROM test1
+--+--------+
|id|name |
+--+--------+
|01|new car |
+--+--------+
//Phase in new data
UPDATE vendortables SET table='test1' WHERE id='1'
//Delete old table
DROP TABLE test
And just have PHP ask vendortables for the name of the table responsible for displaying a vendor's products.
Also, since you mention your site's high usage, have you considered that caching will probably make requesting during an import very unlikely.
make a delete table
delete from del_vendor_article where vendorid='vendorid'
do this on article update
insert into del_vendor_article SET vendorid='vendorid', article_id='articleid'
possibly show them what will be deleted. then they can remove anything they might want to keep
SELECT item FROM articles WHERE NOT EXISTS (SELECT * FROM del_vendor_article WHERE articles.articles_id = del_vendor_article.articles_id);
delete not imported items
delete from articles WHERE NOT EXISTS (SELECT * FROM del_vendor_article WHERE articles.articles_id = del_vendor_article.articles_id);