I'm having a problem with magento partial reindexing in the EE. When the indexers are set to on save (in system->configuration->(Advanced)Index management) the url_rewrites don't update.
If I save a product in the backend, then look in the table enterprise_catalog_product_rewrite there is no entry for this product and I guess because of this there is no entry in enterprise_url_rewrite which means it doesn't work.
I can see the product in the listing but the url is not SEO friendly and if the url key is entered in the browser it will not display the product.
I've been searching information on how this partial indexing works but there seems to be nothing except how better it is.
I've tried truncating url rewrite related tables manually (like this) but it just messed everything up, so I reverted the database.
I had the same problem where catalog_product_entity_url_key would contain the correct data for each store, and enterprise_url_rewrite would contain the correct request_path > target_path pairs. But no matches would happen because enterprise_catalog_product_rewrite was missing an entry for the given product, or had an entry which was pointing to a url_rewrite_id which didn't exist anymore in enterprise_url_rewrite.
My solution was to rebuild the enterprise_catalog_product_rewrite table with this query:
REPLACE INTO enterprise_catalog_product_rewrite
(`product_id`, `store_id`, `url_rewrite_id`)
(SELECT cpeuk.entity_id as product_id, cpeuk.store_id, eur.url_rewrite_id
FROM catalog_product_entity_url_key cpeuk
INNER JOIN enterprise_url_rewrite eur
ON cpeuk.value = eur.identifier)
The new UNIQUE index of Magento CE 1.8 changes the URL keys in the URL rewrite table. If you have multiple store views and import your products, (so that for each store view the URL key is saved) you automatically have duplicated keys. The update script is smart enough to not throw an error, but to rename all the keys so you’ll end up with something like:
- http://de.example.com/someproduct.html
- http://en.example.com/someproduct1.html
- http://nl.example.com/someproduct2.html
To avoid that, the URL rewrites have to be fixed before the update so that there is one rewrite per product and URL key, and the store views use “Use default”. You can manage that with a single SQL query before the update:
DELETE nondefault
FROM catalog_product_entity_varchar AS nondefault
INNER JOIN catalog_product_entity_varchar AS def ON def.value = nondefault.value
AND def.entity_id = nondefault.entity_id
WHERE def.attribute_id =86
AND nondefault.attribute_id =86
AND nondefault.store_id <>0
AND def.store_id =0
Note that 86 is the ID of the URL key attribute. If you already have updated your system without running this query first, you have to remove the wrongly created URL keys first. This can be done with the following query:
DELETE url_table
FROM catalog_product_entity_url_key url_table
INNER JOIN catalog_product_entity_varchar old_url_table ON url_table.store_id = old_url_table.store_id
AND url_table.store_id <>0
AND old_url_table.store_id <>0
AND url_table.attribute_id = old_url_table.attribute_id
AND url_table.entity_id = old_url_table.entity_id;
I hope this helps! The following link might help you if you have further questions:
http://www.code4business.de/update-magento-enterprise-edition-1-13-0-2/
Related
I am working on a project where I am trying to check some validations but I find a risk of 'product_id' parameter is not properly sanitized upon submission to the index.php script, which can disclose the software's installation path resulting in a loss of confidentiality.
so after checking the format of product_id of products, its only in numeric but is there any possibility that the format may be alphanumeric and if yes then what exactly will be the behavior.
Don't change it's data type because it's relation exists in 5-6 tables, SO better create a new field for it's alphanumeric Id and control it using php..
The product id is the foriegn key for 5-6 tables eg) product option contains the product id as referance so please dont
I need to mass update special price from a CSV File (which tells me the sku, the special price, to/from date).
At the moment I do it using magento models method like this (after parsing the CSV rows) in a foreach loop:
$p = Mage::getModel('catalog/product');
$product_id = $p->getIdBySku($product['SKU']);
if (!$product_id) {
throw new Exception('Product Does Not Exists');
}
$p->setStoreId($store_id)->load($product_id);
$p->setSpecialPrice(floatval($product['Price']));
$p->setSpecialFromDate($product['FromDate']);
$p->setSpecialFromDateIsFormated(true);
$p->setSpecialToDate($product['ToDate']);
$p->setSpecialToDateIsFormated(true);
$p->save();
$p = null;
This is okay when a handful of product needs updating. However when you update 100+ products, this becomes incredibly slow and it affects the site performance also.
Is there anyway I can mass import special price and also set the dates via a direct SQL Query?
Whilst researching this issue, I have found a possible solution (based on this article http://fishpig.co.uk/magento/tutorials/update-prices-sql) with setting the special_price in the database directly like this:
Note: bd_ is my table prefix
UPDATE bd_catalog_product_entity AS CPE
INNER JOIN bd_catalog_product_entity_decimal AS CPED ON CPED.entity_id = CPE.entity_id
SET CPED.`value` = 2.99 -- New Special Price
WHERE
CPED.attribute_id = (
SELECT
attribute_id
FROM
bd_eav_attribute eav
WHERE
eav.entity_type_id = 4
AND eav.attribute_code = 'special_price'
)
AND CPED.store_id = 1 -- Retail Store
AND CPE.sku = 'ABS049' -- Sku Being Updated
This appears to be working (i can see the value in magento UI updated). I am not yet sure if this is the correct way to go about setting the special price in the database directly.
Also, I've not yet worked out how to set the from/to dates.
Any help on this will be much appreciated.
I recommend you to use Dataflow Profiles. (I guess you already know its usage, I am writing down the procedure in case it can be helpful to someone who doesn't know the usage of Dataflow Profiles)
Go to System-> Import/Export -> Dataflow Profiles
Run Export All Products Profile.
From the csv exported to your var/export directory of your magento, extract the products whose special price you wish to insert/update.
Update values of special_from_date, special_to_date and special_price fields in a new csv you wish to import.
Import the new csv you just created by running Import All Products Profile.
This is a much reliable way. Hope it helps!!
I'd recommend usage of INSERT with ON DUPLICATE KEY whilst the value for the special_price attribute could be not yet present. Didnt test your SQL but it looks ok - key things to handle is
to retrieve the correct attribute_id from eav_attribute (not releted
tip: don't do it in a subquery cause the performance will suffer
from it)
find the entity_id of the product by SKU from catalog_product_entity
another thing to keep in mind is when you use configurables you probably should set the price for configurable too, and all the simples (but this depends on your business approach)
Both the options will work but you have to do re-indexing of data in 2nd option. I am using the same sql for one of my project to set sale price. I am using it in cron file.
Direct query will save your time, if you have large amount of products.
Mage::getModel('index/process')->load(2)->reindexAll();
OR
php indexer.php --reindex catalog_product_price
I have a table named orders_products which hold all the products associated with each individual orders. Now, if the customer decides to edit the quantity, the attributes, or simply just remove a product, what would be the easiest way to handle this change in the database?
orders_products
id | quantity | fk_products_id | attributes | price | fk_orders_nr
fk_products_id refers to the actual product id in our products table
attributes is a comma sepparated string with id's refering to our attributes table (attributes can be the angle, lenght, diameter etc. for each product. As in 45 degres, left angeled, 20 cm in length, 43mm in diameter).
fk_orders_nr is the actual order the product belongs to.
Since I have a class that handles this part on the client side. Would it be easier to just DELETE all associated products, based on the order id (fk_orders_nr), and just re-insert and updated the whole set based on what's stored in the class?
Or is there a neat way to handle this directly in MySQL?
I've looked a bit into on duplicate key update, and ignore, but they doesn't seem to fit my needs..
I need to check to see if the actual product id exists, then decide to insert new, or update existing (it could be both the quantity, and/or optional attributes), also the product might be removed if not in the list from the class anymore.
When writing this question. I think deleting the whole set, and reinsert it, might be the easiest way.
This database looks badly designed. Firstly I assume by fk_products_id you mean product_id. You do not need to specify that a column is a foreign key in its name.
Secondly, I would advise you to keep all columns atomic, as in, no multi-values. The attributes column keeping a comma-separated list will give you headaches in the future and it also breaks the FIRST normal form (the most basic one).
Thirdly, you don't need (although it could sometimes be useful) an id as a primary key in your junction table. You can just use a compound primary key from your fk_products_id and fk_orders_nr keys.
When writing this question. I think deleting the whole set, and
reinsert it, might be the easiest way.
Yes, that is the way it's usually done.
But I insist you ignore everything about the current problem you're having and redesign your database from scratch, putting special attention into normalization. These are basic database design standards and they exist for a reason. Hopefully you won't learn about the reason when it's too late.
I have about 800,000 products in my magento database, and I need to delete about half of those products (approx 400,000).
Running on magento 1.7.0.2
It's just taking me forever to do it from the "Manage Products" page. The process takes to long and the server keeps timing out.
Is there a SQL statement that can deleted selected products from within the database?
All the products that need to be deleted, the title starts with *NLA
I know magneto stores the product data in several different tables, so I'm trying to figure out how to delete all the data associated with the selected products from all tables.
I know MySQL statements, but I'm not an expert, and can't figure this one out.
For example if it was only one table I would do something like this:
DELETE FROM product_table WHERE title_value LIKE '*NLA%'
I would appreciate it if someone could point me in the right direction.
You can use this query
DELETE FROM product_table WHERE title_value LIKE '*NLA%
Magento used InnoDB engine Storage for MySQL with foreign keys. All data which has FK keys will be also deleted.
Your SQL statement should do the job IF you're deleting from a single table.
If stored on different tables, it depends on the way they are stored. If, for example, products are stored according to their key being a foreign key in the different tables, then all related tables will update when the products are deleted from the main table (depending on whether the foreign keys are set to CASCADE ON DELETE or not).
But be careful about the regex given in LIKE condition, make sure there are no other products that have 'NLA' somewhere in their title.
Edit: you said starting with 'NAL'? In that case your regex should be something like 'NAL*%'
DELETE FROM product_table WHERE title LIKE 'NAL*%'
In Magento product details are stored in data base using EAV concept so there are more then 30 tables where a product data will go.
I suggest you to use Magento import export facility for bulk delete.
refer this link :
magento: bulk delete products via import/export facility?
There is an extension Magmi which can be used as well.
On my website several vendors can import their articles.
I want to give them the option to cleanup their articles (e.g. remove articles not in importfile).
The first thing I thought of is to just first delete all the articles of the vendor before import, however this might bring up a situation that a customer is visiting the site and don't see any products (of the specific vendor).
So after some more thinking I've came up with two other solutions:
import the products and remember what products were imported / updated (if they already existed) and delete the products of the vendor which weren't in the import afterwards
import the products in another temp table, remove the current products of the vendor and then copy the products in the temp table to the 'real' table.
However there might be some issues with both options.
is let's say remembering all imported / updated products really a good idea since sometimes it can be > 1 million products?
will it be possible to delete the current products and copy the products from the temp table to the real table be so fast that there is only a tiny chance that a visiting customer won't see any products?
Perhaps there are any more issues with the options I figured out.
Or perhaps there are some other options to do this?
PS
'Locking' the site / locking out customers while importing is not an option.
When I read your questions, two solutions came to my mind:
Fire up a "maintenance mode" while importing, but thats probably not what you want. Sorry didn't read your last statement.
Import (or delete) the items one by one, because that way the user would be missing at most one product at any given instant. There are a couple of caveats to consider here, though:
What happens if a product is deleted while it is inside the users shopping basket?
The references in already made purchases need to stay intact, in order to be able to reconstruct a bill after the product bought has been deleted.
Also, you can rename a table in PostreSQL like this:
ALTER TABLE TableB RENAME TO TableC;
In case you want to go with your "temporary table" solution, which I too have used, though under much less critical circumstances, admittedly. But note that, at least in MySQL InnoDB, you have to worry about foreign keys, which will still point to the old table if renamed, e.g. if there is a foreign key from TableA to TableB, and you rename TableB to TableB_old and TableB_new to TableB, than the foreign key will point to TableB_old. I don't know if this is the case with PostgreSQL though.
I have a variation on on of your suggested methods:
Have a table called vendortables setup that has columns for vendorname or id and the name of the table that lists their products.
When a vendor wants to import, create a new table for the input (possibly vendorname + upload start time). Upload all of the data into it. Once that's done, grab the name of the current vendor's table (in vendortables) and update so the name of the new table is associated with the uploading vendor. Then delete the old table (that you grabbed before the update).
This way, no table data is every copied.
Example:
SELECT * FROM vendortables
+--+-----+
|id|table|
+--+-----+
|01|test |
+--+-----+
SELECT * FROM test
+--+----+
|id|name|
+--+----+
|01|car |
+--+----+
//Import & upload new data
CREATE TABLE test1 USING('?', 'new car')
SELECT * FROM test1
+--+--------+
|id|name |
+--+--------+
|01|new car |
+--+--------+
//Phase in new data
UPDATE vendortables SET table='test1' WHERE id='1'
//Delete old table
DROP TABLE test
And just have PHP ask vendortables for the name of the table responsible for displaying a vendor's products.
Also, since you mention your site's high usage, have you considered that caching will probably make requesting during an import very unlikely.
make a delete table
delete from del_vendor_article where vendorid='vendorid'
do this on article update
insert into del_vendor_article SET vendorid='vendorid', article_id='articleid'
possibly show them what will be deleted. then they can remove anything they might want to keep
SELECT item FROM articles WHERE NOT EXISTS (SELECT * FROM del_vendor_article WHERE articles.articles_id = del_vendor_article.articles_id);
delete not imported items
delete from articles WHERE NOT EXISTS (SELECT * FROM del_vendor_article WHERE articles.articles_id = del_vendor_article.articles_id);