I'm using WordPress + Woocommerce for one project. The problem is that the CSV file from the client has over 650k records for products and I have to upload all these products to woocommerce store.
Wordpress using 2 tables (wp_posts, wp_postmeta) for products and 3 tables(wp_terms, wp_term_taxonomy, wp_term_relationships) for categories and tags. All products should be uploaded as just simple woocommerce products.
Not sure what could be the best option for such data migration.
I tried to use a WordPress import plugin for this (WP All Import) but it was taking too much time.
By using this script (largeCSV2mySQL) I was able to import 650k products info to the database table in less than 10s. Is there anyway to import all those 650k products to woocommerce store in such a fast way?
What could be the best experience to handle such huge data?
I highly recommend using the Woocommerce REST API to perform the population of the data. You'll definitely need to write code to process your file by utilizing the CRUD actions available to you. The REST API handles most of the heavy lifting, all you need to do is read the data from your file and fit it to the API.
If you do have a large number of products to import, and it's all in a single file, try composing your importer to only process a range of lines (can do with with command line parameters). That would allow you to run the import in multiple terminals and split the load, rather than having to write something that's multithreaded using pthreads.
I recommend creating a function that retrieves the data from the file and sends it to WordPress/Woocommerce with wp_insert_post and the post_type 'product'.
Related
I am making a plugin to custom import a 120mb XML file with about 60k products to WooCommerce.
How It works
I have an Admin Page with some settings and fields etc.
-> Submit posts to self - $_POST[...] is detected as set and it attaches a class and starts the function.
XMLReader - gets the file and starts reading.
When 'Product' Element is found -> I'm making a DOMDocument and using simplexml_import_dom.
Checking for ids, names etc...
Mapping the seller ID with existing Category (XML FIle) -> not all categories are imported
Checking if the product exists: wc_get_product_id_by_sku
Getting the category ID by slugname: get_term_by('slug', $categorySlug, 'product_cat');
Adding the product with another function that uses the ready productArray: WC_Product()
uploading images with media_sideload_image also to get their ID's for WC_Product
Saving product $newProductObj->save();
XMLReader will skip a product if it exists or if the category is Excluded in maping XML file.
The XMLReader and script works fast.
It is getting slower due to database connections wc_get_product_id_by_sku and really by media_sideload_image and $newProductObj->save(); WC_Product.
Biggest slow down is with images upload.
Tried:
I have set php.ini to get more memory etc.
I have used set_time_limit(0); in a while loop
I am unsetting unused vars.
Code without DB calls is fast reading, making objects, checking categories...
Problems :)
After a while server goes 503 or 500 CGI PHP something
Question:
How can I speed it up, make it more robust, so it will run... even for half a day, but run, without making the server stuck.
I hope you gus can help me out with it.
I am using shopware as e-commerce engine, I creating articles from back-end for every articles, but it is a lazy process and is taking too much time. My question is, Is there any way that I can import articles into shopware directly(to the database), because I have thousands number of articles. and It will take so much time if I Insert it from backend for each article.
Shopware provides a free plugin Shopware Import/Export for this. You can use it to import and export product and customer data, categories and product images using CSV and XML files. Here you will find the documentation of the plugin.
I need to import more than 100 products from an CSV file but some of the products already exists in the webshop. I will be using the core importer tool that comes with Woocommerce.
When importing the file I need to update each already existing product by matching the Name of the product. (I know there is an option for matching the SKU or ID but I need to update the product if the name already exists)
Thanks in advance!
This is a pretty common scenario. Of course, the core wp imported plugin will work with this. But it will create new entries regardless of which once you already have. You might have imported content before and noticed that there are duplicate posts, this is something I think you can't avoid with the default importer.
Now to help you with this, there are other solutions. WP import all is one of my favourites. The reason, it lets you map the entries from your CSV to WP. You decide entries from which column go to which field.
Here is a free version of the plugin: https://wordpress.org/plugins/woocommerce-xml-csv-product-import/
This does not work with WooCommerce though, you will need a premium plugin for this: http://www.wpallimport.com/woocommerce-product-import/
Another product I can think of is the one built by the WooCommerce folks itself:
https://docs.woocommerce.com/documentation/plugins/woocommerce/woocommerce-extensions/product-csv-import-suite/
I have not used this myself, but have heard good words about it.
PS: If the data from the previous posts is also there on the CSV, you can delete the previous posts and import them from the CSV, that way you won't have to worry about the duplicated, or the check the names. You can try this on a local install. and then export them with all the updated data, and import it again to the production site.
I hope you find this helpful.
I am currently working on a project where I am looking at converting ExpressionEngine channel entries into individual .txt or .csv files. I can export all channel entries into one file, but I am struggling when it comes to separating each entry into individual files...
I am integrating the website with a clients stock inventory system for their retail stores. The software inputs .txt or .csv files into the system when an order is placed in store, this then updates their stock inventory.
The website is going to display their products with live stock inventory based on in store orders and online orders.
I am looking into pulling data from each order via an sql query and exporting this as a .txt file. Each order will create an individual text file and once an hour the .txt files will be exported to a folder within the server via a cron. This is the folder the clients stock inventory system uploads the .txt files from. so if an order is placed via the website, the file will be exported and uploaded to the stock system, reducing the stock inventory for that item buy the amount in the order.
Any suggestions or pointers would be appreciated.
If this is a complicated way of achieving this and anyone has a more simplified solution then please let me know.
Thanks in advance
What module are you using for your e-commerce setup? Chances are it has an extension hook you could use that fires after each order is completed. You could add your code in that hook to export the order as a text file directly into the aforementioned folder on the server.
CartThrob: cartthrob_on_authorize
Store: store-order-complete-end
Brilliant Retail: br_order_create_after
We have a multi-store magento setup, and we have several stores under the main site. We need to both enable and set search visibility for several hundred products, but we don't want those products to just show in every store, in fact, we only want them to show in one store. I have tried updating the Magento database, only to find out that not every store has a set of attributes to update, which would require some fancy SQL to perform. I have also tried creating an Excel XML import, which did not work. The import file looks like this (I tried it 2 ways):
sku visibility status _store
112001 4 1 otherstore
11924-01 4 1 otherstore
sku visibility status _store
112001 4 1 otherstore
112001 1 2 admin
11924-01 4 1 otherstore
11924-01 1 2 admin
Both ways appeared to just reset the status completely, as in nothing in the status field. I am a little wary of doing this via PHP since I am not well versed in Magento and have heard un-great things about mucking about in the core files, but at this point if it will work I'll do anything. I can't help but think that someone has done this before, via plugin on SQL or PHP, so any help here is appreciated.
Backup your database. Do a product export of the products this applies to. Edit the csv file to show the correct store for all of those products. In magento delete all of the products this applies to. Now import the csv that you previously exported and edited.
I found, and sucessfully used, Magmi. Was incredibly fast, and seemed to do the job well. The documentation isn't great, but it's usable.