Im having a very bizarre problem with a magento store that I work on. I have been doing the SEO and have about 100 404 errors for product urls that SHOULD be valid.
I can see the products and the urls in the product catalog via the admin - everything is correct there. If I try to load the product by its url key using the product model I get the correct product back (a configurable product with id 12457 - this is important for later).
But if I try to load the page by the url path - it wont load - instead I get a 404 error - even with developer mode - I get no insight into the issue as there isnt really ANYTHING to follow in the system.log or in the Varien profiler / debugger except the normal noRouteAction stuff that comes up.
So I started investigating the core_url_rewrites table to see if I could identify multiple (duplicate) urls. Sure enough - I did. The core_url_rewrite table had over 30000 rows with 2.5x the # of products for the product group so I deleted the rows in the table and reindexed.
This gave me the proper count of products - and I can see that there is a duplicate url issue of some sort:
+----------------+----------+--------------------+-------------------------------------------------------------------------------------------------+---------------------------------------------+-----------+---------+-------------+-------------+------------+
| url_rewrite_id | store_id | id_path | request_path | target_path | is_system | options | description | category_id | product_id |
+----------------+----------+--------------------+-------------------------------------------------------------------------------------------------+---------------------------------------------+-----------+---------+-------------+-------------+------------+
| 200513 | 1 | product/12279 | torano-exodus-1959-50-years-bfc-cigars.html | catalog/product/view/id/12279 | 1 | NULL | NULL | NULL | 12279 |
| 200514 | 1 | product/12280 | torano-exodus-1959-50-years-bfc-single-cigar.html | catalog/product/view/id/12280 | 1 | NULL | NULL | NULL | 12280 |
| 200759 | 1 | product/12408 | torano-exodus-1959-50-years-bfc.html | catalog/product/view/id/12408 | 1 | NULL | NULL | NULL | 12408 |
| 200760 | 1 | product/12408/8 | all-brands/torano-exodus-1959-50-years-bfc.html | catalog/product/view/id/12408/category/8 | 1 | NULL | NULL | 8 | 12408 |
| 200761 | 1 | product/12408/3535 | all-brands/torano-cigars/torano-exodus-1959-50-years/torano-exodus-1959-50-years-bfc.html | catalog/product/view/id/12408/category/3535 | 1 | NULL | NULL | 3535 | 12408 |
| 200762 | 1 | product/12408/3708 | all-brands/torano-cigars/torano-exodus-1959-50-years-bfc.html | catalog/product/view/id/12408/category/3708 | 1 | NULL | NULL | 3708 | 12408 |
| 200901 | 1 | product/12457 | torano-exodus-1959-50-years-bfc-12457.html | catalog/product/view/id/12457 | 1 | NULL | NULL | NULL | 12457 |
| 200902 | 1 | product/12457/3535 | all-brands/torano-cigars/torano-exodus-1959-50-years/torano-exodus-1959-50-years-bfc-12457.html | catalog/product/view/id/12457/category/3535 | 1 | NULL | NULL | 3535 | 12457 |
| 200903 | 1 | product/12457/3708 | all-brands/torano-cigars/torano-exodus-1959-50-years-bfc-12457.html | catalog/product/view/id/12457/category/3708 | 1 | NULL | NULL | 3708 | 12457 |
+----------------+----------+--------------------+-------------------------------------------------------------------------------------------------+---------------------------------------------+-----------+---------+-------------+-------------+------------+
It appears that product #12408 is taking the URL that belongs to 12457 - so logically I went to find product # 12408 to correct the issue - however - after searching the admin - as well as just querying the database there ISN'T an entity_id 12408 in catalog_product_flat_x that I can find.
This is obviously a problem - and at this point I dont know where to go looking for 12408 to either correct it or remove it so it stops breaking 12457's url.
Can anyone out there tell me how to fix this?
Thanks in Advance!
Related
Sorry for the difficult description, but I know of no better way to say it.
When I look in certain tables of a MySQL database through the phpMyAdmin web interface, the numbers of the first two columns are prefixed by a '3'.
So if the first column has number 99, phpMyAdmin displays 3939.
The second column has number 3592, phpMyAdmin shows 33353932.
If I check the records with the mysql CLI they are displayed correctly.
It only happens to some columns in some tables of some databases... not to all.
So it might be something with the tables themselves or something with phpMyAdmin.
System/version info:
Ubuntu 16.04.3 LTS
phpmyadmin 4:4.5.4.1-2ubuntu2
apache2 2.4.18-2ubuntu3.5
php7.0 7.0.22-0ubuntu0.16.04.1
oracle-java9 9.0.1-1~webupd8~0
mysql-server 5.7.20-0ubuntu0.16.04.1
I'm hoping someone has some idea of what might be happening here.
If you wish any further clarification please let me know.
mysql CLI:
+------------------------------+
| Tables_in_gsb |
+------------------------------+
| gsb_challenge |
| gsb_challenge_pos |
| gsb_challenge_scope |
| gsb_copyright |
| gsb_correction |
| gsb_correction_gold |
| gsb_correction_literality |
| gsb_correction_sentalign |
| gsb_correction_tag |
| gsb_correction_tokenization |
| gsb_correction_wordalign |
| gsb_doc_counts |
| gsb_doc_status |
| gsb_doc_warnings |
| gsb_documents |
| gsb_log |
| gsb_reports_verbnet |
| gsb_semlex_entry |
| gsb_semlex_entry_cooc_cat |
| gsb_semlex_entry_cooc_lemma |
| gsb_semlex_entry_cooc_ne |
| gsb_semlex_entry_cooc_pos |
| gsb_semlex_entry_cooc_roles |
| gsb_semlex_entry_name |
| gsb_semlex_occurrence |
| gsb_status_flag |
| gsb_subcorpus |
| gsb_synsets |
| gsb_user |
| gsb_userCake_Groups |
| gsb_userCake_Users |
| gsb_v_accepted_docs |
| gsb_v_last_doc_status |
| gsb_v_last_doc_update |
| pmb_annotation_status |
| pmb_correction |
| pmb_correction_sentalign |
| pmb_correction_tag |
| pmb_correction_tokenization |
| pmb_dmatch |
| pmb_doc_counts |
| pmb_doc_lists |
| pmb_doc_status |
| pmb_doc_warnings |
| pmb_documents |
| pmb_lrec |
| pmb_lrec_dmatch |
| pmb_lrec_dmatch_backup |
| pmb_lrec_promatch |
| pmb_promatch |
| pmb_promatch_copy |
| pmb_reports_verbnet |
| pmb_semlex_entry |
| pmb_semlex_entry_cooc_cat |
| pmb_semlex_entry_cooc_lemma |
| pmb_semlex_entry_cooc_pos |
| pmb_semlex_entry_cooc_roles |
| pmb_semlex_entry_cooc_semtag |
| pmb_semlex_occurrence |
| pmb_status_flag |
| pmb_subcorpus |
| pmb_synsets |
| pmb_synsets_new |
| pmb_userCake_Groups |
| pmb_userCake_Users |
| pmb_v_phrase_search |
+------------------------------+
SELECT * FROM pmb_semlex_occurrence ORDER BY part DESC LIMIT 10;
+------+--------+-------+------+------+--------+------+----------------+------+----+-----------+---------+-----------+--------+--------+--------+----------+----------------------------------+
| part | doc_id | tokid | from | to | toknum | snum | cat | pos | ne | roles | wordnet | wordsense | semtag | lemma | token | token_lc | entry_id |
+------+--------+-------+------+------+--------+------+----------------+------+----+-----------+---------+-----------+--------+--------+--------+----------+----------------------------------+
| 99 | 3592 | 15056 | 2202 | 2203 | 453 | 15 | (N/PP)\(N/PP) | LQU | | [] | O | O | NIL | " | " | " | 594f1919d740100ed2f3e4cfd4100329 |
| 99 | 3592 | 15055 | 2201 | 2202 | 452 | 15 | (N/PP)\(N/PP) | . | | [] | O | O | NIL | . | . | . | 594f1919d740100ed2f3e4cfd4100329 |
phpmyadmin:
It's not my database, I only maintain the server it's running on, so I don't think it's a prank... and how would they even do that?
With the additional information that these columns are of BINARY type I was able to find two bug reports describing this behaviour:
broken view of binary fields (every second three)
broken link to table of binary fields (every second three)
The first should have been resolved in phpMyAdmin 4.3, but I see that you're using version 4.5. The related second issue was reported against version 4.7.1 and fixed in version 4.7.3.
Try upgrading to phpMyAdmin 4.7.3 or later (at the time of writing the latest version is 4.7.7).
I want to get that in phpword
------------------------
| | |
|-----------------------
| | |
------------------------
| | | | |
|-----------------------
but I keep getting
----------------------
| | |
|---------------------------------
| | | | |
----------------------------------
I am familiar with the fact that I have to somehow incorporate:
array('gridSpan' => 3) ($table->addCell(15000, array('gridSpan' => 3))->addText(htmlspecialchars('text'), $fontTitle);)
but I can't figure out the correct syntax to actually get the table to look the way I want it to work.
I figured it out,
you have to add
$cell5->getStyle()->setGridSpan(3);
I need one help .I am trying to know last updated time of MySQL table .I used the below command.
show table status from spesh like 'db_restaurant_detail';
But i found the following result.
mysql> show table status from spesh like 'db_restaurant_detail';
+----------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------------+----------+----------------+---------+
| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment |
+----------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-------------------+----------+----------------+---------+
| db_restaurant_detail | InnoDB | 10 | Compact | 1171 | 167 | 196608 | 0 | 0 | 5242880 | 7377 | 2016-04-26 08:34:47 | NULL | NULL | latin1_swedish_ci | NULL | | |
+----------------------+--------+---------+------------+------+----------------+-------------+-----------------
Here update_time is showing null. I need when user has updated data in this table.Please help me.
+----------------------------+------------------------------------------------------------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------------------+------------------------------------------------------------------------------+------+-----+---------+----------------+
| type | enum('Website','Facebook','Twitter','Linkedin','Youtube','SeatGeek','Yahoo') | NO | MUL | NULL | |
| name | varchar(100) | YES | MUL | NULL | |
| processing_interface_id | bigint(20) | YES | MUL | NULL | |
| processing_interface_table | varchar(100) | YES | MUL | NULL | |
| create_time | datetime | YES | MUL | NULL | |
| run_time | datetime | YES | MUL | NULL | |
| completed_time | datetime | YES | MUL | NULL | |
| reserved | int(10) | YES | MUL | NULL | |
| params | text | YES | | NULL | |
| params_md5 | varchar(100) | YES | MUL | NULL | |
| priority | int(10) | YES | MUL | NULL | |
| id | bigint(20) unsigned | NO | PRI | NULL | auto_increment |
| status | varchar(40) | NO | MUL | none | |
+----------------------------+------------------------------------------------------------------------------+------+-----+---------+----------------+
select * from remote_request use index ( processing_order ) where remote_request.status = 'none' and type = 'Facebook' and reserved = '0' order by priority desc limit 0, 40;
This table receives an extremely large amount of writes and reads. each remote_request ends up being a process, which can spawn anywhere between 0 and 5 other remote_requests depending on the type of request, and what the request does.
The table is currently sitting at about 3.5 Million records, and it goes to a snail pace when the site itself is under heavy load and I have more then 50 or more instances running simultaneously. (REST requests are the purpose of the table just in case you were not sure).
As the table grows it just gets worse and worse. I can clear the processed requests out on a daily basis but ultimatly this is not fixing the problem.
What I need is for this query to always have a very low response ratio.
Here are the current indexes on the table.
+----------------+------------+----------------------------------+--------------+----------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------------+------------+----------------------------------+--------------+----------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| remote_request | 0 | PRIMARY | 1 | id | A | 2403351 | NULL | NULL | | BTREE | | |
| remote_request | 1 | type_index | 1 | type | A | 18 | NULL | NULL | | BTREE | | |
| remote_request | 1 | processing_interface_id_index | 1 | processing_interface_id | A | 18 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | processing_interface_table_index | 1 | processing_interface_table | A | 18 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | create_time_index | 1 | create_time | A | 160223 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | run_time_index | 1 | run_time | A | 343335 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | completed_time_index | 1 | completed_time | A | 267039 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | reserved_index | 1 | reserved | A | 18 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | params_md5_index | 1 | params_md5 | A | 2403351 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | priority_index | 1 | priority | A | 716 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | status_index | 1 | status | A | 18 | NULL | NULL | | BTREE | | |
| remote_request | 1 | name_index | 1 | name | A | 18 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | processing_order | 1 | priority | A | 200 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | processing_order | 2 | status | A | 200 | NULL | NULL | | BTREE | | |
| remote_request | 1 | processing_order | 3 | type | A | 200 | NULL | NULL | | BTREE | | |
| remote_request | 1 | processing_order | 4 | reserved | A | 200 | NULL | NULL | YES | BTREE | | |
+----------------+------------+----------------------------------+--------------+----------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Any idea how i solve this? Is it not possible to make some sort of complicated index that would automatic order them with priority, then take the first 40 that match the 'Facebook' type? It currently is scanning more then 500k rows of the table before it returns a result which is grossly inefficient.
Some other version of the query that I have been tinkering with are:
select * from remote_request use index ( type_index,status_index,reserved_index,priority_index ) where remote_request.status = 'none' and type = 'Facebook' and reserv ed = '0' order by priority desc limit 0, 40
It would be amazing if we could get the rows scanned to under 1000 rows depending on just how many types of requests enter the table.
Thanks in advance, this might be a real nutcracker for most except the most experienced mysql experts?
Your four-column index has the right columns, but in the wrong order.
You want the index to first look up matching rows, which you do by three columns. You are looking up by three equality conditions, so you know that once the index finds the set of matching rows, the order of these rows is basically a tie with respect to those first three columns. So to resolve the tie, add as the fourth column the column by which you wanted to sort.
If you do that, then the ORDER BY becomes a no-op, because the query can just read the rows in the order they are stored in the index.
So I would create the following index:
CREATE INDEX processing_order2 ON remote_request
(status, type, reserved, priority);
There's probably not too much significance to the order of the first three columns, since they're all in equality terms combined with AND. But the priority column belongs at the end.
You may also like to read my presentation How to Design Indexes, Really.
By the way, using USE INDEX() shouldn't be necessary if you have the right index, MySQL's optimizer will choose it automatically most of the time. But USE INDEX() can block the optimizer from considering a new index that you create, so it becomes a disadvantage for code maintenance.
This isn't a complete answer but it was too long for a comment:
Are you actually searching on all of those indexes? If not get rid of some. Extra indexes slow down writes.
Secondly use EXPLAIN on your query and don't specify an index when you do. See how MySQL wants to process it rather than forcing an option (Generally it does the right thing).
Finally sorting is likely what hurts you the most. If you don't sort it probably gets the records pretty quickly. It has to scan and sort every row that meets your criteria before it can return the top 40.
Options:
Try creating a VIEW (not as familiar with VIEWS but it might work)
Split this table into smaller tables
use a third party tool such as
Sphinx or Lucene to create specialized indexes to search on. (I've
used Sphinx for something like this before. You can find it at
http://sphinxsearch.com/).
Or look into using a NoSQL solution where you can use a Map function to do it.
Edit I read a bit about using VIEW and I don't think it will help you in your case because you have such a large table. See the answer in this thread: Using MySQL views to increase performance
This question is for my pastebin app written in PHP.
I did a bit of a research, although I wasn't able to find a solution that matches my needs. I have a table with this structure:
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(12) unsigned | NO | PRI | NULL | auto_increment |
| author | varchar(50) | YES | | | |
| authorid | int(12) unsigned | YES | | NULL | |
| project | varchar(50) | YES | | | |
| timestamp | int(11) unsigned | NO | | NULL | |
| expire | int(11) unsigned | NO | | NULL | |
| title | varchar(25) | YES | | | |
| data | longtext | NO | | NULL | |
| language | varchar(50) | NO | | php | |
| password | varchar(60) | NO | | NULL | |
| salt | varchar(5) | NO | | NULL | |
| private | tinyint(1) | NO | | 0 | |
| hash | varchar(12) | NO | | NULL | |
| ip | varchar(50) | NO | | NULL | |
| urlkey | varchar(8) | YES | MUL | | |
| hits | int(11) | NO | | 0 | |
+-----------+------------------+------+-----+---------+----------------+
This is for a pastebin application. I basically want paste revisions so that if you open paste #1234, it shows all past revisions of that paste.
I thought of three ways:
Method 1
Have a revisions table with id and old_id or something and for each ID, I would insert all old revisions, so if my structure looks like this:
rev3: 1234
rev2: 1233
rev1: 1232
The table will contain this data:
+-------+----------+
| id | old_id |
+-------+----------+
| 1234 | 1233 |
| 1234 | 1232 |
| 1233 | 1232 |
+-------+----------+
The problem which I have with this is that it introduces a lot of duplicate data. And the more the revisions get, it has not only more data but I need to do N inserts for each new paste to the revisions table which is not great for a large N.
Method 2
I can add a child_id to the paste table at the top and just update that. And then, when fetching the paste, I will keep querying the db for each child_id and their child_id and so on... But the problem is, that will introduce too many DB reads each time a paste with many revisions is opened.
Method 3
Also involves a separate revisions table, but for the same scenario as method 1, it will store the data like this:
+-------+-----------------+
| id | old_id |
+-------+-----------------+
| 1234 | 1233,1232 |
| 1233 | 1232 |
+-------+-----------------+
And when someone opens paste 1234, I'll use an IN clause to fetch all child paste data there.
Which is the best approach? Or is there a better approach? I am using Laravel 4 framework that has Eloquent ORM.
EDIT: Can I do method 1 with a oneToMany relationship? I understand that I can use Eager Loading to fetch all the revisions, but how can I insert them without having to do a dirty hack?
EDIT: I figured out how to handle the above. I'll add an answer to close this question.
If you are on Laravel 4, give Revisionable a try. This might suite your needs
So here is what I am doing:
Say this is the revision flow:
1232 -> 1233 -> 1234
1232 -> 1235
So here is what my revision table will look like:
+----+--------+--------+
| id | new_id | old_id |
+----+--------+--------+
| 1 | 1233 | 1232 |
| 2 | 1234 | 1233 |
| 3 | 1234 | 1232 |
| 4 | 1235 | 1232 |
+----+--------+--------+
IDs 2 and 3 show that when I open 1234, it should show both 1233 and 1232 as revisions on the list.
Now the implementation bit: I will have the Paste model have a one to many relationship with the Revision model.
When I create a new revision for an existing paste, I will run a batch insert to add not only the current new_id and old_id pair, but pair the current new_id with all revisions that were associated with old_id.
When I open a paste - which I will do by querying new_id, I will essentially get all associated rows in the revisions table (using a function in the Paste model that defines hasMany('Revision', 'new_id')) and will display to the user.
I am also thinking about displaying the author of each revision in the "Revision history" section on the "view paste" page, so I think I'll also add an author column to the revision table so that I don't need to go back and query the main paste table to get the author.
So that's about it!
There are some great packages to help you keeping model revisions:
If you only want to keep the models revisions you can use:
Revisionable
If you also want to log any other actions, whenever you want, with custom data, you can use:
Laravel Activity Logger
Honorable mentions:
Activity Log. It also has a lot of options.