Duplicated content on google. htaccess or robots.txt? [closed] - php

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
In my website I have the following categories url structure:
/category.php?id=6 (id=6 is for internet category)
My SEO friendly url is like:
/category/6/internet/
The problem is it can be accessed in any of those forms, and because of that, I'm getting duplicate content on google.
So, I'm wondering how can I fix that.
Should I disallow on robots.txt any urls with ? on it?
If so, how can I properly set it up?
Should I make a redirection "Permanently Moved" on .htaccess?
If so, how can I properly set it up?
My actual .htaccess for categories is like this:
RewriteRule ^category/([^/]*)/([^/]*)/$ category.php?id=$1&name=$2 [L]

You just need to set the canonical link tag in the head section of your pages
see http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html
and http://support.google.com/webmasters/bin/answer.py?hl=en&answer=139394
It will look something like
<link rel="canonical" href="http://www.example.com/category/6/internet/"/>
on the category 6 page
You could also do a 301 redirect for the category.php pages in your .htaccess by adding
RewriteRule ^category.php?id=([^&]*)&name=([^&]*) /category/$1/$2/ [R=301,L]
If you didn't want to go the route of rewriterules you could put the following code at the top of config.php:
if(preg_match('/^\/config\.php/', $_SERVER['REQUEST_URI'])){
header("HTTP/1.1 301 Moved Permanently");
header("Location: /category/{$_GET['id']}/{$_GET['name']}");
die();
}
Either way is up to you but I would use the rewriterule option to redirect to my SEO friendly URL If I were you

Or you get rid of the non-SEO url.
always generate SEO url (you should do that anyway for)
in category.php check if $_SERVER['REQUEST_URI'] is the seo one and if not redirect to it

I would suggest using a canonical link in the document head to ensure Google uses the correct URL. Google on Rel Canonical.
It's really easy to implement, just post this into the HEAD section of the page.
<link rel="canonical" href="/your/url"/>
Google treats the canonical link as a 301 redirect, which means you won't have any duplicate content issues. It also means most of the link juice gets passed on (between 90% and 99%). If you used robots.txt or .htaccess , the page that you're blocking off would lose all its SEO value.
Just make sure you do this for every page as it's a page specific rule. Linking to the domain root will effectively mean all your pages are 301 redirecting to the home page.

Related

Wordpress Blog Post Redirect (1000 posts)

I have a wordpress blog with the following URL structure:
www.mysite.com/2016/09/the-name-of-the-post/
However i noticed that for SEO Reasons it would be better to have a structure like:
www.mysite.com/blog/the-name-of-the-post/
I have around 1000 posts with the "old" url structure. I can change the url structure in wordpress. However i would need a 301 redirect if someone tries to access a post using the old url.
E.g.
person/ google bot tries to access page via /2016/09/the-name
-> 301 Redirect to /blog/the-name
What is the best practice to do so? Do I add 1000 lines to the .htaccess file and do redirects? Would that negatively influence the server response time cause apache has to check through a long list of redirects first?
Can I tweak a file in the wordpress file structe which checks, if the requested page is included in e.g. an array, it will redirect it to the new url?
Thank you very much for your suggestions
You can simply change the URL structure in WordPress to the one you want to have. You don’t have to modify your .htaccess or anything else.
WordPress will understand which post is referenced by the URL and redirect to the new canonical URL automatically.
Just go to the Permalinks admin page and select the “Post name” option as described in the Codex.
However, if you’re introducing a prefix like /blog/, too, you must redirect to the new URL base through .htaccess, e.g.
RedirectMatch 301 ^/[0-9]{4}/[0-9]{2)/(.+)$ http://example.com/blog/$1 // taken from stackoverflow.com/a/42211746/
You can use the following Redirect
RedirectMatch 301 ^/[0-9]{4}/[0-9]{2)/(.+)$ http://example.com/blog/$1
This will redirect all urls from this form /1234/12/foobar to this /blog/foobar
Change the permalink to the new structure and add this in your .htaccess
RedirectMatch 301 ^/([0-9]{4})/([0-9]{2})/(.*)/$ http://example.com/blog/$3
I had the same problem before but I resolve this issue using simple 301 redirects plugin.
https://wordpress.org/plugins/simple-301-redirects/
You can bulk upload 301 redirects using bulk upload add-ons of this plugin.
Hope this will help you.

SEO - Pretty URLs and Dynamic URLs at the same time [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
I was wondering if a site using both pretty urls and dynamic urls will be penalized for duplicate content.
Let's say
http://example.com/article/1 is the same as http://example.com/?article=1. Is this bad for SEO?
Extra question:
Entering http://example.com/?blabla=qwerty will load the default home page. Is http://example.com/?blabla=qwerty treated as different page than http://example.com ?
What happens if the user enters http://example.com/????article=1, is it different than http://example.com/?article=1? Thanks
Forget end users - if search engine bot can index both the pages then it's bad SEO.
Let's say if Google is indexing http://example.com/article/1 as well as http://example.com/?article=1 then it will be treated as duplicate content on same site.
However http://example.com/?blabla=qwerty and http://example.com and all such variations are treated as a single page.
So it's not bad for SEO, but definitely not a good strategy. Best practice is to redirect http://example.com/?article=1 to http://example.com/article/1.
http://example.com/article/1 and http://example.com/?article=1 is treated as two different URLs to a search engine. They are bad for SEO because of the following reasons:
link juice is split between the 2 URLs
duplicate content on the same site as mentioned by Pavan.
As in 1, the same principle applies. http://example.com/?blabla=qwerty is indeed treated as a different page than http://example.com/
http://example.com/????article=1 is indeed different from http://example.com/?article=1. In the first case, the GET parameter is "???article" having a value of 1 and the second example, the GET parameter is "article" having a value of 1.
Now to solve this, you can use one of several strategies:
Use Canonical URLs
The Canonical URL serves to consolidate link signals for the duplicate or similar content. More to read on Google Webmaster Tools. In your case, you should add a canonical URL in the header such as.
<link rel="canonical" href="http://example.com/article/1" />
More background information can be found on Moz
Use 301 redirects
Where links are not canonical, use 301 permanent redirects to pass over the link juice to the new URL. Reference: Moz 301
Using rel="next" and rel="prev"
Where there's paginated content, use HTML attributes rel="next" and rel="prev" to indicate to google that the pages are paginated and linked. This will solve handle issues such as ?page=1 and ?page=2. Read more about Indicate paginated content at Google.
<link rel="prev" href="http://www.example.com/article?page=1">
<link rel="next" href="http://www.example.com/article?page=3">

Change domain, redirect and don't lose page rank [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
I have a well-known domain name and receive something like 5k visitors per day.
But i really want to change my domain and my pages title, a complete renew!
Have some way to do this without losing my page rank and visitors?
For example, now if someone from my country (brazil) search about a tablet my site will appear in first page.
In really, i have 3 sites with one domain.
shop.mydomain.com
forum.mydomain.com
www.mydomain.com (wordpress blog)
If possible i want to change just the shop domain.
A long time ago i read about a changes like that, 301 redirect or something like this that preserves your page rank and google position.
My new domain is exatcly the keywork that my visitors search, probably with this change i will get the first place in searches.
I do not want lose backlinks
Help please?
As Matt Cutts explains, a 301 redirect is the best way to redirect users:
If you need to change the URL of a page as it is shown in search engine results, we recommend that you use a server-side 301 redirect. This is the best way to ensure that users and search engines are directed to the correct page. The 301 status code means that a page has permanently moved to a new location.
This CSS Tricks article explains how to implement a 301 redirect, but essentially:
If you want to use a 301 redirect (which is the quickest, cleanest, and most search engine friendly option) to direct browsers to an entirely new website, place this line in the .htaccess of the old site
Redirect 301 / http://newsite.com/
If you only want to redirect a particular page, place this in the .htaccess of the old site
Redirect 301 /oldpage.html http://www.yoursite.com/newpage.html

How can I get search engines to index dynamically generated pages [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I'm trying to create a blog ..and for every entry I have a template page called entry.php
I create the pages by creating a link from my home page like this
mywebsite.com/entry.php?title=My Title&content=This is my blog
This link is passed to entry.php and a page is generated on the fly based on the link i wrote but search engines wont index these.
Do I really have to create a unique page for every entry I create?
Do sites like youtube have individual pages for every single video or are they generated dynamically like im trying to do. If so how do the videos show up in search results?
Ive heard something called .htaccess or sitemap.xml can be used for this I have no idea what these are though.
Learn about .htaccess
There are a lot of generators for .htaccess and you can use one of them. The best thing to do is, using a .htaccess file, create the paths this way:
mywebsite.com/entry.php?title=My Title&content=This is my blog
Change them to:
mywebsite.com/My Title/This is my blog
And the code for the same is:
RewriteEngine On
RewriteRule ^([^/]*)/([^/]*)\.html$ /entry.php?title=$1&content=$2 [L]
Websites offering generators:
.htaccess redirect
Mod Rewrite Generator
Search Engine Optimization Tools » mod_rewrite rewriterule generator
As long as the page is linked to from somewhere, search engines will find the page. It does not matter how the page is generated. A link is a URL, when requesting this URL the browser or any other client (including search engines) receive the page content, period. It works like any other regular page, search engines don't know or care what happens behind the scenes to generate this page.

Page not found or redirect? [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
My rewrite rules:
RewriteRule ^articles/([a-z]+)/([0-9]+)/?$ /index.php?page=articles&cat=$1&id=$2 [L]
RewriteRule ^articles/([a-z]+)/?$ /index.php?page=articles&cat=$1 [L]
RewriteRule ^([a-z]+)/?$ /index.php?page=$1 [L]
So, in index.php file, i check for existence of page. If it is found, that page is echoed. Otherwise, what to do ?
Send a redirect to home page ?
Send 404 page not found response using header() ?
Echo a message like "Page Not Found" ?
Which one is good? I want to know search engine friendly option among these.
You should definitely have a header() which returns the proper 404 error code. This will ensure that google does not index your error page. Serving a standard page with a message without the 404 code is called a soft 404 error, and is discouraged by the standards (and by google).
See here: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=181708
But it is not necessary to show an ugly text-only error to your visitor, or rely on his browser's default error page. Even if you output a 404 code, you can still serve a beautiful page, maybe with a link to your home page, or with relevant resources.
In precedence
Send a 404 page not found response using header() ?(hard 404)
Which is ideal as the page was literally not found.
Echo a message like "Page Not Found" ? (soft 404)
This is fine as you are trying to find the page internally but not exactly ideal.
Do not do a:
Send a redirect to home page ?
As this does not portray any information like,
whether the page was not found.
was there any internal error.
etc.
Go through this: 404 errors soft vs. hard you will get the idea.

Categories