I'm doing a manual WordPress update.
I backed up the database to a .sql file.
Is there any way to prevent write to database temporarily while still allowing read to the whole MySQL database?
This is to ensure
The backed up database is up to date
Users are still able to browse contents on my website without disruption (I will put up a maintenance notice that your posts will not be saved etc.)
Update
The upgrading activity is only used as an example here.
I'm planning to make some changes directly to the database as well and it will take a while.
I'm sure I have seen websites (famous one) showing that they were under maintenance and that my comments/ posts would not be recorded (no write) but I was still able to browse their websites (read is ok).
I thought it was quite a reasonable need, was it not?
I'm sure there must be a way to
Serve the caches of webpages (server-side) to users while not connecting the the database at all in up to a couple of hours (logging in/ registration will not available to users but that's ok)? How do you think I can achieve that then?
To lock out any visitors from posting comments and such, you can go into phpmyadmin, find the wordpress user account (DB_USER as defined in wp-config.php) and revoke its insert, edit and update privileges. But this will probably not degrade gracefully into user friendly error messages when they still try anyways ignoring your messages. And if you are doing an update through php script in the way the wordpress installs itself it may still need these privledges to make any necessary modifications like adding options to the options table.
What version are you upgrading from and to? I know the install took me all of 30 seconds, and I can't imagine there would be a huge change in database between updates. But then again WP can be highly customized and I don't know the extent your site deviates from the standard install.
Sorry I'm about to go to sleep so I won't go into depth but one way that works for sure is if you have two MySQL users, 1 for your website to read/write from and another to do your maintenance. Your first user, you can change his privileges temporarily to read only and that should work.
Related
We have Magento EE 1.14.0.1. recently we moved to new AWS EC2 server and ElasticCache Redis server. then some random products start disappearing in the frontend. They exist on backend and configured correctly ( visible , enabled , in stock , etc .... ). And only after you save the product in backend it will show up again in frontend even without flushing any cache.
Is this issue related to Redis cache ?
and if its how to fix it ?
Any input would be appreciated to direct me to a solution.
Thanks
Update: I marked everything under Index Management to Update on Save. so I revert that back to update on schedule. and I think that fixed the issue. but still I want to keep my store inventory up to date.
"It's an index issue, every time you update data (product, stock) from database, you have to manually re-index Magento."
That is true for Community Edition, not Enterprise Edition. In addition, there can be some extra issues when migrating to AWS. After 4 months of troubleshooting on an inherited server migrated into AWS I found a number of issues/solutions.
EE issues
Enterprise Edition indexing is asynchronous for many of the indexes. In addition, not all EE indexes are configured in the typical place.
On the Admin menu, select System > Configuration. In the panel on the left, under Advanced, select Index Management.
http://docs.magento.com/m1/ee/user_guide/system-operations/index-configuration.html
Even when set to "update on save" in my experience it frequently does not update on save.
AsyncIndexing was unstable in versions prior to 1.14.3.x .
Upgrade! It was possible for the partial process to break in such a way as to make it impossible for indexing to proceed. One way this will occur is if you are running PHP for the website[typically via PHPFPM] with a different userid and group then you run the cronjobs[shell access]. Indexing depends on the creation of a file to 'lock' the process - the file may only be written/deleted by the user which creates it.
I have found that for performance reasons, it is best to set ALL indexes to "update manually". Do not schedule a periodic reindex all process, it is useless due to async indexing. Just make sure your cron is running and everything should be fine.
The AsyncIndex process uses MySQL triggers...which have an issue when trying to migrate a magento database from one server to another. The way they are created initially, they can ONLY be used by the database user that when the triggers where first created. If you change the database user for the new server, the triggers will not migrate. Even worse, there is almost no indication that this occurred and everything except indexing runs perfectly so how can you tell?
Lastly, "reindex all" does not always reindex all. Thanks to various posts on the internet, I created a shell script to make Magento think all the products were updated and the index needs to be rebuilt:
https://gist.github.com/gamort/5dc5e16bdec00a8bb3b922fc463af17c
AWS issues
Using AWS Elasticache Redis has a hidden gotcha - the default zone it is launched in may be different then your server zone. In my case, the server was in USEAST-1a while Redis defaulted to USEAST-1b. This resulted in occasional timeouts when looking up data from the cache. While the website code can usually recover, the indexing code does not. This leads to the index cron process being in a broken state.
Almost as importantly, you will pay a trivial amount per GB for data transfer from zone 1a to 1b. But when your cache is working, this "trivial" amount can amount to a lot! We had a recurring $10+/day [$500-$600 a month] intrazone data transfer fee! Launch a new redis server in your actual zone, use the redis cli on your web server to make sure you can connect[we had firewall configuration issues] and then only THEN update your configuration.
AWS RDS server also have a hidden gotcha[hope your not too overwhelmed yet]. Migrating the database from another server to Amazon RDS has issues where there was an extremely slight change in what MySQL considers valid SQL for a specific function...which Magento EE just happens to use. :-) . I ended up installing a new copy of Magento EE and using Navicat to sync the database structures.
Solr issues
Suffice to say, there are Solr issues as well. Mostly due to the schema, but I also found that wiping the solr database and letting it reindex helped.
Magento Rewrite/Form issues
This issue occurs when you upgrade to 1.14.3 - which of course you should do since it fixes so many indexing issues. Version 1.14.3.x added form keys to a number of forms, including the customer sign up form. So if you created your own custom phtml templates for the logon they will not work! You need to add that form key field to your customization. Not a big deal though, since you documented what template file you copied it from initially right?
All in all, I would estimate going through the checklist for migration to be a good 20 hours, and possibly up to 80 depending on what issues you run into. And at the end of the day, since the fixes are mainly in cron jobs which are not easily visible the website owner will be hard pressed to tell how they benefitted from all that work. In my case, disappearing products had already been an issue for over a year before we inherited the site the client was understanding about the difficulties.
It's an index issue, every time you update data (product, stock) from database, you have to manually re-index Magento. If you don't do that, you'll have corrupted data on index and you'll lose SQL join on product request list.
I have a PHP application which is on a production server and it`s meant to register users to some services. It has two forms which registers a user in a different table from my database.
Problem is that today one of the tables disappeared and I was able to restore it from a backup. But this dose`t get rid of the problem.
How do I investigate this in order to determine how that table got lost and most likely dropped by some bot or something.
How would you proceed in a situation like this?
There are two ways:
Have a working backup of the system, and restore the files from it.
An undelete tool might help, if you deleted the db very recently (and ideally, if you unplugged the computer right afterward).
As for doing it with MySQL, though...on all systems i'm aware of, no. MySQL tables are files in the server's data directory, and dropping a table deletes those files. Once they're gone, they're gone, and only the methods above can get them back. A database is a directory of those files, and dropping it deletes the whole directory.
Check this free software
http://www.majorgeeks.com/Restoration_d4474.html
More information here - http://emaillenin.blogspot.com/2010/11/recover-accidentally-deleted-mysql.html
If your tables got dropped, find out what mysql users have privileges to drop a table (there shouldn't be many) and what services log in with that user credentials.
Maybe you have a web form with a php backend that doesn't clean up (escape) input, so you were maybe open to sql injection.
In that case, you could check you webserver access log.
I have created a PHP+MYSQL web app and I am trying to implement now a logging system to store and track some actions of each user.
The purpose of this is the following: track the activity of each user's session by logging the IP+time+action, then see which pages he accessed later on by logging time+pagename; for each user there will be a file in the format: log{userid}_{month}.log
Each log will then be viewed only by the website owner, through a custom admin panel, and the data will be used only for security purposes (as in to show to the user if he logged in from a different IP or if someone else logged in from a different IP and to see which areas of the website the user accessed during his login session).
Currently, I have a MYSQL MyISAM table where I store the userid,IP,time,action and the app is still not launched, but we intend to have very many users (over 100k), and using a database for this solutions feels like suicide.
So what do you suggest? How should the logging be done? Using files, using a table in the current database, using a separate database? Are there any file-logging frameworks available for PHP?
How should the reading of the file be done then? Read the results by row?
Thank you
You have many options, so I'll speak from my experience running a startup with about 500k users, 100k active every month, which seems to be in your range.
We logged user actions in a MySQL database.
Querying your data is very easy and fast (provided good indexes)
We ran on Azure, and had a dedicated MySQL (with slaves, etc) for storing all user data, including logs. Space was not an issue.
Logging to MySQL can be slow, depending on everything you are logging, so we just pushed a log to Redis and had a Python app read it from Redis and insert into MySQL in the background. This made that logging basically had no impact on loading times.
We decided to log in MySQL for user actions because:
We wanted to run queries on anything at any time without much effort. The structured format of the user action logs made that incredibly easy to do.
It also allows you to display certain logs to users, if you would require it.
When we introduced badges, we had no need to parse text logs to award badges to those who performed a specific action X number of times. We simply wrote a query against the user action logs, and the badges were awarded. So adding features based on actions was easy as well.
We did use file logging for a couple of application logs - or things we did not query on a daily basis - such as the Python app writing to the database, Webserver access and error logs, etc.
We used Logstash to process those logs. It can simply hook into a log file and stream it to your Logstash server. Logstash can also query your logs, which is pretty cool.
Advanced uses
We used Slack for team communications and integrated the Python database writing app with it, this allowed us to send critical errors to a channel (via their API) where someone could action a fix immediately.
Closing
My suggestion would be to not over think it for now, log to MySQL, query and see the stats. Make updates, rinse and repeat. You want to keep the cycle between deploy and update quick, so making decisions from a quick SQL query makes it easy.
Basically what you want to avoid is logging into a server, finding a log and grep your way through it to find something, the above achieved that.
This is what we did, it is still running like that and we have no plans to change it soon. We haven't had any issues where we could not find anything that we needed. If there is a massive burst of users and we scale to 1mil monthly active users, then we might change it.
Please note: whichever way you decide to log, if you are saving the POST data, be sure to never do that for credit card info, unless you are compliant. Or rather use Stripe's JavaScript libraries.
If you are sure that reading the log will mainly target one user at a time, you should consider partioning your log table:
http://dev.mysql.com/doc/refman/5.1/en/partitioning-range.html
using your user_id as partitioning key.
Maximum number of partitions being 1024, you will have one partition storing 1/1000 of your 100k users, which is something reasonable.
Are there any file-logging frameworks available for PHP?
There is this which is available on packagist: https://packagist.org/packages/psr/log
Note that it's not a file logging framework but an API for a logger based on the PSR-3 standard from FIG. So, if you like, it's the "standard" logger interface for PHP. You can build a logger that implements this interface or search around on packagist for other loggers that implement that interface (either file or MySQL based). There are a few other loggers on packagist (teacup, forestry) but it would be preferable to use one that sticks to the PSR standard.
We do logging with the great tool Graylog.
It scales as high as you want it, has great tools on data visualization, is incredibly fast even for complex querys and huge datasets, and the underlying search-enginge (elasticsearch) is schemaless. The latter may be an advantage as you get more possibilities on extending your logs without the hassle mysql-schemas can give you.
Graylog, elasticsearch and mongodb (which is used as to save the configuration of graylog and its webinterface) are easily deployable via tools like puppet, chef and the like.
Actually logging to graylog is easy with the already mentioned php-lib monolog.
Of curse the great disadvantage here is that you have to learn a bunch of new tools and softwares. But it is worth it in my opinion.
The crux of the matter is the data you are writing is not going to be changed. In my experience in this scenario I would use either:
MySQL with a blackhole storage engine. Set it up right and its blisteringly fast!
Riak Cluster (NoSQL solution) - though this may be a learning curve for you it might be one you may need to eventually take anyway.
Use SysLog ;)
Set it up on another server and it can log all of your processes seperately (such as networking, servers, sql, apache, and your php).
It can be usefull for you and decreasing the time spend of debugging. :)
building a CMS and looking to find out the correct way of showing a maintenance mode message to users.
My plan is to have the option in my admin backend. I already have the table in my DB for website config and i have a field that just has a value. I was using 1 for normal and 2 for maintenance mode. I just want to be able to update the database.
My question is however, whats the proper way to do this?
My first thought was to check the DB and then if the value is "2" i would redirect the user (all PHP) to e.g. 503.php.
But i have also seen that a better way is in the .htaccess and then i can allow a certain IP to still have access. But can this in the .htaccess be managed from a database check?
Looking to do things properly so curious as the the 'standards' way of doing this.
Is my first suggestion feasible? The main reason for the 'maintenance mode' is actually to protect myself. My site has gone through lots of testing, but i want a way i can quickly shut down access to the site (SEO freindly) just whilst i fix any issues. Or whilst i am updating the site itself.
Thanks
You shouldn't control the maintenance mode setting from the database, if you don't have to.
The database adds another layer of complexity which may fail at some point.
In practice, you have a few options:
Add a text file with the setting for maintenance mode. Prior to serving a request, use PHP to open and read the value of the file. Then, you can handle it appropriately by issuing a redirect if needed.
Use an external service like Pingometer to monitor your site(s). This automates the process and, if any of them go down, use their web hook functionality to send a request to a PHP script that switches this value (perhaps using the method described in #1) to maintenance mode.
I think your goal is that, if happens to the site (externally), your maintenance mode is automatically tripped -- #1 and #2 offer an easy solution.
I'm helping a friend migrate her wordpress server to GoDaddy, and I think I may have bitten off more than I can chew... I've never migrated a wordpress before. This page here is the Wordpress wiki for moving Wordpress when your domain isn't changing. It doesn't seem to complex, but I'm terrified of accidentally ruining this website and I don't understand a couple of things on the wiki.
The Wiki says
If database and URL remains the same, you can move by just copying your files and database.
Does this mean that I can just log in to her server from Filezilla and copy all of the files on the server? What does database mean, is that something separate from the files on the server?
If database name or user changes, edit wp-config.php to have the correct values.
This sort of goes with my first question.. What initiates a database name or user change?
Apologies for my ignorance, but after an hour or so of searching around for these answers I'm left just as confused.
Last but not least, is there anything else I should be aware of when migrating a wordpress? I'm a little nervous..
You are going to need to migrate you instalation in two parts.
Part 1 you already eluded to. You will need to copy the files from one server to another. I am guessing you know how to do this so I will not dive any deeper into it. If you do need more explanation, please let me know and I will edit the question.
Part 2 is what you mentioned but said you did not understand. Copying the database of wp install. Wordpress runs off of PHP and MySQL. The "files" part in part 1 is the PHP files (along with some html and css). You need to log into his MySQL server and do an export of his database. You should be able to export the database (How to export mysql database to another computer?) and import it into his new server on GoDaddy. (Error importing SQL dump into MySQL: Unknown database / Can't create database).
Just take things slow, follow the guides that I have linked and do not delete anything from the first server until everything is working on the second. Please let me know if you do not understand anything.
if you don't feel confortable with database exports and imports, try using plugins like:
http://wordpress.org/plugins/duplicator/
or
http://wordpress.org/plugins/wordpress-move/
Check his docs for info.
Luck!
• A database is literally a data base. It's where websites (and other applications) store their data eg. For Wordpress, it would be data such as posts, user information etc.
If you are using a cPanel setup then you would need to get access to it and navigate to phpMyAdmin which is the GUI for managing a database.
Now I'm not sure what type of setup you're using but that should be a start.
• A database has a connection server address (usually localhost), a database name, username and password. These are setup at the time of setting up a database.
When migrating servers, you would need to update those details in the wp-config.php file (I think around line 19 or so).
• The annoying part about migrating Wordpress to another server is the domain change as you have to update the old domain with the new domain throughout the database. However since you're not changing domain names, it should be a smooth ride as long as the new server supports PHP and has a database.