For large arrays, is it better to save the data to global variables or query the database each time I need them? In my situation keeping them local scope and passing them to functions isn't an option.
I'm using wordpress and in most pages I get every user and all metadata attached to them. Often times I use these variables in multiple places on the same page. Unfortunately wordpress won't let me pass variables between templates so I'm stuck either using global variables or calling the database each time. Eventually, this will be hundreds of users with a lot of metadata attached to each. Should I call the database each time to keep the variables local, or should save them to global variables to save on database queries? What are the considerations? Should I worry about performance, overhead, and/or other issues?
Thanks so much!
The only real solution to your problem is using some kind of cache system (Memcache and Redis are your best options). Fortunately, there are plenty of Wordpress plugins that make the integration an easy thing. For instance:
Redis: https://wordpress.org/plugins/redis-object-cache/
Memcache: https://wordpress.org/plugins/memcached/
EDIT
If you only want to cache a few databases calls, you can forget about Wordpress plugins and start coding a bit. Let's say you only want to cache the call for retrieving the list of users from database, and let's assume you are using Memcache to accomplish this task (Memcache stores key-value pairs and allows super fast access to a value given a key).
Query Memcache asking for the key "users".
Memcache still doesn't have such key, so you'll have a cache fail and after it, you'll query your database to retrieve the user list. Now serialize the database response (serialize and json_encode are two different ways to do this) and store the key "users" along this serialized value in your memcache.
Next time you query your memcache asking for "users", you'll get a hit. In this moment you just have to unserialize the value and work with your user list.
And that's all. Now you just have to decide what you want to cache and apply this procedure to those elements.
You shouldn't have to perform the calls but once per page, you might have to execute the call once for every page. So I would suggest you creating some sort of class to interact with your database that you can call on to get the data that you need. I would also recommend using stored procedures and functions on your database instead of straight queries since this will help both with security and separation of application logic and data functionality.
Related
I am making a program that would pose a question and users would be able to answer it. This is a Internet of things project and as such there will be a lot of calls to the database, which would only want to increase the value of answer A or B by one. For the record I am using PHP and Mysql and there are many unique items that would send update requests to the server.
What can I do to reduce the calls to the database?
The solution I came up with was to store somehow the data on the server, then sync the data with the database on a scheduled interval.
To update the results I would need to know only 3 things - item id, and both results. For code clarity and simplicity I made a model object with those attributes.
So far I came up with/found several ideas:
Sessions - make session an array and just put the model objects inside it
Create a file on the server that would store the data
Use superglobal variables
Create a PHP class that would have an array, in which the objects would go to and interact with the class
Use some API - but I would be completely dependent on it working
Which one of the solutions given be the best in terms of simplicity, security and performance or is there a better way to do this whole thing?
You can have one database for row data and once you authorize then you can sync the data into actual database which your application needs. After sync, you can run a job of deleting unwanted data from the row database. Let me know if it works for you.
I read a lot about how evil global variables are in PHP, but I am trying to optimize a code I am writing. In this webapp a lot of functions are using the same data (about up to 50 items at once) to perform numerous operations and the data itself is stored in a database.
I have two options which are a) fetching data from the database EVERY TIME a function needs it or b) fetching the data ONCE and storing it in (a) global variable(s).
When it comes to performance, which option is the best ?
There is nothing wrong with "global variables". It is passing data into functions using global keyword is prohibited (but nevertheless, using this keyword to pass indeed global variable is okay).
Yes, speaking of one script instance (and sane amount of data), there is no use for reaching for database for the same data again. Fetch it once and then use in whatever functions you need. It's okay and nothing wrong with it.
When it comes to performance, here comes the best option ever:
do care of performance only if you have a certain reason to.
I've implemented an Access Control List using 2 static arrays (for the roles and the resources), but I added a new table in my database for the permissions.
The idea of using a static array for the roles is that we won't create new roles all the time, so the data won't change all the time. I thought the same for the resources, also because I think the resources are something that only the developers should treat, because they're more related to the code than to a data. Do you have any knowledge of why to use a static array instead of a database table? When/why?
The problem with hardcoding values into your code is that compared with a database change, code changes are much more expensive:
Usually need to create a new package to deploy. That package would need to be regression tested, to verify that no bugs have been introduced. Hint: even if you only change one line of code, regression tests are necessary to verify that nothing went wrong in the build process (e.g. a library isn't correctly packaged causing a module to fail).
Updating code can mean downtime, which also increases risk because what if the update fails, there always is a risk of this
In an enterprise environment it is usually a lot quicker to get DB updates approved than code change.
All that costs time/effort/money. Note, in my opinion holding reference data or static data in a database does not mean a hit on performance, because the data can always be cached.
Your static array is an example of 'hard-coding' your data into your program, which is fine if you never ever want to change it.
In my experience, for your use case, this is not ever going to be true, and hard-coding your data into your source will result in you being constantly asked to update those things you assume will never change.
Protip: to a project manager and/or client, nothing is immutable.
I think this just boils down to how you think the database will be used in the future. If you leave the data in arrays, and then later want to create another application that interacts with this database, you will start to have to maintain the roles/resources data in both code bases. But, if you put the roles/resources into the database, the database will be the one authority on them.
I would recommend putting them in the database. You could read the tables into arrays at startup, and you'll have the same performance benefits and the flexibility to have other applications able to get this information.
Also, when/if you get to writing a user management system, it is easier to display the roles/resources of a user by joining the tables than it is to get back the roles/resources IDs and have to look up the pretty names in your arrays.
Using static arrays you get performance, considering that you do not need to access the database all the time, but safety is more important than performance, so I suggest you do the control of permissions in the database.
Study on RBAC.
Things considered static should be coded static. That is if you really consider them static.
But I suggest using class constants instead of static array values.
Assumption
I understand that it's not good to store to much data and it is needed to be as simple.
State today
Now I use as minimum needed and using simple data types (int and strings)
mainly for storing user's id and to tell if he is logged in.
must of my functions are static or singleton that has to be built each post/get.
I have trouble to representing the current state and changing it.
and get a largely static site.
most of state representing goes into javascript .
Target
for the other hand if I'll create a object that represent the entire website it will be much easier for me to maintain user's input , including database interaction.
simple question, how much data should be stored there?
example
One of the things i want to implement is
objects that relate to Database tables,
Let's take a page for a "car.update()".
Now if i store an object for it, that extends a connection to the Database with methods
for CRUD.
When I handle a post back from that page with details i could just put them in properties needed and call the update method.
situation now: I need to create a new object with that details and make an static update
Another example
storing previous search result and filter it using new data
In many cases the ideal amount would be none. Store the username in a cookie along with an HMAC hash used to verify the cookie was created by your site, and get everything else from the database (or cache) as needed. This makes it easy to load balance across servers because any server can handle any request and there's no state that needs to be shared between them.
This approach wouldn't be appropriate for banking or other top-security uses because if someone gets your cookie they connect as you. But for sites where you're not doing anything super critical it's great. The risk can also be mitigated somewhat by adding an expiration mechanism to your cookie handling. See chubbards great answer related to another HMAC question for more info.
note you can switch the way PHP stores data using session_set_save_handler. Then you don't have to change the calls and you improve performances/maintenance with the efficiency of database.
The minimum would be the user I.D.—assuming it is a logging in type of interface. But it is often helpful to include the most common aspects of that, like the user's permission and other items which are stored in the database, but are frequently referenced when constructing pages.
You shouldn't store an enormous amount of data, but you can without problems store some user-information if it helps you server you pages faster.
But if you want to build a more dynamic website, you will probably retreive more and more data from the database. So when you're connecting to a database after all, you could skip storing all kinds of information in the session, because you can just as well get them from the database. Most databases (including MySQL) have a quite efficient query cache that will make repeated queries lightning fast.
So in that case you'll need to save little more than the userid and maybe a small amount of flags.
I want to add some static information associated with string keys to all of my pages. The individual PHP pages use some of that information filtered by a query string. Which is the better approach to add this information? Generate a 100K (or larger if more info is needed later) PHP file with an associated array or add an other DB table with this info and query that?
The first solution involves loading the 100K file every time even if I use only some of the information on the current page. The second on the other hand adds an extra database call to the rendering of every page.
Which is the less costly if there are a large number of pages? Loading a PHP file or making an extra db call?
Unless it is shown to really be a bottleneck (be it including the php file or querying the database), you should choose the option that is best maintainable.
My guess is that it is the second option. Store it in a database.
Storing it in a database is a much better plan. With the database you can provide better data constraints, more easily cross reference with other data and create strong relationships. You may or may not need that at this time, but it's a much more flexible solution in the end.
What is the data used for? I'm wondering if the data you need could be stored in a session variable/cookie once it is pulled from the database which would allow you to not query the db on the rendering of every page.
If you were to leverage a PHP file then utilizing APC or some other opcode cache will mitigate performance concerns as your PHP files will only be loaded each time the file changes.
However, as others have noted, a database is the best place to store this stuff as it is much easier to maintain (this should be your priority to begin with).
Having ensured ease of maintenance and a working application, should you require a performance boost then generally accepted practice would be to cache this static data in an in-memory key/value store such as memcached. This will give you rapid access to your static values (for most requests).
I wouldn't call this information "static".
To me, it's just a routine call to get dome information from the database, among other calls being made to assemble whole page. What I am missing?
And I do agree with Dennis, all optimizations should be based on real needs and profiling. Otherwise it's effect could be opposite.
If you want to utilize some caching, consider to implement Conditional GET for the whole page.