Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
The Problem
I have an app that scrapes data and presents it to the user, directly, because of lack of disk space.
This data is very volatile, it can change within minutes. Much like the stock market.
Since the data changes so often, and it varies from user to user, it is useless to save it in a database.
The question
I need to sort the data presented to the user, compare it, link it etc. A lot of functions that a database provides. Yet I cannot save it in said database because of the above conondrums, what should I do?
What I've Thought of Doing So Far
I've tried organizing the data presented to each user using just PHP but seems troublesome, fragile and inefficient.
Should I just create some sort of virtual table system in MySQL just for data handling? Maybe use a good database engine for that purpose?
Maybe I can save all data for each user but have a cron job remove the old data in the database in a constant fashion? Seems troublesome.
The Answer
I'd like some implementation ideas from folks who have encountered a similar problem. I do not care for "try all of the above and see what is faster" type of answers.
Thanks all for your help.
If the data is of the type you would store in a db and you would benefit from being able to query it in ways that are more difficult in PHP, but you just don't want to keep it, you can still use a database. You can create temporary tables, insert raw data, and query it to get what you want. When you close the db connection, the tables disappear. Even though the script names them the same, the database will actually create a unique set per connection so each user will have unique data. This solution may not perform as well as you need so do some testing to see if it's suitable for your situation.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I need to store one global decimal value.
Would the best way to approach this be to store the value in a .txt file, or make an entire table for it in MySQL which will contain a single row for the value, or is there a different way?
It deppends on how often you will access it and how performance critical your application is. In brief DataBase should be your default unless you have a strong reason not to.
The reasons to keep the data in the database it's because it makes everything simplier and neater, backups don't need to worry about random .txt files that may be overlooked (And this a incomplete backup). If you need to set up a cluster it's a real PITA to keep files real-time synched while most databases support it easily, etc.
Why wouldn't you keep something in the database?
It's a enviroment value and thus in each installation of your
software the value may be different while using the same DB (In this
case people use File storage like the usual config .yml)
It's really performance critical to
have a top notch access-time to this data, for this people use
specialized engines to store such data (Redis, memcached, etc)
Now, answering your question, how to store it in your own db, that's up to you, but I would think something like a table named "GlobalConfigurations" with a column key and value could be a good approach. So that if you find any other variable with the same behavior you can put it there.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
Skip that if you value your time.
I am making a website that has a special "control panel" for a few (less than 20) people. It is a calendar where you can see which days aren't occupied yet and book "events" to them if it's needed. Panel uses php and mysql database to hold usernames, events etc. It is pretty much finished, but i noticed that i need to check if exact same data isn't stored in database already (it wouldn't cause problems, but it would be better to avoid redundancy).
First solution i came up with is just add a query (checking if there is an appointment in the time) to every added row (they are added in a loop) and do the INSERT only if checking query returned 0 rows.
You can really ignore anything higher than that, i added explanation only to clarify situation more.
My question is: Is it really bad for the server if there are many small queries run in a loop, or that doesn't really matter because queries are not that hard for a server? Maybe it really matters and i should for example run one, bigger query and save it into an array or something?
It probably doesn't matter but i am using object oriented setup from there
I am asking mostly to expand my knowledge, because this utility will be used only by one person anyways, so server will keep up even if it is not really efficient.
This is too long for a comment.
Is it a really bad idea? Depends on your definition of bad. In general, multiple such queries are more expensive. And multiple queries introduce more opportunities for race conditions. Doesn't sound like a "good" idea to me.
More importantly, though, is checking for duplicates in the application. You have a database and can use the database to prevent duplicates. This is called a "unique" constraint or a unique index (the former is implemented using the latter). Your question doesn't have enough information to make a concrete suggestion, but that sounds like what you really need.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I'm programming a LAMP stack online application that uses a very complex search form, I want to give the user the ability to name and save their current search for faster use in the future (they will be checking these results daily). What is the best methodology to do this? I've been coming across stored procedures, but this doesn't seem like what I'm looking for.
My current idea:
Pull php generated query into dedicated database for saving queries (all form input data is sanitized / validated). Is this a security risk? I know all form generated SQL is a risk of course. When the user wants to query with it, the PHP code will simply use the saved query over the form generated one. If I change the form generated query code in the future, this should prevent conflicts, but of course, it won't take advantage of any new design features.
I don't imagine this is "best practice" (or that there is one, in this case). Personally I think I'd rather store their search terms in a format devoid of context (say in a JSON-encoded object if there are multiple search terms or conditions), and then when they recall the search rebuild the queries from the JSON object.
(Storing the actual query seems to run the risk of old queries becoming obsolete if/when the underlying database structure changes. Storing only what they're searching for and rebuilding it allows you to accommodate for that.)
My $0.02.
To answer your question, yes. Regardless of how you store it, you would store the values they entered in whatever your form collects, then when they rerun the stored "search" you would go through that structure and remake your query.
The table might have search_id, user_id, search_name, parameters and whatever else. They pull up a list of their saved searches, choose one, execute it, you pull parameters, rebuild the query, run it, and display the results, just as you would when they did the original search through the normal form.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I am working on a project that I need to let users to create pages on my server. however, I do not want to let users clutter my mysql database by storing the stuff in there so I cannot use mysql database for creating the pages.
I did research this topic and there seem to be a some sort of a plugin for WP that will allow virtual page creation.
is this possible using pure php WITHOUT the use of any database ?
It's possible, but wrong.
You can use php to write a html file to your web directory, sure. But that
solution is in no way cleaner or less cluttered than putting stuff in your
database, for a few reasons:
It's easier to have structured information in the database
It's a good thing conceptually to separate user data from your program
It's easier to control access to your database in a safe way, compared
to writing user data to the file system
"I really do not want to use mysql database" is not a good reason to give this
up. You might have a good reason, but it's not easy to guess what that is, which
makes suggesting alternatives very difficult.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I am new to making games, and I am writing one for fun using php and javascript. I am using MySQL to store all of the info of the users of the game. The game is kind of like managing a sports team. You have a few variables (cash, assets, players, staff, etc...) and you take on all the roles of a sports manager. I know it exists, this is just a personal challenge.
My question is, what is the best and most efficient way to get information from the database into the game?
1. Do I have to run an sql query on every page?
2. Do I have to update my database EVERY single time something is updated?
3. Is it possible to get all of the information from the database when the user logs in, let him/her play, then only update the database with the new information when the session is killed?
Sorry for the lack of code, just looking for a starting point because it would be helpful to me to know this before I start writing a lot of the game.
Thanks
No, you don't necessarily have to run a MySQL query on every page load. You could store the results of such queries in a cache system such as memcached, or keep necessary data in $_SESSION.
No, you can use similar workarounds as before, but if the user disconnects you may end up with unsaved changes.
Well, you could load the data relevant to the user and write your own session handler for saving the data when the session is destroyed, but although I haven't ever tried it I would say there's a very real risk of losing data if, for example, your server is restarted or PHP's garbage collector callback is not called for some reason.
Overall, I think you may perceive SQL queries as much heavier than they actually are. If your database structure and indexes are set up correctly, your queries and updates shouldn't take longer than about 0.01 seconds each to complete.