I am trying to create a world application using jQuery (JS) and PHP. I originally tried doing this by using a MySQL database, which didn't work well - the server got overloaded with database queries and crashed.
This time I want to store the data in a text file... maybe use JSON to parse it? How would I do this? The three main things I want are:
Name
x-position
y-position
The x and y positions are given from the JS. So, in order:
User loads page and picks username
User moves character, the jQuery gets the x and y position
The username, x and y position are sent to a PHP page in realtime using jQuery's $.post()
The PHP page has to find some way to store it efficiently without crashing the database.
The PHP page sends back ALL online users' names and x and y coordinates to jQuery
jQuery moves the character; everyone sees the animation.
Storing the data in the file instead of the MySQL database isn't an option if you want to improve performance. Just because MySQL stores its data in the files too, but is use some technics to improve performance like caching and using indexes.
The fastest method to save and retrieve data on server is using RAM as a storage. Redis for example do that. It stores all the data in the RAM and can backup it to the hard drive to prevent data loss.
However I don't think the main problem here is MySQL itself. Probably you use it in an inappropriate way. But I can't say exactly since I don't know how many read and write requests your users generate, what the structure of your tables etc.
Text files are not the best performing things on Earth. Use a key-value store like Redis (it has a PHP client) to store them. It should be able to take a lot more beating than the MySQL server.
You can store the data in a text file in CSV (Comma separated values) format.
For example, consider your requirements.
1,Alice,23,35
2,Bob,44,63
3,Clan,435,322
This text file can be stored and read anytime, and use explode function to separate values.
Related
Its known fact that we can use Json/XML parsing or Database to maintain a limited amount of data consistent on a given application and Website at any given time.
However, the dilemma is a project having few display textlines 3k-4k that are supposed to be consistent on both(App and website)on a selected UI, these text files may change at any given point of time. What will be the optimized method or steps of implementation for this technique?
Just my bit would be to store the data in on server as a text file and later use the website or the app to parse the given text file at given location to display it
I have a JSON file which displays data from my MySQL database.
I have an app that fetches and stores that data in the app on the first launch. It's stored in CoreData so the user doesn't have to be connected to internet to be able to use the app.
I'd like to update the data if anything has changed in the database. The first method I've been using is storing the total length of the JSON data fetched and then comparing it, but I was wondering if there was a better method to do this.
Could I fetch the modification date of the JSON file? If so, how?
Thanks
Storing the length will still leave you with a place for error, and the data could change but keep the same length, what can be more efficient is to store the checksum of the file in your app and whenever it checks for updates from the server it will compare it with the newly generated file's checksum, if the checksum matches then there is no changes. you can use md5_file for this.
You can also cut bandwidth, please check ETag that could save you some time. http://bitworking.org/news/ETags__This_stuff_matters
I've currently got a database with just short of 2000 client locations in Australia. What I am trying to do is to display this data on a heatmap, to be embedded into an existing website.
I've done a heap of looking around, and can't seem to find exactly what I'm after.
http://www.heatmapapi.com/sample_googlev3.aspx
http://www.heatmaptool.com/documentation.php
These are along the right lines of what I want to achieve, however I cannot see these working with data from a mysql database (require the data to be hard-coded, or uploaded through CSV files).
Has anyone come across this sort of thing before, or managed to achieve it?
Both of the examples you provide would potentially work.
With the first you would need to use the data you have to dynamically generate the javascript, or at least the values that go into the javascript.
The second is probably the better option. You would provide a path to the script that would dynamically generate a CSV file.
I'm creating my first web application - a really simplistic online text editor.
What I need to do is find the best way to store text based files - a lot of them.
These text files can be past 10,000 words in size (text words not computer words.) in essence I want the text documents to be limitless in size.
I was thinking about storing the text files in my MySQL database - but thought there was a better way.
Instead I'm planing on storing the text files in XML based format in a directory on my server.
The rows in the database define the name of the xml based text file and the user who created the text along with basic metadata.
An ID is generated using a V4 GUID generator , which gives the text an id and stores the text in the "/store" directory on my server. The text definitions in my server contain this id, and the android app I'm developing gets the contents of the text file by retrieving the text definition and then downloading the text to the local device using the GUID in the text definition.
I just think this is a botch job? how can I improve this system?
There has been cases of GUID colliding.
I don't want this to happen. A "slim" possibility isn't good enough - I need to make sure there is absolutely no chance in a GUID collision.
I was planning on checking the database for texts that have the same id before storing the text with a particular id - I however believe with over 20,000 pieces of text in my database this would take an long time and produce unneeded stress on the server.
How can I make GUID safe?
What happens when a GUID collides?
The server backend is going to be written in PHP.
You've got several questions here, so I'll try to answer them all.
Is XML with GUID the best way to do this?
"Best" is usually subjective. This is certainly one way to do it, but you're probably adding unneeded overhead. If it's just text you're storing, why not put it in the SQL with varchar(MAX)?
Are GUID collisions possible?
Yes, but the chance of that happening is small. Ridiculously small. There are much bigger things to worry about.
How can I make GUIDs safe?
Stop worrying about them.
What happens when a GUID collides?
This depends on how you're using them. In this case, the old data stored in the location indicated by the GUID would probably be overwritten by the new data.
Well i dont know if id use a guid i would probably just use the auto_increment key on the db table and name the files like that because unless you have deleted records from the db without cleaning up the filesystem they will always be unique. I dont know if the GUID is a requirement on the android side though.
There's nothing wrong with using MySQL to store the documents!
What is storing them in XML going to provide you with? Adding an additional format layer will only increase the processing time when they are to be read and formatted.
Placing them as files on disk would be no different than storing them in an RDBMS and in the longer-term probably cause you further issues down the line. (File access, disk-seek, locking, race conditions come to mind).
I'm making an android application which takes a photo and push the image (as a base64 encoded string) to a PHP script, from here I'll be storing data about the image inside a MySQL database.
Would it be wise to store the image inside the database (since it's passed as a base64 string), would it be better to convert it back to an image and store it on the filesystem?
A base64 encoded image takes too much place (about 33% more than the binary equivalent).
MySQL offers binary formats (BLOB, MEDIUM_BLOB), use them.
Alternatively, most people prefer to store in the DB only a key to a file that the filesystem will store more efficiently, especially if it's a big image. That's the solution I prefer for the long term. I usually use a SHA1 hash of the file content to form the path to the file, so that I have no double storage and that it's easy to retrieve the record from the file if I want to (I use a three level file tree, first two levels being made respectively from the first two characters and the characters 3 and 4 of the hash so that I don't have too many direct child of a directory). Note that this is for example the logic of the git storage.
The advantage of storing them in the DB is that you'll manage more easily the backups, especially as long as your project is small. The database will offer you a cache, but your server and the client too, it's hard to decide a priori which will be fastest and the difference won't be big (I suppose you don't make too many concurrent write).
I've done it both ways, and every time I come back to code where I stored binary data in a MySQL table I always switch it to filesystem with a pointer in the MySQL table.
When it comes to performance, you're going to be much better off going to the FS as pulling multiple large BLOBs from a MySQL server will tend to saturate its pipe quickly. Usually it's a pipe you don't want clogged.
You could always save the base64_encode($image) in a file and only store the file path in the database, then use fopen() to get the encoded image.
My apologies if I didn't understand the question correctly.
"wise" is pretty subjective, I think. I think it would be wise from a "keep people from directly linking to my images" perspective. Also, it may be helpful as far as if you decide you need to change up dir structures etc.. it might make it easier on you (but this really depends on how you wrote your scripts to begin with..) but other than that... offhand I can't really think of any benefits to doing this.