There have been many questions along these lines but I'm struggling to apply them to my scenario. Any help would be be greatly appreciated!
We currently have a functioning mySQL database hosted on a website, data is entered from a website and via PHP it is put into the database.
At the same time we want to now create a python application that works offline. It should carry out all the same functions as the web version and run totally locally, this means it needs a copy of the entire database to run locally and when changes are made to such local database they are synced next time there is an internet connection available.
First off I have no idea what the best method would be to run such a database offline. I was considering just setting up a localhost, however this needs to be distributable to many machines. Hence setting up a localhost via an installer of some sort may be impractical no?
Secondly synchronization? Not a clue on how to go about this!
Any help would be very very very appreciated.
Thank you!
For binding Python to MySql you could use HTSQL:
http://htsql.org
You can then also query your MySQL DB via http requests, either from AJAX calls or server-side e.g. cURL (and of course still have the option of writing standard SQL queries).
There is a JQuery plugin called HTRAF that handles the client side AJAX calls to the HTSQL server.
The HTSQL server runs on localhost as well.
What OS would you be using?
How high-performance does your local application need to be? Also, how reliable is the locally available internet connection? If you don't need extremely high performance, why not just leave the data in the remote MySQL server?
If you're sure you need access to local data I'd look at MySQL's built-in replication for synchronization. It's really simple to setup/use and you could use it to maintain a local read-only copy of the remote database for quick data access. You'd simply build into your application the ability to perform write queries on the remote server and do read queries against the local DB. The lag time between the two servers is generally very low ... like on the order of milliseconds ... but you do still have to contend with network congestion preventing a local slave database from being perfectly in-sync with the master instantaneously.
As for the python side of things, google mysql-python because you'll need a python mysql binding to work with a MySQL database. Finally, I'd highly recommend SQLalchemy as an ORM with python because it'll make your life a heck of a lot easier.
I would say an ideal solution, however, would be to set up a remote REST API web service and use that in place of directly accessing the database. Of course, you may not have the in-house capabilities, the time or the inclination to do that ... which is also okay :)
Are you planning to run mysql on your local python offline apps ? I would suggest something like sqlite. As for keeping things in sync, it also depends on the type of data that needs to be synchronized. One question that needs to be answered:
Are the data generated by these python apps something that is opague ? If yes (i.e. it doesn't have any relations to other entities), then you can queue the data locally and push it up to the centrally hosted website.
Related
I'm not a back-end developer and I don't know much about devops, but I will try.
I've been working with Docker containers as a front-end developer with many applications, mostly PHP-based, Laravel, Phalcon, Symphony, you name it.
One things that has been bugging me for years now is the performance of local environment, especially when it comes to (any?) databases - it doesn't matter if I'm using remote MySQL database on a server or setup an MySQL container on localhost - requests are always so super slow compared to production, in worst cases reloading some pages took tens of seconds if not minutes.
I'm wondering whether is it possible for a simple front-end dev to setup mocked MySQL database somehow, like an object that will pretend to be my MySQL database with all the structure and data but much faster? 99% of the time I do not want to update records in my bases anyway, so something read-only would be totally sufficient. Of course I want this to work as a normal db, it means it can receive and send data.
I won't lie I had the same problem with almost every single startup I worked on (excepting... WordPress :P), not only in PHP but also in other languages, I'm using top-tier MacBook so it's performance is still far away from real web servers but I think it should be sufficient.
Thanks for any hints :)
I'm making a program which will be always gathering some data and putting them into a MySQL database, now i have two way to accomplish that and i wanted to knwo which one is better:
1- As i'm using Qt i can use QtSql module to connect directly to the DB and insert data
2- I can aslo write a PHP script using GET or POST varibles to insert data into the DB and just call the URL from my program with the suitable data.
I'm most worried about performance, as there will a lot of insertion all the time (about 100inserts / second). But the data size of each insertion is not that big, it will not exceed 10 charcaters.
I would point out that the Web server(where the php script will be stored), the DB server and the Server where the program will be running are all on the same local network
It depends on your environment. Is the MySQL DB running on your local PC? Then, QtSql should be perfectly fine.
If the DB is publicly accessible (eg. via the internet), then this would be a bad idea. You generally should avoid exposing MySQL DB servers directly to public network and instead provide access e.g. via locked-down interfaces such as your PHP one.
edit: From a pure performance-centric point of view, the QtSql solution should definitely be faster (though the only way to be sure is to benchmark it!). The PHP solution would have at least the following elements that generate overhead:
Data transfer over a network socket
Startup of the PHP interpreter (is it still the case that PHP is started once for each request? If so, this is going to be significant.)
This is not a codefix question but please help me where possible.
I am developing an application which needs to store information in a database. This information needs to be stored off-device for security reasons (patient data).
I have explored using PHP as a bridge to the external MySQL database hosted locally via WAMP, however I have recently been informed of servlets and also of SQLite.
As I am learning these technologies under a limited time frame I need to know which to invest my time into to get the job done as easily as possible. I have no experience with any query language but I did get a simple login screen to work on Android using PHP and MySQL on WAMP using HTTP post/fetch within android.
Also, is it possible to store the information in SQLite within android and write that data to a server, which can then load the SQLite database again on re-launch?
Many thanks for your time!
From our expericence these kind of services need to work offline if possible. Both 3g and especially wifi has been unreliable in several occasions.
You do good if you save the data locally in e.g. sqlite database but be warned the data can get wiped on OS update or software update.
Also try to write your changes/additions in batches and servr updater can send all the batches that are not yet on the server. this way it's more reliable if you miss updates.
These kind of interfaces to another systems take a lot of time in testing. Consider getting a new timeframe in advance.
You will find more about Android connection to MySQL here
I want to create a live, checkers-like app, which will work like this: There will be multiple icons/avatars displayed on this checkerboard like surface. I want to have a command prompt beneath this board, or some other sort of interface, that will allow them to control a certain avatar, and get it to preform actions. Multiple users will be using it at one time, and I will all be able to view the other user's changes/actions to the checkerboard.
What I'm wondering is: what's the best way to do this? I've got my HTML, CSS, and JS approach down, but not my data storage method. I know that, using PHP, I've got the choices to use either: file-based storage, MYSQL, or some other method. I need to know which is better, because I don't want to have server-lag, poor-response time, or some other issue, especially in this case since actions will be preformed every other second 2 or so, by these multiple users.
I've done similar stuff before, but I'm wanting to hear how others would handle it (advice, etc.) from more experienced programmers.
Sounds like a great project for node.js!
To clarify, node.js is a server-side implementation of javascript. What you'll want is a comet based application (a web-based client application that receives server side pushes instead of the client constantly polling the server), which is exactly what node.js is good at.
Traditional ajax calls for your clients to poll the server for data. This creates enormous overhead for both the client and the server. Allowing the server to push requests directly to the client without the client repeatedly asking solves the overhead issue and creates a more responsive interface. This is accomplished by holding asynchronous client connections on the server and only returning when the server has something to respond with. Once the server responds with data, another connection is immediately created and held by the server again until data is ready to be sent.
You may be able to accomplish the same thing with PHP, but I'm not that familiar with PHP and Comet type applications.
Number of users and hosting costs will play into your file vs DB options. If you're planning on more than a couple of users, I'd stick to the database. There are some NoSQL options available out there, but in my experience MySQL is much faster and more reliable than those options.
Good luck with your project!
http://en.wikipedia.org/wiki/Comet_%28programming%29
http://www.nodejs.org/
http://zenmachine.wordpress.com/2010/01/31/node-js-and-comet/
http://socket.io/ - abstracts away the communication layer with your clients based on their capability (LongPolling, WebSockets, etc.)
MySQL and XCache !!!!
Make sure you use predefined statements so MySQL does not need to compile the SQL again. Also memtables could be used to use memory storage
Of course make use of indexes appropriately.
If the 'gamestate' is not that important you can even store everything in XCache.
Remember that XCache does not store data persistently (after Apache restart)
i have a database that is written in access. the access mdb file connects via ODBC to a local mysql database. i have a bunch of sql and vba code in the access file. i dont expect the database to surpass 100mb. currently it is around 10mb. i will need to have multiple user access. (no more than 10 users at a time)
i need to convert this database from being a local one to a web server, and i need to make a web interface for it.
how do i get the current local instance of mysql database to run off a webserver? i am currently running it off wampserver 2.0. i dont have experience putting a database on a webserver.
i have an OK vb.net background. i have never done any web applications. here's a picture of the access form that i may need to replicate to work off a website:
alt text http://img42.imageshack.us/img42/1025/83882488.jpg
which platform should i use as the front end to this thing?
would it be possible to just run this access file off a webserver instead of programming a new front end for it? is that not a smart idea?
thank you for your help!
If your webserver has TCP connectivity to your existing database server, and its hosted in a suitable place (eg, don't have your webserver in a datacenter connecting to a database server on your office DSL connection), then no move is required.
If you do need to move it, it's as easy as creating a backup/dump, and restoring it elsewhere.
As far as the frontend, there are MANY technologies that will do what you need (ASP.NET, PHP, Python, Ruby, Perl, Java being the most popular ones, not necessarily in that order).
Use something you are comfortable with, or that you are interested in learning (provided you have the time to do so)
Use something that runs properly on your target webserver. Really, ASP.NET is the only one that has any major issue here, as it's limited to Windows.
Access itself has no direct web-accessible version. A Google search finds some apps that claim to convert Access forms to web-based, but I will not link to any because I don't know how well they work. I'm certainly leary of anything like that, because web apps are a different breed from Windows apps. If you are going to go that route, be sure they actually generate HTML output; make sane, clean source; and offer a free trial so you can verify it actually works.
Really though, a form like that is reasonably easy to reproduce with some basic knowledge of server-side programming and some HTML.
I don't have any experience migrating access to a web-based interface, although I have heard of people going straight from access to a web page. MySql is exceptionally easy to migrate. MySQL.com has a program called mysqldump that comes with the standard install of MySQL that allows you to export your database straight to a text file that can be used then with mysqldump to import it on another server. I don't believe the WAMP server comes with the command line tools although they can be downloaded from mysql.com. However, if it has phpMyAdmin, then there is also an export feature with that as well that will generate a .sql file that can be imported to the webserver using phpMyAdmin. One thing to keep in mind though is that I have had very little success mixing and matching these methods: ie, I've never been able to get a mysqldump-created file to work with phpMyAdmin and vice versa.
Good luck!
The link will help you to export and import mySQL database
May be on Windows web server there is an opportunity to run Access files, you can check, but any way if you have some programming skills, I would say that it is not difficult to crate a php script which will query your database info and will edit.
Migrating an Access application to the web is quite difficult, because you can't translate an Access form 1:1 into a web page. Web apps are stateless, whereas Access is built around the concept of bound controls and bound datasets.
Secondly, it is impossible to easily replicate an Access subform.
Third, you lose tons of events that Access forms and controls are built around.
In general, a web page that performs the same task as an Access form will bear little or no resemblance to the Access form, simply because the methods for accomplishing the same tasks and the UI widgets available to you are so completely different.
One thing to consider is whether your users need a web application or if they just need to use your existing Access application over the Internet. If the latter is the case, Windows Terminal Server/Citrix can do the job for a lot less money, since there's no conversion needed. You do need to provision a Windows Terminal Server, set up a VPN and purchase CALs for the users, but the costs of those are going to be much less than the cost of rebuilding the app for web deployment.
It may not be an appropriate solution, but it's one that you should consider, I think.