I never made something similar .
I have a system and i need to relate my data with external data (in another database).
My preference is get these data and create my own tables, but in this case when the other dbs are updated my personal tables will be obsolete.
So, basically i need to synchronize my tables with external tables, or just get the external data values.
I don't have any idea how i can connect and relate data from ten external databases.
I need to check if an user is registered in another websites basically.
Any help?
I am crrently doing something similar.
Easiset way I found is to pull the data in, though I do bi-directional syncronisation in my project you haven't mentionned this so I imagine it's a data pull you are aiming for .
You need to have user accounts on the other servers, and the account needs to be created with an ip instead of 'localhost'. You will connect from your end through mysql client using the ip of distant host instead of the ususal localhost.
see this page for a bit more info.
If, like me you have to interface to different db server types, I recommend using a database abstraction library to ease the managing of data in a seamless way across different sql servers. I chose Zend_db components, used standaline with Zend_config as they Support MySQL and MSSQL.
UPDATE - Using a proxy DB to access mission critical data
Depending on the scope of your project, if the data is not accessible straight from remote database, there are different possibilities. To answer your comment I will tell you how we resolved the same issues on the current project I am tied to. The client has a big MSSQL database that is is business critical application, accounting, invoicing, inventory, everything is handled by one big app tied to MSSQL. My mandate is to install a CRM , and synchronise the customers of his MSSQL mission-critical-app into the CRM, running on MySQL by the way.
I did not want to access this data straight from my CRM, this CRM should not ever touch their main MSSQL DB, I certainly am not willing to take the responsibility of something ever going wrong down the line, even though in theory this should not happen, in practice theory is often worthless. The recommandation I gave (and was implemented) was to setup a proxy database, on their end. That database located on the same MSSQL instance has a task that copies the data in a second database, nightly. This one, I am free to access remotely. A user was created on MSSQL with just access to the proxy, and connection accepted just from one ip.
My scipt has to sync both ways so in my case I do a nightly 'push' the modified records from MSSQL to the crm and 'pull' the added CRM records in the proxy DB. The intern gets notified by email of new record in proxy to update to their MSSQL app. Hope this was clear enough I realize it's hard to convey clearly in a few lines. If you have other questions feel free to ask.
Good-luck!
You have to download the backup (gzip,zip) of the wanted part(or all) of the Database and upload it to the another Database.
Btw. cronjobs wont help you at this point, because you cant have an access to any DB from outside.
Does the other website have an API for accessing such information? Are they capable of constructing one? If so, that would be the best way.
Otherwise, I presume your way of getting data from their database is by directly querying it. That can work to, just make a mysql_connect to their location and query it just like it was your own database. Note: their db will have to be setup to work with outside connections for this method to work.
Related
I have to develop a Project with php mysql for sales management system.There are many
outlet. I want to keep a databse centrally and every outlet have a databse locally. user
entry data to local databse. after a while local data can be uploaded to central databse.
Local data will go to central database, but central data will not go to local database.
what will be the procedure for that. (e.g: Synchronization, Replication)
I wouldn't use syncronisation or replication. I would use an import/export mechanism.
Write a little tool which will export the last day/week/month and than send it with an secure line to your main database for import.
Depending on the specs of your project (size of data, longevity of data, frequency of sync, etc.) you might have to implement a one-way synchronization. E.g. your clients will upload data incrementally where only new changes (no need to re-send all information on each sync) are uploaded to the server.
You can achieve this in various ways. The simple way is uploading your data to the server and removing them from the local storage. If your clients need to keep the uploaded data then introduce the additional field “Dirty” in your tables on the client side and use it for designating the new changes.
Recently I blogged about bi-directional sync algorithm, which includes the upload changes functionality using dirty field, which might be helpful to you.
Maybe SymmetricDS (http://www.symmetricds.org) can solve your problem. We're having a similar problem and we've decided to use it.
My client has an offline product database for a high street shop that they update fairly frequently for their own purposes. They are now creating an online store which they want to use product information from this database.
Migrating the database to a hosted server and abandoning the offline database is not an option due to their current legacy software set up.
So my question is: how can I get the information from their offline database to an online database? Their local server is always connected to the internet so is it possible to create a script on the website that somehow grabs the data from their server and imports it into the online server? If this ran every 24 hours it would be perfect. But is it even possible? And if so how would I do it?
The only other option I can think of is to manually upload the database after every update, but this isn't really a viable idea.
I did something like this with quickbooks using an odbc connection. Using that I synced data to MySQL. This synchronization however, was just one way. Unless you have keys in the data that indicates when something was changed (updated date), you will end up syncing alot of extra data.
Using SQLYog, i set up a scheduled job that connected to the odbc data source, and pushed the changes since last sync to the mysql database I was using to generate reports. If you can get the data replicated into MySQL it should be easy at that point to make use of it in your online store.
The downside is that it wont be realtime. Inventory could become a problem.
In an ideal world I would look at creating a restful API that would run on the same server or at least run on the same network as your offline database. This restful API would run as a web server via http and return JSON or even XML structures of data from the offline database. Clients running on the internet would be able to connect and fetch any data they need, at any time. A restful API like this has a number of advantages.
Firstly it's secure. You don't have to open up an attack vector to the public by making connections to your offline database public. The only thing you have to do is enable public access to your restful API. In your API's logic you might not even include functionality to write to the database so even if your API's security is compromised at very worst all attackers can do is read your data, not corrupt it.
Having a restful api in this situation represents a good separation of concerns. Your client code should not know anything about the database nor should it know about any internal systems that the offline database uses. What happens when your clients want to update their offline system or even change it? In this situation all you would have to do is update the restful api. Your client that is connecting to the data no longer cares about anything else but the api so changing databases would be easy.
Another reason to consider an API is concurrency. I hinted at this before but having an API would be great if you ever need to have more than one client accessing the offline databases' data. In a web server set up where you would have the API sitting and waiting for requests there is no reason why you could not have more than one client connecting to the api at the same time. HTTP is really good at this!
You talked about having to place old data in a new database. Something like this could be done easily with a restful API as you would just have to map the endpoints of your API to tables in the new database and run that when you need. You could even forgo the new database and use the API as your backend. This solution would require some caching but it would cut down on the duplication of a database if you don't feel it's needed.
The draw back to all of this is the fact that writing an API over a script is more complex. So in this situation I believe in horses for courses. If this database is the backbone of a long term project that will be expanding in the future an API is the way to go. If its a small part of your project then maybe you can swing it with a script that runs every 24 hours however I have done this before and the second I have to change/edit the solution things start getting a little "hairy". Hope this helps and good luck with it.
I have a web applications that stores data in a MySQL database on-line. It also retrieves data using PHP code, performs calculations on the server and sends the result back to the user.
Data it's quite simple: names, descriptions, prices, VAT, hourly charges that are read from the database and manipulated on the server side.
Often client work in environments where the internet connection is poor or not available. In this case I would like the client to be able to work offline: enter new names, descriptions, prices and use the last VAT to perform calculations. Then synchronise all data as soon as a connection is available.
Now the problem is that I do not know what is the best way or technologies for achieving this. Don't worry, I am not asking to write code for me. Can you just explain to me what is the correct way to build such a system?
Is there a simple way to use my online MySQL and PHP code locally?
Should I save the data I need in a local file, rebuild the calculation in JavaScript, perform them locally and then synchronise the data if database is available.
Should I use two MySQL database, one local and one online and do a synchronisation between the two when data is available? If yes which technology (language) shall I use to perform this operation?
If possible, I would like an answer from PHP coders that worked on a similar project in the past and can give me detailed information on framework structure and technology to use. please remember that I am new to this way of writing application and I would appreciate if you can spare few minutes and explain everything to me like if I am six year old or stupid (which I am!)
I really appreciate any help and suggestion.
Ciao,
Donato
There are essentially 3 ways to go:
Version 1: "Old school": PHP-Gtk+ and bcompiler
first, if you not have done so already, you need to separate your business logic from your presentation layer (HTML, templating engines, ...) and database layer
then adapt your database layer, so that it can live with an alternative DB (local SQlite comes to mind) and perform synchronisation when online again
Finally use PHP-Gtk+ to create a new UI and pack all this with bcompiler
Version 2: "Standard": Take your server with you
Look at Server2Go, WampOnCD and friends to create a "double clickable webserver" (Start at Z-WAMP)
You still need to adapt your DB layer as in Version 1
Version 3: "Web 2.x": Move application from server to browser
Move your application logic from the server side (PHP) to the client side (JS)
Make your server part (PHP) only a data access or sync layer
Use the HTML5 offline features to replace your data access with local data if you are offline and to resync if online
Which one is best?
This depends on what you have and what you want. If most of your business logic is in PHP, then moving it into the browser might be prohibitingly expensive - be aware, that this also generates a whole new class of security nightmaares. I personally do not recommend porting this way, but I do recommend it for new apps, if the backing DB is not too big.
If you chose to keep your PHP business logic, then the desicion between 1 and 2 is often a quiestion of how much UI does your app have - if it's only a few CRUD forms, 1. might be a good idea - it is definitly the most portable (in the sense of taking it with you). If not, go with 2.
I have worked with similar system for ships. Internet is expensive in the middle of the ocean so they have local web servers installed with database synchronization via e-mail.
We also have created simple .exe packages so people with no experience can install the system or update system...
I have a web portal, consisting of various projects, based on MySQL database. The project administrators need to administrate their tables. I would like to provide them some already existing, free, simple interface with as cheap setup as possible (like 10-15 minutes max of my time per each new project - IMPORTANT! The number of project administrators and their requests grows...). It should support:
required: table export, import, insert new row, modify existing rows
not needed, but a plus: foreign keys replaced with some other field value from the foreign table (usualy a "dictionary"), display only certain columns, etc.
The problem is that it is a hosting environment, so I have no superuser permissions to the MySQL database. I don't have the grant permission so I must ask my hosting provider to run every grant command, so I want to minimize these requests as it is an above-standard service (it's their grace to me). And these requests would have to be done quite often.
What database administration tools/solutions can I use?
My ideas:
1) MySQL ODBC connector + MS Access as a client. The MS Access would connect via ODBC connector to the MySQL server. I can prepare small MS Access file that would contain a link to desired tables, and also quickly generated forms!
This is cool, however, I would need to contact my provider every time to create db user with desired permissions... to prevent users from changing table structure or destroying other tables...
2) Client -> Proxy -> MySQL server. Like in 1), but with some proxy. I'm now theorizing, but the Access could also use other protocol (e.g. HTTP) to connect some proxy that would handle the permissions and this proxy would then pass it to MySQL server. Does there exist something like that?
3) PHPMyADMIN. The problem from point 1) remains. However, the permission checking could be theoretically implemented on the PHP level here, so no need to change any MySQL permissions! Is PHPMyADMIN capable of that, out of the box? Can I simply configure a new user which can only see table A & B and only modify column C, etc.?
However, the import is not much user friendly (no XLS, only CSV, no delimiter autodetection etc.), as well as inserting new records...
4) There are plenty of modern web tools with spreadsheet-like look like GoogleDocs. Could these be used for the task. Again, in theory the permission checking could be done at the web-server (not database) layer... and set up easily... (?)
I'm sure many people had to solve the same issue, so I'm looking forward your experiences and ideas!
My final solution was a deal with a hosting provider - I asked him to create 5 dummy database users for future usage and also asked him to grant me the GRANT OPTION privilege. So I can configure privileges of those users, without having to ask the hosting provider! I did'n know of this possibility at the time of asking.
And then, I use MS Access with MySQL ODBC Connector as a front-end to the MySQL database. Great1!
I have two shared hosting accounts, each with a database of its own (its own cPanel login). Assuming these two databases have the same structure, what do I need to do in order to synchronize them?
The synchronization script would be on a domain connected with one of the hosting accounts. I know the MySQL/PHP for synchronizing databases that are on the same account is fairly simple, but what's confusing me here is how to access the database which is on different hosting?
This isn't a one-time thing, I need to be able to do this by clicking a button/link.
The only thing that comes to mind is having the remote database export everything to .csv files on a regular basis and have the script on the domain connected to the first database import everything, but there's gotta be a better way?
In case this whole question is confusing, the gist of the problem is - is there a way to have a script on a domain access a database on a completely different shared hosting account?
In short, no, there's no way.
Usually, hosting providers allow DB access only to localhost users. Meaning that script from another machine can't access it.
Also, what kind of synchronization is it? One-way or two-way? (but, I guess, this is out of scope here)
The only viable solution that comes to mind is some kind of dump/restore procedure.
Example:
webserver A (source of data) defines an URL, by requesting which you can get dumped content of the DB
webserver B (destination of data) defines a page with button 'Sync'.
upon clicking the 'Sync' button, server B will fetch that URL from server A, receive A's data and merge it with its own.
NOTE
It is important to secure data export URL. In that script you can check, for example, IP of incoming request, or presence and correctness of "access_token" or whatever you like.
Can you connect to the database via the SSL/SSH or PHP tunnel? If so, try Data Comparison tool in dbForge Studio MySQL.
Data Comparison tool will allow you to compare data between different databases. You may test it with a trial version.