I want to achieve high availability with SolR Cloud.
I need to dev a SolR PHP Client supporting node failure.
My lead is to work with guzzle RetryMiddleware and somehow keeptrack of up or down nodes.
My question is : is it a good lead ? (I'm not very familiar with Guzzle)
I'm not familiar with Solr Cloud, but IMO if you want to create a proper client, you need to write your own middleware for Guzzle with the specific fallback logic inside.
RetryMiddleware is basically for retrying the same request after a delay period, nothing more. You cannot change the request (send it to a different node or something). That's why I think it could be only a part of the solution.
Otherwise the question is too broad at the moment.
Related
I was checking a part of my application in which I connect to elasticsearch host server and then I realized for every time the front-end sends an report request to my back-end I'm creating an instance of elasticsearch client class using the following code :
$elasticClient = ClientBuilder::create()->setHosts($this->setHostsParams())->build();
Since our application sends about 20 requests to the back-end by loading the first page, I was considering if PHP's elasticsearch library might be capable of optimizing the initiation phase, or if anyone has a better solution for this, or it might not be a big of a deal after all and it's not a real overhead!?
PS : I did some research with it and didn't find any resources covering this subject.
Sharing an object instance is already discussed here and elsewhere so I'm not going to go into that.
What I'd point out, though, is there there's an elasticsearch API called _msearch which enables you to send multiple search payloads at the same time and the system will respond after all the individual requests have resolved. Here's some sample PHP usage.
This might be useful if you need all your ~20 requests resolved at once -- though it may be useless if you defer some of those requests only after, say, a user scrolls down and what not.
I'm building an MVC web application (let's call it xyz.com). I want it to also support mobile apps via an API on the same server (let's call this api.xyz.com).
I'm confused about how to structure the web app vs. the API:
Should the web app use the API to query the database? Or should it perform them independently (like using a Model)? I mean, should the flow be (User > Controller > API > Database) or (User > Controller > Model > Database)?
If we use the APIs to query the database, how would you query the API? Would you use something like cURL?
If we use cURL, wouldn't that slow down the process? (As we are making a second request from the controller)? What's the ideal way to do this?
I've tried reading up on API-centric web apps, but there's not too much information about this on the net.
Any guidance would be appreciated.
since I have not enough points to comment I'll just leave my thoughts here.
1) I would make the web app use the API for 2 main reasons:
I don't like having multiples components accessing my database, you'll most likely have a lot of duplicate code on the data extraction and if you have a lot of filtering you might make mistakes and they'll behave differently.
I believe that (if I'm wrong please correct me) it'll be easier to increase performances on the app rather than the database. Since the API will handle resources it'll be easier to cache things than a full HTML web page.
2) To query the API I would most likely use composer and find a nice HTTP library, not sure which one though since I didn't use php in a while (symfony's and laravel's are quite nice if I remember well).
3) Yeah it might slow down the process a little but I believe it won't be enough to be noticed by the user. As I said above, with a correct cache handling you'll do just fine.
Hope having my thoughts on this can help, if I was wrong somewhere please feel free to correct me in the comments below (Don't know if I'll be able to respond tho... :s).
Have a nice day !
You really want to know the difference between an n-tier architecture and a single-tier architecture. An n-tier architecture is comprised of several tiers, which are connected via an internal protocol and API. E.g.:
backend frontend
data --- HTTP server --- HTTP client --- application --- HTTP server --- browser
application
You could have many more tiers in there. Those tiers can all run on the same physical machine, but they still talk to each other over HTTP; more likely you'd want to run each tier on a different machine though.
Yes, this will obviously incur some overhead in talking to your database backend over HTTP instead of doing it within the same PHP process. However, this is offset by:
Caching is built-in in this architecture. If you make proper use of HTTP caching by using a fully capable HTTP server and client and are using the HTTP cache mechanism well, you can reduce the absolute amount of queries made enormously. Practically you'd set up a reverse proxy on the web server on your frontend server, so your queries go PHP → curl → web server reverse proxy → backend server. If the reverse proxy is caching properly, that's where the chain often stops, which can be much faster than executing the actual query on the database. If the backend server is using HTTP caching effectively, it too can probably often respond with a simple 304 Not Modified, which is also very fast.
Load is distributed among more machines, which speeds up each individual machine. If your frontend servers can largely work with cached data without needing to bother the database, the database is much faster for the queries that it does need to handle. You can also scale up your frontend servers to many instances, each of which is faster because it has less load.
These are the advantages of such an architecture. The further advantage is that you can also directly expose your internal HTTP API to the outside world (again, reverse proxies make a lot of sense here). If you're not using an internal HTTP API, you have to write:
A controller and view which gets data from the model and renders to HTML.
A controller and view which gets data from the model and renders to JSON (or whatever).
To some degree controllers can be reused and simply the view switched out, if everything else is the same, but often you'll find that you need to duplicate each logical "thing" to be served as HTML in one case and JSON in another, and you need to keep those in sync.
I have web requests that fail every now and then, however my application really needs the data that service provides.
What is the best pattern for retying the request?
I know there would be issues with cascading if I just kept trying indefinitely and straight away.
I am using the cURL library in PHP
Google uses an algorithm that tries after 2^retrycount seconds. I think that is a good algorithm, but if you need the information right now, try to cache the answer and use the cache until the resource is available again. If it's possible to wait that long, I'd recommend the Google algorithm.
I'm setting up a realtime app that will be using socket.io. There's currently some core functionally in php, that utilizes memcache and mysql backend.
Would it make sense in the socket.io server to do an ajax request (if that's even possible) to the php page that handles this? There's a lot of MySQL querying, I know it can be done in node.js, but I'd rather keep this part abstracted in php if possible.
So, my question is, is that a proper thing to do? Call a php page from within the socket.io server to then return to the client?
Thanks!
I don't see any problems with having your node.js app communicate with your PHP app by exposing a RESTful API or some PHP script that you can POST to or GET from your socket.io node.js server. There are plenty of npm modules (like request) that can make HTTP requests like that a breeze for you. After retrieving the data from PHP in your node app, you can use socket.io to emit() the data to the socket.io client on the frontend.
There is nothing wrong with that. You are simply using a RESTful API to access the MySQL data, thus isolating the database details.
If one day you are tired of PHP, you can easily switch to Ruby, Python or Whatever for that part without even touching the node.js. If your logic is already written in PHP (you are upgrading an old app), it make even more sense as you can reuse what has already been tested and debugged. A lot of folks are advocating for that kind of separation between systems. Just look at all the SOA (Service Oriented Architecture) buzz.
Where I work we are using this very architecture in a project (though in this case its an ASP.NET MVC Website calling a Java EE app) and it served us very well. With the event model of node.js, its even better since you won't block waiting for the PHP.
But of course, there are some drawback
Performance overhead
Architecture is more complicated
You now work with two language instead of only one (though javascript and PHP are so often used together that I don't think it's really is a problem in this case)
So you need to ask yourself if your problem really need that solution. But in a lot of case the answer may be yes. Just don't forget the virtue of Keeping It Simple and Stupid (the KISS principle)
I'm currently re-developing a fairly large-scale PHP web application. Part of this redevelopment involves moving the bulk of some fairly hefty business logic out of the core of the web app and moving it into a set of SOAP web services.
What's currently concerning me (just slightly) is perceived overhead this brings with it in terms of local HTTP traffic. I should explain that the SOAP web services currently, and for the foreseeable future, will reside on the same physical server, and if and when they move they will remain on the same network. What concerns me is the fact that for each call that was an internal php function call, it is now an http request invoking a similar function call.
Obviously this is something I can measure as we move further along the line, but I was wondering if anyone could offer any advice, or more importantly share any previous experience of taking an application down this route.
Are you doing hundreds or thousands of these calls a second? If not, then you probably don't have to worry.
But profile it. Set up a prototype of the system working across the network with a large number of SOAP calls, and see if it slows down to unacceptable levels.
If the server is running on the same physical box then you can't have any privilege seperation. And increasing capacity by adding tiers to the stack (instead of euivalent nodes) is a recipe for non-availability.
If you're hiding something behind SOAP, the the HTTP overhead is likely to be relatively small in comparison to what the 'something' is doing (e.g. reading from database). However when you add up the cost of constructing the SOAP request, decomposing the soap request the compositing the response, add the overhead for HTTP then one has to wonder why you don't provide a shortcut by calling the PHP code implemented within the server directly in the same thread of execution.
If you were to design an abstract soap interface which mapped directly to and from php types then this could be shortcutted without having any overhead in maintining different APIs.
Does it HAVE to be a SOAP web service? Is this the client telling you this, or is it your decision?
You seem concerned about HTTP calls, which is fine, but what about the overhead of unnecessarily serializing/de-serializing to/from XML to be transferred over the "wire" - that "wire" being the same machine. =)
It's doesnt really make sense to have a SOAP-based web service where it's only to be consumed by a client on the same machine.
However I agree with #Skilldrick's answer - its not going to be an issue as long as you are intelligent about your HTTP calls.
Cache whenever you can, batch your calls, etc.
SOAP is more verbose that REST. REST uses HTTP protocol to do the same with less network bandwith, if that's your concern.
See:
WhatIsREST
Wikipedia
As to really answer your question, remember the 80/20 rule. For that use a benchmarking/tracing tool to help you finding where your hotspots are. Fix those and forget about the rest.