I am working on a little test project to implement a chat server for a site I am working on. I have followed the guide from http://www.sanwebe.com/2013/05/chat-using-websocket-php-socket and I've got it all pretty much working, I've also managed to amend it slightly so that you can send messages to individual users instead of globally to all online users.
What I am looking for now though whether using the implementation as shown in the tutorial above whether I can show that the user is currently typing.
I was thinking about looking for key events and just send a system message to the user and I can handle this in the client to show typing and then if after a second nothing else has come in hide it again, but this seems a bit inefficient and network.
Would my idea above be the only solution or is there a better implementation?
It actually depends on your definition of "currently typing".
For me, that would be "there is some text in the input field".
This would require you to listen to change events on the input field and only send a message when the field turns from empty to non-empty or the other way round.
And, of course, the server would need to send a "stopped typing" message if the user disconnected.
However, if your definition is "user has touched a key in the last X milliseconds", then there is no other way but to constantly send messages.
Besides being more network-intensive, this option has the disadvantage of lags being shown as disconnects, possibly resulting in a shaky UI appearance.
Either way, I would listen for change on the input field, and not for actual key presses.
Related
I was wondering how to set up a system in which an authenticated user could send, with a simple graphical interaction (cliccking a button or so) a non-replayable request/message to the server from an application or a web page.
It's crytical there's must not be a way to set up an automated system that replaces user interaction automating the request as this would totally break up my entire project.
Moreover, as this action must be frequently repeated, it should not implement boring stuff like chaptas or so.
A pratical example: let's say the web page, shown after the login, displays a button that sends the server a request. How can I be sure the request was sent because the user actually clicked the button and it wasn't some sort of bot that forged the message?
Is that even possible to check? I'm sure it is and I'm quite sure there's must be some simple implementation I'm missing, and I'm sorry if this is a trivial question.
Also, if the solution is hiding ('cause I already searched a lot!) out there, please point me to it.
Thanks for your attention.
You could use a non-graphical captcha like a simple question.
Generate a simple addition of two random integers between 0 and 10.
Add a text field to ask for the result.
The result is very easy to find (for a human being), and very quick to type.
Example:
What is the result of 7+5? Write your result here: [_]
It should only block robots and very young or very stupid people.
I recently found a huge security problem with my PM system that allows users to send a message as much as they want with a for loop in the address bar. Someone put this into the address bar:
javascript:for(x=0;x<10000;x++){ $('#compose form').submit(); }
And the message was sent 1000 times to me and my inbox was full of the same message and my database was so full that phpMyAdmin was being very laggy.
My question is, how can I prevent this? This is a major issue.
Also, the form is submitted with AJAX.
Edit:
I use PHP, so how can I prevent this? Like how could I make it to where a message can only be sent every 5 minutes or so and if they submit more than one within 5 minutes it will display an error (or not show any user feedback at all and just stop it from being submitted)?
There is one obvious way to fix it and the problem lies not on the client side - it lies on the server side.
Your server script should not allow sending messages too often - eg. often than, say, once each 10 minutes. To do it you can use session mechanism on the server side and save the information when the user sent the email. If user has not sent an email, you should also save that information in session - to distinguish people with session enabled from people with session disabled (and you should block the latter from sending emails at all).
The way session should be implemented (the specific code) depends on the language you use for server side scripting (PHP, Python, JSP etc.).
If someone has the knowledge to do this to you, you likely cannot do anything on the client side so the only option I'd say is that you log or keep a count etc of the number of requests (perhaps to a particular resource) and deny the request (or send a "busy" http code etc) for the particular user.
I think the simplest way would be to count requests from a specific IP address (this obviously has drawbacks such as multiple users behind a proxy or NAT etc).
The actual solution will depend on your server side language and web server but you could perhaps frame up a rule and see how it works. Something like 5 requests per minute (or whatever is appropriate for your usage) per IP address.
As others have said, you have to implement protection on the server. No amount of client-side coding will provide protection.
The common way of protecting a server against this type of abuse is called rate limiting. You decide how often you want a given client to be able to submit a message and you code the server to ignore any messages that fall outside that limit.
For example, you could decide that you will allow no more than one message a minute and no more than two messages every 10 minutes and no more than four messages an hour. You pick whatever you think seems reasonable and you code to that algorithm.
In your server, you'd have to be able to identify which user the message was coming from (most likely via an authentication cookie) and in your database, you'd have to be able to find out when the last message was sent by that user. You may even be able to cache that information in RAM in your server (depending upon how your server works) to avoid a database lookup on every request since you only have to keep recent info and you only need to performance optimize on the repeat offenders (the ones that are trying to abuse your server and thus just recently sent a request).
I agree with what everyone is posting about rate-limiting.
However, this can be very complicated to implement on the server side. Especially when you start scaling out. And, honestly, if 1000 messages hurt you that bad - my suggestion may apply to you even more so.
Instead of implementing this yourself, you may want to look into a third party service to do it for you, such as this super cool web services proxy Apigee
One of their features is, specifically, API rate throttling. (see link)
Hopefully this is the right place for this question.
I have done a fair amount of research and yet to find anything that matches what I want. What I'm envisioning is the following. Let me know if any of you know of a program that will do what I want.
Also it must be web-based
anom user -> fills out form ->
email gets sent to admin saying xyz has filled out form abc with links to approve/disapprove request.
admin can also login and edit form and resent results to original submitter. Also once the admin approves/disapproves request the original submitter gets an approve/disapprove email.
and you can search by date submitted, specific project/form, status of request(submitted, approved, disapproved).
any ideas all on where I could find this? I started to look into drupal with workflows and actions but it just doesn't flow right for this
This sounds like you need to program the solution yourself, using something like PHP or ASP.NET (as you specified) as possible technologies, and a backend database like MySQL. If your process takes more than one sentence to describe, the exact system you want usually does not yet exist.
If you don't have the knowledge to code this, it sounds like you will need to hire someone who does. This isn't exactly a specific "how do I do this" programming question; this is a specification for an application, which a competent programmer should be able to turn into a working application for you.
Best of luck!
First,
the set up:
I have a script that executes several tasks after a user hits the "upload" button that sends the script the data it need. Now, this part is currently mandatory, we don't have the option at this point to cut out the upload and draw from a live source.
This section intentionally long-winded to make a point. Skip ahead if you hate that
Right now the data is parsed from a really funky source using regex, then broken down into an array. It then checks the DB for any data already in the uploaded data's date range. If the data date ranges don't already exist in the DB, it inserts the data and outputs success to the user (there is also some security checks, data source validation, and basic upload validation)... If the data does exist, the script then gets the data already in the DB, finds the differences between the two sets, deletes the old data that doesn't match, adds the new data, and then sends an email to each person affected by these changes (one email per person with all relevant changes in said email, which is a whole other step). The email addresses are pulled by means of an LDAP search as our DB has their work email but the LDAP has their personal email which ensures they get the email before they come in the next day and get caught unaware. Finally, the data-uploader is told "Changes have been made, emails have been sent." which is really all they care about.
Now I may be adding a Google Calendar API that posts the data (when it's scheduling data) to the user's Google Calendar. I would do it via their work calendar, but I thought I'd get my toes wet with Google's API before dealing with setting up a WebDav system for Exchange.
</backstory>
Now!
The practical question
At this point, pre-Google integration, the script takes at most a second and a half to run. It's pretty impressive, at least I think so (the server, not my coding). But the Google bit, in tests, is SLOOOOW. We can probably fix that, but it raises the bigger question...
What is the best way to off-load some of the work after the user has gotten confirmation that the DB has been updated? This is the part he's most concerned with and the part most critical. Email notifications and Google Calendar updates are only there for the benefit of those affected by the upload, and if there is a problem with these notifications, he'll hear about it (and then I'll hear about it) regardless of the script telling him first.
So is there a way, for example, to run a cronjob that's triggered by a script's last execution? Can PHP create cronjobs with exec() ability? Is there some normalized way of handling post-execution work that needs getting done?
Any advice on this is really appreciated. I feel like the scripts bloated-ness reflects my stage of development and the need for me to finally know how to do division-of-labor in web apps.
But I also get worried that this is not done, as user's need to know when all tasks are completed, etc. So this brings up:
The best-practices/more-subjective question
Basically, is there an idea that progress bars, real-time offloading, and other ways of keeping the user tethered to the script are --when combined with optimization of the code, of course-- the better, more-preferred method then simply saying "We're done with your part, if you need us, we'll be notifying users" etc etc.
Are there any BIG things to avoid (other than obviously not giving the user any feedback at all)?
Thanks for reading. The coding part is crucial, so don't feel obliged to cover the second part or forget to cover the coding part!
A cron job is good for this. If all you want to do when a user uploads data is say "Hey user, thanks for the data!" then this will be fine.
If you prefer a more immediate approach, then you can use exec() to start a background process. In a Linux environment it would look something like this:
exec("php /path/to/your/worker/script.php >/dev/null &");
The & part says "run me in the backgound." The >/dev/null part redirects output to a black hole. As far as handling all errors and notifying appropriate parties--this is all down to the design of your worker script.
For a more flexible cross-platform approach, check out this PHP Manual post
There are a number of ways to go about this. You could exec(), like the above says, but you could potentially run into a DoS situation if there are too many submit clicks. the pcntl extension is arguably better at managing processes like this. Check out this post to see a discussion (there are 3 parts).
You could use Javascript to send a second, ajax post that runs the appropriate worker script afterwards. By using ignore_user_abort() and sending a Content-Length, the browser can disconnect early, but your apache process will continue to run and process your data. Upside is no forkbomb potential, Downside is it will open more apache processes.
Yet another option is to use a cron in the background that looks at a process-queue table for things to do 'later' - you stick items into this table on the front end, remove them on the backend while processing (see Zend_Queue).
Yet another is to use a more distributed job framework like gearmand - which can process items on other machines.
It all depends on your overall capabilities and requirements.
For my current web development project I'm implementing a back end system that will flag errors and send an email to the administrator automatically with details about what occurred. Trapping the error and generating the email with appropriate error information is pretty straight forward; but a problem arises when one considers certain groups of error types especially if the site is being visited frequently.
Consider a couple of examples:
An unplanned database outage that prevents all of the scripts on the web server from being able to connect. If it takes say 2 minutes (120 seconds) for the database server to come back online, and the web server is receiving unique requests at a rate of 10/second, in the time it takes the database server to come back online the admins email would be flooded with 1200 identical emails all screaming about a failure to connect to the database.
A bug in a script somewhere managed to sneak by testing and is of the variety that completely screws up content generation and occurs only in a specific set of circumstances (say once every 100 requests). Using the unique request rate of 10/second again means the administrator is going to be getting the same email every 10 seconds about the same bug until it is fixed.
What are some approaches/strategies I can use to prevent this scenario from occurring? (I am only interested in monitoring of errors generated by the script, infrastructure issues are beyond the scope of this solution)
I going to assume that I can almost always uniquely identify errors using a digest of some of the values passed to the error handler callback set by set_error_handler.
The first and probably most obvious solution is recording in a database and only send the email if a reasonable minimum period of time has passed since it last occurred. This isn't the ideal approach especially if the database is causing the problem. Another solution would be to write files to disk when errors occur and check if a reasonable minimum period of time has passed since the file was last modified. Is there any mechanism to solve this problem beyond the two methods I have described?
Why not simply allow them all to be sent out and then collect and store them in a database on the recipient end. That way you bypass the possibility of the database being the problem in the server.
Also, a greater advantage in my opinion, is that you don't arbitrarily throw out valuable forensic data. Post hoc analysis is very important and any kind of filtering could make it incredibly difficult, or impossible.
Have you tried looking into monitoring software like SiteScope?
What i did was monitoring the error log, and sending a digest every 5 minutes. I'd like to think it's because of my high quality code (versus an unpopular app!), but i don't get hassled too much :P I basically read the log file from end to start, parse error messages, and stop when the timestamp < the last time i ran the job, then send a simple email.
This works well enough. However, if you use POST alot, there is a limited amount of information you could get from correlating the apache access log with your php error log. I remember reading about a module to log POSTs to a file from within apache, but don't remember the specifics.
However, if you're willing to use the error handler to write somewhere, that might be best as you've got access to much more information. ip, session id (and any user information, which might impact settings, like pagination or whatever), function arguments (debug_backtrace, or whatever it is) ... Write every error, just send messages when new errors occur, or after an error has been acknowledged (if you care to write such a system).
You should go ahead and generate whatever log files you want. But instead of sending the emails yourself, hook the logs up to a monitoring system like Nagios. Let the monitoring solution decide when to alert the admins, and how often.