PHP Server-side rate limit on AJAX callbacks? - php

I'm using PHP in conjunction with AJAX for a lot of functions on my site. I've implemented a sort of rate limiting system on the client-side with Javascript. Basically, disabling a button for 1 sec after it's clicked.
Although this works fine for the more innocent cases, but I feel like I need something on the server-side of things to limit requests as well.
Basically, I want users to have a maximum amount of AJAX calls they can make per second. In fact, one per second seems reasonable.
One way would be somehow logging every request to my AJAX callback, and reading from that table before a new request is made. But this would immensely increase the work load on my server and database.
Are there any alternative methods of doing this?
PHP
function comment_upvote_callback() {
// Some sort of rate limit??
// $_POST data validation
// Add upvote to database
// Return success or error
}
add_action( 'wp_ajax_comment_upvote_callback', 'comment_upvote_callback' );
jQuery
var is_clicked = false;
if ( is_clicked == false ) {
$('#comments').on('click', '.comment-upvote', function(event) {
event.preventDefault();
// Button disabled as long as isClicked == true
isClicked = true;
// Data to be sent
var data = {
'action': 'comment_upvote_callback',
'security': nonce,
'comment_id': comment_id
};
// Send Data
jQuery.post(ajaxurl, data, function(response) {
// Callback
// Client-side rate limit
setTimeout( function() {
is_clicked = false;
}, 1000);
});
});
}

As already told above, you should be using rate limiting at server-side
I have written a code to implement the same. You can copy the code into a file and simply include in your server-side script at the top. It accepts maximum 3hits/5secs. You can change the rate according to your needs
session_start();
const cap = 3;
$stamp_init = date("Y-m-d H:i:s");
if( !isset( $_SESSION['FIRST_REQUEST_TIME'] ) ){
$_SESSION['FIRST_REQUEST_TIME'] = $stamp_init;
}
$first_request_time = $_SESSION['FIRST_REQUEST_TIME'];
$stamp_expire = date( "Y-m-d H:i:s", strtotime( $first_request_time )+( 5 ) );
if( !isset( $_SESSION['REQ_COUNT'] ) ){
$_SESSION['REQ_COUNT'] = 0;
}
$req_count = $_SESSION['REQ_COUNT'];
$req_count++;
if( $stamp_init > $stamp_expire ){//Expired
$req_count = 1;
$first_request_time = $stamp_init;
}
$_SESSION['REQ_COUNT'] = $req_count;
$_SESSION['FIRST_REQUEST_TIME'] = $first_request_time;
header('X-RateLimit-Limit: '.cap);
header('X-RateLimit-Remaining: ' . ( cap-$req_count ) );
if( $req_count > cap){//Too many requests
http_response_code( 429 );
exit();
}

Related

JSON reponse for Jquery/Ajax in Wordpress fails for bigger size file

I am implementing own custom function for historical extract in CSV format from MySQL database using jQuery-Ajax in WordPress environment. I have an HTML where user selects the start date and end date and clicks on a button and then the process works.
When the JSON response is in the range of 900kb to 1 MB, then extraction works. But when the response size increases beyond this then AJAX callback goes in error and returns nothing.
Below is the JavaScript file:
jQuery(document).ready(function(jQuery) {
jQuery('#extract_btn').click(function(){
var startdate = jQuery( '#from-date' ).val();
var enddate = jQuery( '#to-date' ).val();
var data1 = {
action: 'hist_extract',
fromdate: startdate,
todate: enddate
};
// since 2.8 ajaxurl is always defined in the admin header and points to admin-ajax.php
jQuery.ajax({
type:"post",
url:MyAjax1.ajaxurl,
data:data1,
success:function(response) {
if (response == '')
return;
alert('Got this from the server: ' + JSON.parse(response));
JSONToCSVConvertor(response, "Historic Price", true);
},
error:function(xhr, status, error){
alert('Error in response');
var err = JSON.parse(xhr.responseText);
alert(err.Message);
}
});
});
});
function JSONToCSVConvertor(JSONData, ReportTitle, ShowLabel) {
alert('Start of Json Convertor')
//If JSONData is not an object then JSON.parse will parse the JSON string in an Object
var arrData = typeof JSONData != 'object' ? JSON.parse(JSONData) : JSONData;
var CSV = '';
//Set Report title in first row or line
//CSV += ReportTitle + '\r\n\n';
//This condition will generate the Label/Header
if (ShowLabel) {
var row = "";
//This loop will extract the label from 1st index of on array
for (var index in arrData[0]) {
//Now convert each value to string and comma-seprated
row += index + ',';
}
row = row.slice(0, -1);
//append Label row with line break
CSV += row + '\r\n';
}
//1st loop is to extract each row
for (var i = 0; i < arrData.length; i++) {
var row = "";
//2nd loop will extract each column and convert it in string comma-seprated
for (var index in arrData[i]) {
row += '"' + arrData[i][index] + '",';
}
row.slice(0, row.length - 1);
//add a line break after each row
CSV += row + '\r\n';
}
if (CSV == '') {
alert("Invalid data");
return;
}
alert(CSV);
//Generate a file name
var fileName = "Edding_";
//this will remove the blank-spaces from the title and replace it with an underscore
fileName += ReportTitle.replace(/ /g, "_");
//this trick will generate a temp "a" tag
var link = document.createElement("a");
link.id="lnkDwnldLnk";
//this part will append the anchor tag and remove it after automatic click
document.body.appendChild(link);
var blob = new Blob([CSV]);
if (window.navigator.msSaveOrOpenBlob) // IE hack;
window.navigator.msSaveBlob(blob, fileName+".csv");
else
{
var a = window.document.createElement("a");
a.href = window.URL.createObjectURL(blob, {type: "text/plain"});
a.download = fileName+".csv";
document.body.appendChild(a);
a.click(); // IE: "Access is denied"
document.body.removeChild(a);
}
}
Below is the functions.php having custom hook:
//----------------------------------------------------------------------------------
//Below is the custom Javascript hook for Historic Extract
//----------------------------------------------------------------------------------
function price_history() {
$handle = 'hist_extract';
$list = 'enqueued';
if (wp_script_is( $handle, $list )) {
return;
}
else
{
// registering and enqueueing the Javascript/Jquery
wp_enqueue_script('jquery');
wp_register_script('hist_extract', get_template_directory_uri() . '/js/Historic_Price.js', array( 'jquery' ), NULL, false );
wp_enqueue_script('hist_extract');
wp_localize_script('hist_extract', 'MyAjax1', array(
// URL to wp-admin/admin-ajax.php to process the request
'ajaxurl' => admin_url('admin-ajax.php'),
// generate a nonce with a unique ID "myajax-post-comment-nonce"
// so that you can check it later when an AJAX request is sent
'security' => wp_create_nonce('my-special-string')
));
error_log('Js for Historic Price loaded successfully');
error_log(admin_url('admin-ajax.php'));
}
}
add_action('wp_enqueue_scripts', 'price_history');
//----------------------------------------------------------------------------------
// Custom function that handles the AJAX hook for Historic Extract
//----------------------------------------------------------------------------------
function historic_data_extract() {
error_log('Start of report data function on ajax callback');
// check_ajax_referer( 'my-special-string', 'security' );
$from_date = $_POST['fromdate'];
$to_date = $_POST['todate'];
$convert_from_date= date("Y-m-d", strtotime($from_date));
$convert_to_date = date("Y-m-d", strtotime($to_date));
error_log($from_date );
error_log($to_date);
error_log($convert_from_date);
error_log($convert_to_date);
//******************************************
//Custom Code for fetching data from server database
//********************************************
//header("Content-Type: application/json; charset=UTF-8");
define("dbhost", "localhost");
define("dbuser", "xxxxxxxxx");
define("dbpass", "xxxxxxxxx");
define("db", "xxxxxxxx");
$emparray = array();
$conn = mysqli_connect(dbhost, dbuser, dbpass, db);
// Change character set to utf8
mysqli_set_charset($conn,"utf8");
if ($conn )
{
$query = "SELECT PR_PRICE_HIST_TBL.PR_PRODUCT_ID,PR_PRICE_HIST_TBL.PR_URL_ID,PR_PRICE_HIST_TBL.PR_SHOP_NAME,PR_PRICE_HIST_TBL.PR_LAST_CHECKED,PR_PRICE_HIST_TBL.PR_CUST_PROD_CODE,PR_PRICE_HIST_TBL.PR_PRODUCT_NAME,PR_PRICE_HIST_TBL.PR_LAST_PRICE,PR_PRICE_HIST_TBL.PR_CONV_PRICE,PR_PRICE_HIST_TBL.PR_DOMAIN,PR_PRICE_HIST_TBL.PR_COUNTRY_CODE,PR_PRICE_HIST_TBL.PR_AVAILABLE,PR_PRICE_HIST_TBL.PR_AVAIL_DESCR,PR_PRICE_HIST_TBL.PR_PRICE_TIME,PR_PRICE_HIST_TBL.PR_FAULT_FLAG,PR_PRICE_HIST_TBL.PR_FAULT_TIME,PR_PRICE_HIST_TBL.PR_FAULT_MSG,TABLE_72.MIN_PRICE,TABLE_72.MAX_PRICE,TABLE_72.AVG_PRICE,TABLE_72.DEV_PRICE
FROM PR_PRICE_HIST_TBL
INNER JOIN TABLE_72 ON PR_PRICE_HIST_TBL.PR_URL_ID=TABLE_72.PR_URL_ID AND
PR_PRICE_HIST_TBL.PR_SHOP_NAME=TABLE_72.PR_SHOP_NAME AND
PR_PRICE_HIST_TBL.PR_PRODUCT_NAME=TABLE_72.PR_PRODUCT_NAME
AND PR_PRICE_HIST_TBL.PR_LAST_CHECKED BETWEEN '$convert_from_date' AND '$convert_to_date';";
error_log($query);
$result_select= mysqli_query($conn,$query);
error_log(mysqli_num_rows($result_select));
error_log(mysqli_error($conn));
while($row = mysqli_fetch_assoc($result_select))
{
error_log(json_encode($row));
$emparray[] = $row;
}
//error_log(json_encode($emparray));
echo json_encode($emparray);
die();
}
}
add_action('wp_ajax_hist_extract', 'historic_data_extract');
add_action('wp_ajax_nopriv_hist_extract', 'historic_data_extract');
From the code above, you can see, I have tried to implement many things by going over different forums. But I am stuck here. I am not able to understand where could be the potential problem. FYI..I am hosting this on GoDaddy Server. I tried below things:
Tried to make query execution faster by removing views from join. It seems, query is fetching results in around 15 seconds
Format of the data in JSON and tried async: false, but not working
Tried to modify values in init.php. But of no use.
pload_max_filesize = 64M
post_max_size = 64M
memory_limit = 400M
file_uploads = On
max_execution_time = 300
Tried to implement (error:function) for AJAX response. Where only the first alert('Error in response'); is throwing. But can not see the XHR response text.
Any help is appreciated. Please let me know if I miss something or want more information.
The best way to solve the issue is to have a good night's sleep.
Thanks for your clues.
Issue was: In xhr my ajax request was going in cancelled status.
Solution: I was missing preventdefault() in my function.
Now I can see MBs of JSON response. Thanks again for provided clues.
Currently preventdefault() has solved my issue. If you feel, anything else also needs to be taken care as a best practice in my code. Please do not hesitate to comment.
Thanks.

Echo current date and time and free space on disk every second in php

I'm new to php and I'm trying to get the current date and time and the available space on disk and update it every second on a web page.
For the date and time I use: date("d-m-Y H:i:s").
For getting the free space I know I can use the diskfreespace() function that takes the path as argument.
In my case I'm trying with diskfreespace("C:").
It returns the number of bytes and since I have Gigabytes of space I divide the number of bytes in order to get the number of Gigabytes.
diskfreespace("C:") / pow(1024, 3)
It works though it's executed only once and I'd like the function to execute every second and display the value through the echo function.
Then I tried using an infinite loop with a sleep() of 1 second but it seems there is a problem because the values aren't updated every second and it seems like if the page doesn't load properly.
<?php
while(1)
{
echo "Current date and time: " . date("d-m-Y H:i:s");
echo "</br></br>Free space on C: disk " . (diskfreespace("C:") / pow(1024, 3)) . " Gb";
sleep(1);
}
?>
If you use Server Sent Events you can have a connection to a PHP script that runs in an infinite loop that pushes out the data to a javascript listener.
<?php
/*
diskspace_sse.php
*/
set_time_limit( 0 );
ini_set('auto_detect_line_endings', 1);
ini_set('max_execution_time', '0');
/* -- Edit to suit your location -- */
date_default_timezone_set( 'Europe/London' );
ob_end_clean();
/* -- set headers -- */
header('Content-Type: text/event-stream'); /* !important! */
header('Cache-Control: no-cache');
header('Access-Control-Allow-Credentials: true');
header('Access-Control-Allow-Methods: GET');
header('Access-Control-Expose-Headers: X-Events');
/* -- utility function to send formatted sse message -- */
if( !function_exists('sse_message') ){
function sse_message( $evtname='gas', $data=null, $retry=1000 ){
if( !is_null( $data ) ){
echo "event:".$evtname."\r\n";
echo "retry:".$retry."\r\n";
echo "data:" . json_encode( $data, JSON_FORCE_OBJECT|JSON_HEX_QUOT|JSON_HEX_TAG|JSON_HEX_AMP|JSON_HEX_APOS );
echo "\r\n\r\n";
}
}
}
/* -- How often to send messages -- */
$sleep=1;
$disk='c:';
while( true ){
if( connection_status() != CONNECTION_NORMAL or connection_aborted() ) {
break;
}
/* Infinite loop is running - perform actions you need */
$payload=array(
'date' => date(DATE_COOKIE),
'diskspace' => disk_free_space($disk),
'totalspace'=> disk_total_space($disk),
'formatted_diskspace' => round( disk_free_space($disk) / pow( 1024,3 ), 2 ).'Gb',
'formatted_total' => round( disk_total_space($disk) / pow( 1024,3 ), 2 ).'Gb'
);
/* -- prepare sse message -- */
sse_message( 'diskspace', $payload );
/* -- Send output -- */
if( #ob_get_level() > 0 ) for( $i=0; $i < #ob_get_level(); $i++ ) #ob_flush();
#flush();
/* wait */
sleep( $sleep );
}
if( #ob_get_level() > 0 ) {
for( $i=0; $i < #ob_get_level(); $i++ ) #ob_flush();
#ob_end_clean();
}
?>
In your html page
<div id='diskspace'></div>
<script type='text/javascript'>
function bindEvtSource(){
var url='http://localhost/diskspace_sse.php';
if ( !!window.EventSource ) {
var evtSource = new EventSource( url );
evtSource.addEventListener( 'open', function(e){
console.log(e.type);
},false);
evtSource.addEventListener( 'error', function(e){
console.error('%o %s',e,e.type);
},false);
evtSource.addEventListener( 'diskspace', function(e){
var json=JSON.parse(e.data);
/* you could work with the json data here */
getobject('diskspace').innerHTML=e.data;
},false);
} else {
alert('Server Sent Events are not supported in this browser');
}
}
document.addEventListener( 'DOMContentLoaded', bindEvtSource, false );
</script>
You should get feedback about the diskspace usage every second with no ( very little ) slowdown in page load times.
Echo shows result after code finishs the job. In your case, it may work in console application, not in web. To get your goal you have to look at ajax. In front side you need to call your php code, and render result. After a second do it again.
Check this answer.
Also, instead of the infinite loop in PHP, you may want to implement a simple polling mechanism with AJAX requests. Otherwise, depending on your server settings, your infinite loop may be stopped after a period of time. See max_execution_time.

PHP MYSQL JQuery Long Polling - Not working as expected

My long polling implementation isn't working. Been having a very difficult time understanding where to look toward debugging said code.
Key Points
No Errors
Long polling working randomly (only responds to some changes in MySQL with no distinct pattern)
MySQL is updating correctly
I'm testing this via Localhost WAMP and two browsers with two different sessions
PHP Portion -
$path= $_SERVER[ 'DOCUMENT_ROOT'];
$path .= "/config.php" ;
require_once($path);
require_once(PHP_PATH . "/classes/user.php");
session_start();
require_once(PHP_PATH . "/functions/database.php");
// Return to Login if no Session
if(!isset($_SESSION['user'])){
header("Location: /login");
die();
}
$db = connectdatabase();
$timeout = 40;
// if no post ids kill the script // Should never get here
if(!isset($_POST['post_ids'])){
die();
}
if(!isset($_POST['timestamp'])){
die();
}
$last_ajax_call = $_POST['timestamp'];
$post_ids = trim(strip_tags($_POST['post_ids']));
$id = $_SESSION['user']->getID();
// Check if there are posts from the last search that need to be updated with a comments or the like number has to be updated
$query = "SELECT posts.*, users.first_name, users.last_name, users.picture
FROM posts
LEFT JOIN users
ON users.id = posts.user_id
WHERE ((UNIX_TIMESTAMP(posts.date) > :last_ajax_call OR UNIX_TIMESTAMP(posts.last_modified) > :last_ajax_call)
AND posts.parent IN (:post_ids)) OR (posts.id IN (:post_ids) AND UNIX_TIMESTAMP(posts.last_modified) > :last_ajax_call)";
while ($timeout > 0) {
$check_for_updates = $db->prepare($query);
$check_for_updates->bindParam(':post_ids', $post_ids);
$check_for_updates->bindParam(':last_ajax_call', $last_ajax_call);
$check_for_updates->execute();
$r = $check_for_updates->fetchAll();
if(!empty($r)){
// Get current date time in mysql format
$unix_timestamp = time();
// Cofigure result array to pass back
$result = array(
'timestamp' => $unix_timestamp,
'updates' => $r
);
$json = json_encode($result);
echo $json;
return;
} else {
$timeout --;
usleep( 250000 );
clearstatcache();
}
}
// you only get here if no data found
$unix_timestamp = time();
// Cofigure result array to pass back
$result = array(
'timestamp' => $unix_timestamp
);
$json = json_encode($result);
echo $json;
JQuery Ajax -
function getUpdates(timestamp) {
var post_ids = $("#newsfeed").find("#post_ids").attr('data-post-ids');
var data = {'timestamp' : timestamp,
'post_ids' : post_ids};
poll = $.ajax({
type: 'POST',
url: '/php/check_for_updates.php',
data: data,
async: true, /* If set to non-async, browser shows page as "Loading.."*/
cache: false,
success: function(data) {
try {
// put result data into "obj"
var obj = jQuery.parseJSON(data);
// put the data_from_file into #response
//$('#response').html(obj.data_from_file);
// repeat
console.log("SQL: " + obj['timestamp']);
setTimeout( function() {
// call the function again, this time with the timestamp we just got from server.php
getUpdates(obj['timestamp']);
}, 1000 );
} catch( e ) {
// repeat
// Get mysql formated date
var unix_timestamp = Math.floor(Date.now() / 1000);
console.log("JS: " + unix_timestamp);
setTimeout( function() {
getUpdates(unix_timestamp);
}, 1000 );
}
}
}
);
}
Thanks for all the help guys! I asked around a lot of people and got a bunch of great places to look to debug the code.
I finally found the answer here -
http://blog.preinheimer.com/index.php?/archives/416-PHP-and-Async-requests-with-file-based-sessions.html
http://konrness.com/php5/how-to-prevent-blocking-php-requests/
It looks like I the PHP checking for updates was blocking any updates from happening till the PHP stop checking for updates.
Couple things you can do is:
1.) Open the Chrome Developer tools and then click on the Network tab and clear everything out. Then click on submit. Look at the network tab and see what is being posted and what isn't. Then adjust accordingly from there.
2.) Echo out different steps in your php script and do the same thing with the Network tab and then click on the "results" area and see what's being echoed out and if it's as expected.
From there, you should be able to debug what's happening and figure out where it's going wrong.

getting return data from url

I'm trying to submit user information to a URL using GET, and then get the errors (if there any) and use them to tell the customer what went wrong. So, currently I have a form that submits this customer info into an iframe (so the page is not redirected and I can see the response from my shopping cart software). when the info is submitted, this is the response I get from the shopping cart server:
errorFound=1&responseCode=329&...etc.
I need to get this response code, and was wondering what the most simple way would be to do it. Once I get it I can tell the customer what the problem is... Should I use java to read the
data in the iframe once it loads? or can I use something like Fopen to open the URL and get the return data (can't enable fopen on my server though, but something like it?).
Java != javascript
A quick way to do it:
$errorcodes = array("329" => "Wrong blabla");
if( isset( $_GET['errorFound'] ) && isset( $_GET['responseCode'] ) ){
$errorNr = (int) $_GET['responseCode'];
$error = getErrorFromDB();
//OR
//$error = isset( $erorCodes[ $errorNr ] )? $errorcodes[ $errorNr] : false;
if( $error !== false){
exit( "<script type='text/javascript'>
alert( '".htmlspecialchars($error)."' )
</script>");
}
}
function getError( $code )
{
$code = (int) $code;
$db = getYourPdoInstance();
$q = $db->prepare("SELECT `message` FROM `errorCodes` WHERE `errorCode` = ?");
$q->execute( array( $code) );
$return = $q->fetch(2);
return isset($return['message'])?$return['message']:false;
}

How to automatically update google index when content website has changed

Yesterday, i post an 'ad' on an ebay site (marktplaats.nl) and the ad was immediately visible in google when i search for this ad on Google.
How do they do this?
Because it is a very large site, i don't think they repost the sitemap. I have made such function to update the sitemap but don't know this is a good method. What i have is this (part of my library):
// returns false when input invalid, returns a number (array index) when url is invalid or // request fails, returns true when complete:
function suSubmitSitemap( $sXmlUrl, $asSearchEnginePingUrl = 'http://www.google.com/ping?sitemap=%s', $bContinueOnError = false )
{
if( !suIsValidString( $sXmlUrl ) || ( !suIsValidString( $asSearchEnginePingUrl ) && !suIsValidArray( $asSearchEnginePingUrl )))
{ return false; }
$a = (is_array($asSearchEnginePingUrl))?$asSearchEnginePingUrl:explode(',', $asSearchEnginePingUrl );
$sXmlUrl = urlencode( $sXmlUrl );
$ret = false;
foreach( $a as $i=>$sUrl )
{
$sUri = str_replace( '%s', $sXmlUrl, $sUrl );
$bValid = (!is_bool( strpos( $sUrl, '%s' )) && suGetUrlContent( $sUri ));
if( !$bValid )
{
if( !$bContinueOnError )
{ return $i; }
if( !is_array( $ret ))
{ $ret = array(); }
$ret[$i] = $sUri;
}
}
return ret;
}
Is this a safe way to do this (Google will not ban you when frequently called), when this is not the way to do it, does anyone know how implement such index update functionality in PHP?
Google will constantly crawl frequently updated/popular sites and update their search results. This has the benefit of making Google more relevant.
Resubmitting your site map will not help you get crawled as fast as Ebay. Get more visitors and post content more frequently to get Google to visit your site more often.
Also, check out:
http://www.google.com/support/webmasters/bin/answer.py?answer=48620
http://www.searchenginejournal.com/10-ways-to-increase-your-site-crawl-rate/7159/

Categories