I'm doing QA on this new app I've created using CI. With pagination implemented, what's the typical way of dealing with invalid page numbers? Nothing stops me from manually changing the offset in the URL manually. So if the max is 20, if I modify to 100 what should happen? I'm brainstorming on ways to check if the offset is valid and if it isn't redirect somewhere or display an error message (not sure if I care to do so).
You could do a check on the numbers of results returned and return a 404.
if ($this->page->get_pages(20, 100) {
$this->load->view('our_view');
} else {
show_404();
}
If anyone comes across this later, here's how I tackled the issue.
$max_offset = (ceil($config['total_rows'] / $config['per_page']) - 1) * $config['per_page'];
if (($this->start > $max_offset) && ($max_offset >= 0)) {
redirect("/report/filter");
} else {
$data['records'] = $this->report_model->filterItem($records, false, $config['per_page'], $this->start);
$this->load->view('report_view', $data);
}
Related
I have been using laravel + mysql for my project but and it was working perfectly fine until now. The records keep on increasing and now they have reached almost a million record. Problem is when i try to fetch sms from my database using this query
$smsHistory = SmsLog::where('created_at', '>=', $startDate)->where('created_at', '<=', $endDate)->whereNotNull('gateway_statuscode')->get();
It gives a 500 error without showing anything in error log. What i assume is that as i decreased the time period it gives me record so the problem is the bulk it can not handle. What could be the possible solution as i have to give it today.
I am not worried about the error log.. i want to get a count of those million record but i want to apply some algorithm on it before doing it
This is the check which i have to perform afterwards to see how many sms are in sms log
foreach ($smsHistory as $sms) {
$sms_content = SmsService::_getSmsContent($sms);
if ($sms_content->business_id && array_key_exists($sms_content->business_id, $smsCredits )) {
if (floor(strlen($sms_content->content) / 160) == 0) {
$smsCredits[$sms_content->business_id]['count'] += 1;
}
if (floor(strlen($sms_content->content) / 160) == 1 && floor(strlen($sms_content->content) / 306) == 0) {
$smsCredits[$sms_content->business_id]['count'] += 2;
}
if (floor(strlen($sms_content->content) / 306) == 1 && floor(strlen($sms_content->content) / 459) == 0) {
$smsCredits[$sms_content->business_id]['count'] += 3;
}
if (floor(strlen($sms_content->content) / 459) == 1 && floor(strlen($sms_content->content) / 621) == 0) {
$smsCredits[$sms_content->business_id]['count'] += 4;
}
if (floor(strlen($sms_content->content) / 621) == 1 && floor(strlen($sms_content->content) / 774) == 0) {
$smsCredits[$sms_content->business_id]['count'] += 5;
}
if (floor(strlen($sms_content->content) / 774) == 1 && floor(strlen($sms_content->content) / 927) == 0) {
$smsCredits[$sms_content->business_id]['count'] += 6;
}
}
this is the database field
content is the sms that i have to get and count
Regarding the 500 error:
If you are getting a 500 error, there should hopefully be some clue in the actual server error log (your laravel application error log may not have caught it depending on the error handlers, etc and what the cause was). php_info() should show you the location of the physical error log with the error_log setting:
<?php phpinfo(); ?>
If I had to guess, possibly something memory related that is causing it to choke. But that is just a guess.
As a side question, why are you trying to retrieve so many at once? Can you split them up somehow?
Edit based on updated question:
You may need to use some raw-expressions here to get what you really want: https://www.laravel.com/docs/4.2/queries#raw-expressions
MySQL for example provides the ability to get a column length:
FLOOR(CHAR_LENGTH(content) / 160) as len1, FLOOR(CHAR_LENGTH(content) / 306) as len2, FLOOR(CHAR_LENGTH(content) / 459) as len3
So probably with some magic we could take your query and let the database system do it all for you. And there are probably more efficient ways to do it, if I knew more about the significance of those numbers, but I am trying to help you at least get on one possible path.
You should setup index for mysql, or implement a search engine like elastic search
Im trying to do a verify level function to use on top of my pages, to control access to each page depending on the level of administrator.
I did this function now for this purpose:
function verifyLevel($userId){
$pdo = start();
$read = $pdo->prepare("SELECT * FROM admins where id = :userId");
$read->bindValue(":userId", $userId);
$read->execute();
$result = $read->fetch(PDO::FETCH_ASSOC);
echo $result['level'];
}
And now Im trying to use this function.
Admins have level 1, 2 or 3.
In this page, I want to allow that users with level 1 and 3 can see the content.
Only admins with level 2 can´t see content of this page and I want to show a message saying that they dont have permissions to admins with level 2.
Im trying like this:
if(function_exists('verifyLevel')){
if(verifyLevel($_SESSION['admin']['id'] == '2')){
echo 'Sorry, but you dont have permissions to acess this page.';
}
else{
//I show page content
}
}
But its not working, all levels are having acess to content of this page.
Do you see where is my error?
You're echoing the result from verifyLevel(), you need to return it for it to be comparable:
function verifyLevel($userId){
// ...
return $result['level'];
}
// bracket in wrong place -----------V--not--v
if(verifyLevel($_SESSION['admin']['id']) == '2') {
echo 'Sorry, but you dont have permissions to acess this page.';
}
As mentioned by Jeremy Miller, the way you had your function call set up, it would be passing in a boolean of either true or false, which is loosely equivalent to 0 or 1, so your user would probably not be returned correctly from the database either:
// incorrect comparison here, passing boolean instead (0 or 1)
var_dump(verifyLevel($_SESSION['admin']['id'] == '2'));
Should be:
// true representation of your function's processing and comparison
var_dump(verifyLevel($_SESSION['admin']['id']) == '2');
// verifyLevel($_SESSION['admin']['id']) will either be 1, 2 or 3
We have a web page want to limit uo to 100 people can access concurrently, so we use a memcached to implement a global counter, e.g.
We are using http://www.php.net/manual/en/class.memcache.php so there is not cas, current code is something like
$count = $memcache_obj->get('count');
if ($count < 100) {
$memcache_obj->set('count', $count+1);
echo "Welcome";
} else {
echo "No luck";
}
As you can see there is race condition in the above code and but if we are not going to replace memcached extension which support cas, it is able to support it using PHP code only?
As answer to "emcconville". This is non-blocking even without CAS.
If your concerned about race conditions, and the count value is completely arbitrary, you can use Memcache::increment directly before any business logic.
The increment method will return the current value after the incrementation takes place; of which, you can compare results. Increment will also return false if the key has yet to be set; allowing for your application to deal with it as needed.
$current = $memcache_obj->increment('count');
if($current === false) {
// NOT_FOUND, so let's create it
// Will return false if has just been created by someone else.
$memcache_obj->add('count',0); // <-- no risk of race-condition
// At this point 'count' key is created by us or someone else (other server/process).
// "increment" will update 0 or whatever it is at the moment by 1.
$current = $memcache_obj->increment('count')
echo "You are the $current!";
}
if ($current < 100) {
echo "Hazah! You are under the limit. Congrats!";
} else {
echo "Ah Snap! No Luck - you reached the limit.";
// If your worried about the value growing _too_ big, just drop the value down.
// $memcache_obj->decrement('count');
}
If your concerned about race conditions, and the count value is completely arbitrary, you can use Memcache::increment directly before any business logic.
The increment method will return the current value after the incrementation takes place; of which, you can compare results. Increment will also return false if the key has yet to be set; allowing for your application to deal with it as needed.
$current = $memcache_obj->increment('count');
if($current === false) {
// NOT_FOUND, so let's create it
$memcache_obj->set('count',1); // <-- still risk of race-condition
echo "Your the first!";
} else if ($current < 100) {
echo "Hazah! Your under the limit.";
} else {
echo "Ah Snap! No Luck";
// If your worried about the value growing _too_ big, just drop the value down.
// $memcache_obj->decrement('count');
}
function memcache_atomic_increment($counter_name, $delta = 1) {
$mc = new Memcache;
$mc->connect('localhost', 11211) or die("Could not connect");
while (($mc->increment($counter_name, $delta) === false) &&
($mc->add($counter_name, ($delta<0)?0:$delta, 0, 0) === false)) {
// loop until one of them succeeds
}
return intval($mc->get($counter_name));
}
The comments in Memcache::add include an example locking function, have you tried it out?
Ok, here's what I'm looking for: from a list of links, I'm stripping everything but the domains. The result is a mixed list of domains and domain-names which represent subdomains.
stackoverflow.com
security.stackexchange.com
whoknows.test.co.uk
you.get.it.by.now.dont.you.com
What I want to do is to trim all list entries down to their VALID (=only existing) root domains like this:
stackoverflow.com
security.stackexchange.com
test.co.uk
-fail-
Currently I explode each line into an array and work my list from back to front, using curl to check each potential root domain for it's existance... as soon as curl throws back a HTTP code >= 200 and < 400, I regard the root domain to be found. When the end of each potential domain lookup is done and no valid domain has been found at all, the domain is considered to be non-existant.
input: stackoverflow.com
test: stackoverflow.com - succeeds and is the root domain
result: stackoverflow.com - valid root domain
input: whoknows.test.co.uk
test: co.uk - fails
test: test.co.uk - succeeds (theoretically) and is the root domain
result: test.co.uk - valid root domain
input: you.get.it.by.now.dont.you.com
test: you.com - fails
test: dont.you.com - fails
test: now.dont.you.com - fails
test: by.now.dont.you.com - fails
test: it.by.now.dont.you.com - fails
test: get.it.by.now.dont.you.com - fails
test: you.get.it.by.now.dont.you.com - fails
result: you.get.it.by.now.dont.you.com - invalid domain
Is there any alternative way to do this? I would like to stop heating up my webserver's CPU with 2 to X (=near to unlimited) curl look-ups for every domain on my 100.000+ list. Also, all these lookups take a bunch of time. Maybe - so I hope - there is a quicker solution to do this.
The catch? It has to work with PHP.
There are a bunch of shortcuts to acheive what you need.
For example, you already know that .co.uk and .com are TLDs, so checking these you can obviously skip.
The problem is with all the other crazy TLDs.
I suggest you take a look at the source for ruby-domain-name
They have done a lot of work using RFCs and known data, to try and process it the right way.
So...
I've been fiddling around with this for a long time now, looking for the potential bottlenecks and after a few days of back and forth I discovered that it's actually CURL (that's waiting for the individual servers to respond with a HTTP code) that's making things slower than needed.
In the end, I opted in for a different "gethostbyname" function that takes IP6 into account to solve my problem(s).
function my_gethostbyname($host, $try_a = FALSE)
{
$dns = gethostbynamel6($host, $try_a);
if ($dns == FALSE)
{
return FALSE;
}
else
{
return $dns[0];
}
}
function gethostbynamel6($host, $try_a = FALSE)
{
$dns = array();
$dns6 = #dns_get_record($host, DNS_AAAA);
if($dns6!== FALSE)
{
$dns = array_merge($dns, $dns6);
}
if ($try_a == TRUE)
{
$dns4 = #dns_get_record($host, DNS_A);
if($dns4!== FALSE)
{
$dns = array_merge($dns, $dns4);
}
}
else
{
$dns = $dns6;
}
$ip6 = array();
$ip4 = array();
foreach ($dns as $record)
{
if ($record["type"] == "A")
{
$ip4[] = $record["ip"];
}
if ($record["type"] == "AAAA")
{
$ip6[] = $record["ipv6"];
}
}
if (count($ip6) < 1)
{
if ($try_a == TRUE)
{
if (count($ip4) < 1)
{
return FALSE;
}
else
{
return $ip4;
}
}
else
{
return FALSE;
}
}
else
{
return $ip6;
}
}
As soon as the first domain-part actually resolves to an IP, (a) the domain exists and (b) is the root domain.
This spares me major time and the trick of this is that you're only as slow as your DNS resolution and some microsecs. The curl option I used before took around 3 seconds per call (sometimes up to the full timeout range I had set to 20 secs), depending on the target server's response time - if any.
The path I now chose is easy to understand: I end up with a list of resolving domains and - when needed - I can check those using curl "on demand" or using one or more CRON jobs "on interval".
I know that it's kind of a workaround, but splitting the problem into two tasks (1 = pre-check for root domain, 2 = check if domain returns expected HTTP code) makes the whole thing faster than trying to do the complete job at once using curl.
What I've learned from this...
When checking domains, try to resolve them first so you can spare yourself the timeout burden that comes with curl.
Curl is great for many tasks, but it's not the smartest way to try to do everything with it.
When you think you can't solve a problem more than you've tried to do, split the problem in two or more parts and check again. Chances are big that you'll discover a whole new world of options to enhance what you've got.
I hope this spares someone the burden of fiddling around with an alike problem for weeks. ;)
class DomainUtils {
function __construct(){
//only these super domains
$this->superDomains = array(
'.com',
'.gov',
'.org',
'.co.uk',
'.info',
'.co',
'.net',
'.me',
'.tv',
'.mobi'
);
}
//get super domain
public function getMainDomain($domain = null){
$domainChunks = explode('.', str_ireplace($this->superDomains, '', $domain));
if(sizeof($domainChunks) == 0){
return false;
}
foreach($domainChunks as $key => $domainChunk){
if($key < sizeof($domainChunks) - 1){
$domain = str_ireplace($domainChunk . '.', '', $domain);
}
}
return $domain;
}
}
Okay normally I'm all fine about the facebook API but I'm having a problem which just keeps me wondering. (I think it's a bug (Check ticket http://bugs.developers.facebook.net/show_bug.cgi?id=13694) but I wanted to throw it here if somebody has an idea).
I'm usng the facebook PHP library to count all attendees for a specific event
$attending = $facebook->api('/'.$fbparams['eventId'].'/attending');
this works without a problem it correctly returns an array with all attendees...
now heres the problem:
This event has about 18.000 attendees right now.
The api call returns a max number of 992 attendees (and not 18000 as it should).
I tried
$attending = $facebook->api('/'.$fbparams['eventId'].'/attending?limit=20000');
for testing but it doesn't change anything.
So my actual question is:
If I can't get it to work by using the graph api what would be a good alternative? (Parsing the html of the event page maybe?) Right now I'm changing the value by hand every few hours which is tedious and unnecessary.
Actually there are two parameters, limit and offset. I think that you will have to play with both and continue making calls until one returns less than the max. limit.
Something like this, but in a recursive approach (I'm writting pseudo-code):
offset = 0;
maxLimit = 992;
totalAttendees = count(result)
if (totalAttendees >= maxLimit)
{
// do your stuff with each attendee
offset += totalAttendees;
// make a new call with the updated offset
// and check again
}
I've searched a lot and this is how I fixed it:
The requested URL should look something like this.
Here is where you can test it and here is the code I used:
function events_get_facebook_data($event_id) {
if (!$event_id) {
return false;
}
$token = klicango_friends_facebook_token();
if ($token) {
$parameters['access_token'] = $token;
$parameters['fields']= 'attending_count,invited_count';
$graph_url = url('https://graph.facebook.com/v2.2/' . $event_id , array('absolute' => TRUE, 'query' => $parameters));
$graph_result = drupal_http_request($graph_url, array(), 'GET');
if(is_object($graph_result) && !empty($graph_result->data)) {
$data = json_decode($graph_result->data);
$going = $data->attending_count;
$invited = $data->invited_count;
return array('going' => $going, 'invited' => $invited);
}
return false;
}
return false;
}
Try
SELECT eid , attending_count, unsure_count,all_members_count FROM event WHERE eid ="event"