Codeigniter Rest Server fail to output longer JSON - php

Does anybody know why
$this-response($somearray, 200);
Can't output a long JSON? (Or long arrays)
For example I can output DB result if I limit query to last 10 rows but if I limit it to 100
JSON gets trimed and in my client app I get 'unexpected end of input'. If I change above statement in simple
echo json_encode($somearray);
It works flawlessly.
I am sorry. I need to clarify my question. I am using
https://github.com/chriskacerguis/codeigniter-restserver
function transits_get()
{
$sessionId = $this->input->get_request_header('Sessionid', TRUE);
$transits = $this->Transit_model->get_transits( $sessionId );
if($transits)
{
//$this->output->set_content_type('application/json');
//$this->output->set_output(json_encode($transits)); //works
//echo json_encode($transits); //works
$this->response($transits, 200); // 200 being the HTTP response code //Does not work
}
else
{
$this->response(array('error' => 'There is no transit records in the database!'), 404);
}
}

Alright, I've found what was the problem.
I simply commented out the code in REST_controller that sets content length.
Now it works but I am still wondering why strlen($output) is not as same length as output JSON for larger JSON files and it is ok for short ones.
// If zlib.output_compression is enabled it will compress the output,
// but it will not modify the content-length header to compensate for
// the reduction, causing the browser to hang waiting for more data.
// We'll just skip content-length in those cases.
if ( ! $this->_zlib_oc && ! $this->config->item('compress_output')) {
//header('Content-Length: ' . strlen($output));
}

This works for me, and I have lots of JSON data (+- 250 records):
$this->output->set_content_type('application/json');
$this->output->set_output(json_encode($data));
As found in this article, you can do the following:
Open REST_Controller.php
Locate and delete the following code (at the end of the response function):
if ( ! $this->_zlib_oc && ! $CFG->item('compress_output')) {
header('Content-Length: ' . strlen($output));
}

In libraries folder, line no. 267 of REST_Controller.php file, comment the following code. it's working.
header('Content-Length: ' . strlen($output));

Related

Smarty returns an empty result every now and then

I am facing a weird problem using Smarty. I am generating an email's body through a template. Most times it works as expected, but from time to time, the returned data is empty. However, I do not see any error in my logs, neither I catch any exception. It is just as if the template was empty.
This is the piece of code I am using to get the email's body:
// $data is an array with template's data
// $tpl is the template's path
$s = new Smarty();
$s->assignArray( $data );
try {
$body = $s->fetch( $tpl );
} catch ( \Exception $e ) {
Debug::Log( $e->getMessage() );
}
// Sometimes $body is empty, but no exception is thrown.
I checked that the template has no errors, after all, it works in most cases.
I also saved $data contents when $body is empty and I ran the code manually to get $body content, but it worked, so I do not think the problem is related to template vars.
Another test I did is to try to process the template up to 5 times, sleeping for a second between the tries, but the result was always empty.
The template's cache path is writable.
I am using PHP 5.6.40, Smarty 3.1.21 and Apache2.
Can you give me a hand to debug this issue?
Update
I have been able to reproduce the problem. Smarty always returns an empty result whenever the fetch method is called after PHP detected that the client closed the connection. For example, take this code:
ignore_user_abort(1); // Continue running even if the connection is closed
set_time_limit(180); // 3 minutes
$s = new Smarty();
$s->assignArray( $data );
// Keep writing data untill PHP realises that connection was closed
while( 1 ) {
if(connection_status() != CONNECTION_NORMAL || connection_aborted( ) ) {
break;
}
echo "123456789";
}
$body = $s->fetch( $tpl );
if ( '' == $body ) {
throw new Exception("Result is empty");
}
die('Code never reaches this point');
If I call the script above and I close the connection immediately, the result of the fetch method is always empty.
However, if PHP did not detect that the connection was closed, even though it really was, the result of fetch is not empty.
ignore_user_abort(1); // Continue running even if the connection is closed
set_time_limit(180); // 3 minutes
$s = new Smarty();
$s->assignArray( $data );
// Sleep to make sure the connection was closed
// PHP do not realise the connection is closed untill it tries to write something
sleep( 60);
$body = $s->fetch( $tpl );
if ( '' == $body ) {
throw new Exception("Result is empty");
}
echo "Now the result is not empty";
This is the code I used to call the above scripts:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://myhost/test.php');
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 1);
curl_exec($ch);
curl_close($ch);
echo "all done";
This seems to be related to this question: PHP ob_get_contents "sometimes" returns empty when it should not?
My script does a lot of things so it takes quite a long time to finish. Some users close their browser before the script finished, and that is when Smarty returns an empty result, as it uses ob_start a lot.
Best wishes,
blanking the page without exceptions seems to be the way smarty does everything..
im not familiar with python, however, i suspect that only exceptions are thrown. not notices. you might try at the end of your code to check of thrown notices or other warning, hoever it is called in python.
it might still be a folder persmission, have you also checked that the templates_c directory exists, and has opermissions? or any {var.name} without $.
it can be anything, smarty nevers throws exceptions, it just blanks the page.
if it still does not help, crate a basic overly-simplified template, and try that for some time to see if it still happens. if it does, it is a mistake in your template.
As far as I'm concerned, it turns out to be a bug in PHP 5.6. I made some tests using print_r with the return flag set to true, and the result after closing the connection was never empty with PHP 7.0 and PHP 8.0. However, when I used PHP 5.6 the result was empty.
Example:
<?php
error_reporting( E_ALL );
ini_set('display_errors', 1);
ignore_user_abort(true);// (curl disconnects after 1 second)
ini_set('max_execution_time','180'); // 3 minutes
ini_set('memory_limit','512M'); // 512 MB
function testPrint_r($length)
{
$test1 = array('TEST'=>'SOMETHING');
$test2 = print_r($test1, true);
$test3 = "Array\n(\n [TEST] => SOMETHING\n)\n";
if(strcmp($test2, $test3)!==0) {
throw new Exception("Print_r check failed, output length so far: ".$length);
// consult your error.log then, or use some other reporting means
}
}
$message = "123456789\n";
$length = strlen($message);
$total_length = 0;
while(1)
{
echo $message;
$total_length += $length;
testPrint_r($total_length);
}
die('it should not get here');
Using PHP 5.6, if you call the script and close the connection, the Exception is thrown because print_r returns an empty result. However, using PHP 7.0 or PHP 8.0 the script keeps running until it reaches the maximun execution time.
Kind regards,

Output json to php (frontend)

Here is the output as its beeing displayed right now in my index file (on the wesbite)
So this is what i have right now
https://gyazo.com/e3b5efb2c0ac9c7225c5c322ae92b0e9
And i want to have that displayed more nicley ( perhaps in a table with header from and then text )
That is my problem, im usure of what i can google or search for. Get the data that is saved in my json file. And Display it nicley in my index.
Get some of the values from my .json file and display them in my index.
Here is my php code.
$outfile= 'result.json';
$url='https://newsapi.org/v2/top-headlines?country=us&category=business&apiKey=MYAPIKEY';
$json = file_get_contents($url);
if($json) {
if(file_put_contents($outfile, $json, FILE_APPEND)) {
echo "Saved JSON fetched from “{$url}” as “{$outfile}”.";
}
else {
echo "Unable to save JSON to “{$outfile}”.";
}
}
else {
echo "Unable to fetch JSON from “{$url}”.";
}
$option = $_GET['option'];
And here its how i's getting displayed in my .json file.
{"status":"ok","totalResults":20,"articles":[{"source":{"id":"cnn","name":"CNN"},"author":"Matt McFarland","title":"Has Elon Musk lost control of his hype machine?","description":"For years, the entrepreneur controlled public opinion of Tesla. That's changing.","url":"https://money.cnn.com/2018/07/26/technology/elon-musk-tesla/index.html",
You might have invalid json response. verify first whether do you have valid json. May be copy your response and check it in some validator website like www.jsonlnt.com
if json is fine then PHP function json_decode($you_json_string_php_variable); will work fine.

file_get_contents() returns extra underscore on angular.callbacks

I built a PHP file with the sole purpose of hiding the API keys for Google Search, but part of the file_get_contents() always echo angular.callbacks._0_({ instead of angular.callbacks._0({
This small change makes the rest of the response worthless as Angular throws Uncaught TypeError: angular.callbacks._0_ is not a function. Although the workaround does works flawlessly, I would like to know if someone found the root of this issue or a better solution that is strictly PHP (no curl or any other package.)
search.php
<?php // Created by Deovandski on 2/14/2016
header('Content-type: application/json');
# Setup Base URL and array for Parameters
$host = 'https://www.googleapis.com/customsearch/v1?';
$queries = array();
$queries['cx'] = "XXX";// CSE KEY
$queries['key'] = "XXX"; // API KEY
# Setup possible incoming params
if (isset($_GET['search_term'])) $queries['q'] = $_GET['search_term'];
if (isset($_GET['result_count'])) $queries['result_count'] = $_GET['result_count'];
if (isset($_GET['callback'])) $queries['callback'] = $_GET['callback'];
# Build query and Final URL
$queriesURL = http_build_query($queries) . "\n";
$finalURL = $host.$queriesURL;
echo $finalURL;
/* echo $finalURL output (I only edited the keys out):
https://www.googleapis.com/customsearch/v1?cx=XXX&key=XXX&q=Hatsune+Miku&result_count=10&callback=angular.callbacks._0
*/
// Setup Response
$response = file_get_contents($finalURL);
// workaround
$fixedResponse = str_replace("angular.callbacks._0_", "angular.callbacks._0", $response);
echo $fixedResponse;
?>
This is part of a correct Google API response:
// API callback
angular.callbacks._0({
"kind": "customsearch#search",
"url": {
"type": "application/json",
"template": "https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&lr={language?}&safe={safe?}&cx={cx?}&cref={cref?}&sort={sort?}&filter={filter?}&gl={gl?}&cr={cr?}&googlehost={googleHost?}&c2coff={disableCnTwTranslation?}&hq={hq?}&hl={hl?}&siteSearch={siteSearch?}&siteSearchFilter={siteSearchFilter?}&exactTerms={exactTerms?}&excludeTerms={excludeTerms?}&linkSite={linkSite?}&orTerms={orTerms?}&relatedSite={relatedSite?}&dateRestrict={dateRestrict?}&lowRange={lowRange?}&highRange={highRange?}&searchType={searchType}&fileType={fileType?}&rights={rights?}&imgSize={imgSize?}&imgType={imgType?}&imgColorType={imgColorType?}&imgDominantColor={imgDominantColor?}&alt=json"
},
I put up a live version of this issue that can be seen on my FTP server. The PHP file can be viewed through this link (AngularJS parameters included on it).
The problem is the escape sequences \n. Which is passed as part of the request. And which is interpreted as space and as part of the callback function name and replaced by the side of the API to underline.
To understand just try this option and look at the result:
$queriesURL = http_build_query($queries) . "\n" . "after";
So just take away a newline.

Twitter API always saying 400 Bad Request

I am using the following code to retrieve an amount of Tweets from the Twitter API:
$cache_file = "cache/$username-twitter.cache";
$last = filemtime($cache_file);
$now = time();
$interval = $interval * 60; // ten minutes
// Check the cache file age
if ( !$last || (( $now - $last ) > $interval) ) {
// cache file doesn't exist, or is old, so refresh it
// Get the data from Twitter JSON API
//$json = #file_get_contents("http://api.twitter.com/1/statuses/user_timeline.json?screen_name=" . $username . "&count=" . $count, "rb");
$twitterHandle = fopen("http://api.twitter.com/1/statuses/user_timeline.json?screen_name=$username&count=$count", "rb");
$json = stream_get_contents($twitterHandle);
fclose($twitterHandle);
if($json) {
// Decode JSON into array
$data = json_decode($json, true);
$data = serialize($data);
// Store the data in a cache
$cacheHandle = fopen($cache_file, 'w');
fwrite($cacheHandle, $data);
fclose($cacheHandle);
}
}
// read from the cache file with either new data or the old cache
$tweets = #unserialize(file_get_contents($cache_file));
return $tweets;
Of course $username and the other variables inside the fopen request are correct and it produces the correct URL because I get the error:
Warning: fopen(http://api.twitter.com/1/statuses/user_timeline.json?screen_name=Schodemeiss&count=5) [function.fopen]: failed to open stream: HTTP request failed! HTTP/1.1 400 Bad Request in /home/ellexus1/public_html/settings.php on line 187
that ^^ error returns whenever I try and open my page.
Any ideas why this might be? Do I need to use OAuth to even just get my tweets!? Do I have to register my website as somewhere that might get posts?
I'm really not sure why this is happening. My host is JustHost.com, but I'm not sure if that makes any diffrence. All ideas are welcome!
Thanks.
Andrew
PS. This code lies inside a function where username, interval and count are passed in correctly, hence in the error code its created a well formed address.
Chances are you are getting rate-limited
400 Bad Request: The request was invalid. An accompanying error
message will explain why. This is the status code will be returned
during rate limiting.
150 requests per hour for non authenticated calls (Based on IP-addressing)
350 requests per hour for authenticated calls (Based on the authenticated users calls)
You have to authenticate to avoid these errors popping up.
And also please use cURL when dealing with twitter. I've used file_get_contents and fopen to call the twitter API, and found that it is very unreliable. You would get hit with that every now and then.
Replace the fopen with
$ch = curl_init("http://api.twitter.com/1/statuses/user_timeline.json?screen_name=$username&count=$count");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$it = curl_exec($ch); //content stored in $it
curl_close($ch);
This may help
Error codes
https://developer.twitter.com/en/docs/basics/response-codes.html
Error codes defination is given in above link

PHP Zend Gdata error 400 when inserting row in spreadsheet

I'm doing an small class to load spreadsheets and insert data in them but when I insert a row i retrieve an error:
Expected response code 200, got 400 We're sorry, a server error
occurred. Please wait a bit and try reloading your spreadsheet.
I have looking in this site and google and some people has this error when they trying to insert data with capital letters or spaces... i'm not trying to insert data like this so i suppose that this is not my error. I the code that is giving me problems:
try {
echo $this->_spreadId . '<br>';
echo $this->_defaultWorksheetId . '<br>';
echo get_class($this->_spreadClient) . '<br>';
$rowData = array('stuff' => 'smurf');
$this->_spreadClient->insertRow($rowData, $this->_spreadId, $this->_defaultWorksheetId);
} catch (Exception $exc) {
echo '<br><pre>' . $exc->getTraceAsString() . '</pre>';
echo '<br>' . $exc->getMessage();
}
The output of this script is:
tVe4Mr82qD3LRhubQrcLxMQ
od6
Zend_Gdata_Spreadsheets
#0 /home/sergi/projects/quiniela2gdocs/lib/Zend/Gdata.php(219): Zend_Gdata_App->performHttpRequest('POST', 'https://spreads...', Array, 'performHttpRequest('POST', 'https://spreads...', Array, 'post('insertEntry('insertRow(Array, 'tVe4Mr82qD3LRhu...', 'od6')
Expected response code 200, got 400 We're sorry, a server error occurred. Please wait a bit and try reloading your spreadsheet.
Another method of this class is creating the spreadsheet and it's doing well, so i think there aren't connecting issues with gDocs.
Some help is welcome.
Lots of thanks !!
I had a similar problem which has arisen due to the presence of spaces in the array. Remove spaces helped me:
$rowData = array('stuff' => 'smurf'); - your code
$rowData = array('stuff'=>'smurf'); - my code

Categories