Yii2 cUrl Bad Request (#400) - php

While trying to use cUrl with the Post method in Yii 2, I receive a 400 error code.
Bad Request (#400)
Unable to verify your data submission.
The above error occurred while the Web server was processing your request.
Please contact us if you think this is a server error. Thank you.
This is my code, where i instantiate the CurlTool class:
public function actionSend() {
$model = new \app\models\Licitatie;
if ($model->load(Yii::$app->request->post())) {
$curl_tool = new \common\components\CurlTool();
$result = $curl_tool->fetchContent('http://www.william.ro/licitatia_bursa/frontend/web/index.php/organizator/licitatie/evrika', $model->attributes);
print_r($result);
}
}
public function actionEvrika() {
return json_encode(
array(
'a' => 'b',
)
);
}
this is the curltool class code:
<?php
namespace common\components;
class CurlTool {
public static $userAgents = array(
'FireFox3' => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9) Gecko/2008052906 Firefox/3.0',
'GoogleBot' => 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
'IE7' => 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)',
'Netscape' => 'Mozilla/4.8 [en] (Windows NT 6.0; U)',
'Opera' => 'Opera/9.25 (Windows NT 6.0; U; en)'
);
public static $options = array(
CURLOPT_USERAGENT => 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)',
CURLOPT_AUTOREFERER => true,
CURLOPT_FOLLOWLOCATION => false,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_FRESH_CONNECT => true,
CURLOPT_COOKIEJAR => "cookies.txt",
CURLOPT_COOKIEFILE => "cookies.txt",
CURLOPT_SSL_VERIFYPEER => false,
//CURLOPT_COOKIESESSION => false,
);
private static $proxyServers = array();
private static $proxyCount = 0;
private static $currentProxyIndex = 0;
public static $getinfo;
public static function addProxyServer($url) {
self::$proxyServers[] = $url;
++self::$proxyCount;
}
public static function fetchContent($url, $fields = null, $verbose = false) {
//print '*'.$fields.'*';
if (($curl = curl_init($url)) == false) {
throw new Exception("curl_init error for url $url.");
}
if (self::$proxyCount > 0) {
$proxy = self::$proxyServers[self::$currentProxyIndex++ % self::$proxyCount];
curl_setopt($curl, CURLOPT_PROXY, $proxy);
if ($verbose === true) {
echo "Reading $url [Proxy: $proxy] ... ";
}
} else if ($verbose === true) {
echo "Reading $url ... ";
}
//$verbose=TRUE;
//print_r($fields);
// debug_print_backtrace();
//url-ify the data for the POST
$fields_string = '';
if (is_array($fields))
foreach ($fields as $key => $value) {
if (empty($key))
continue;
$fields_string .= $key . '=' . urlencode($value) . '&';
if ($verbose === true) {
echo $key . ": " . $value;
}
}
rtrim($fields_string, '&');
if (count($fields) > 0) {
curl_setopt($curl, CURLOPT_POST, count($fields));
curl_setopt($curl, CURLOPT_POSTFIELDS, $fields_string);
}
if ($verbose === true) {
echo "Fields string $fields_string ... ";
}
curl_setopt_array($curl, self::$options);
$content = curl_exec($curl);
self::$getinfo = curl_getinfo($curl);
if ($content === false) {
throw new Exception("curl_exec error for url $url " . curl_error($curl));
}
curl_close($curl);
if ($verbose === true) {
echo "Done.\n";
}
$content = preg_replace('#\n+#', ' ', $content);
$content = preg_replace('#\s+#', ' ', $content);
return $content;
}
}

class Controller extends \yii\base\Controller
{
/**
* #var boolean whether to enable CSRF validation for the actions in this controller.
* CSRF validation is enabled only when both this property and [[Request::enableCsrfValidation]] are true.
*/
public $enableCsrfValidation = false; <- set this to false
...
be careful, i just found out that if this setting is used within the action itself, it might fail;
it might fail within beforeaction;

you can disable csrf in beforeaction
public function beforeAction($action)
{
if($action->id == 'source-in')
{
return true;
}
return parent::beforeAction($action);
}

Is there another way to fix it? Maybe generate a new CSRF token and send it in the POST data?

Related

PHP class method with closure CURL headers

I have the class below, in order to analyze a URL
<?php
class URLFetcher {
private $ch;
private $url = '';
public function __construct(string $url) {
$this->url = $url;
}
public function fetch(): URLFetcher {
$headers = [];
$this->ch = curl_init($this->url);
curl_setopt_array($this->ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0',
CURLOPT_HEADERFUNCTION => function($curl, $header) use (&$headers) {
var_dump($header);
$header2 = trim($header);
if($header2 !== '') $headers[] = $header2;
return strlen($header);
}
]);
exit(var_dump($headers));
return $this;
}
}
?>
I use var_dump in the closure in order to see the headers, as they come from CURL, and at the end, I use var_dump again in order to see all headers.
The problem is that there are no headers printed in the function and at the end, the $headers is printing the empty array.
If I do this without the class, everything works correctly. My question is what am I missing and the code does not work? I hypothesize that it has to do with the closure.
You simply forgot curl_exec($this->ch); at the end of the method.
public function fetch(): URLFetcher {
$headers = [];
$this->ch = curl_init($this->url);
curl_setopt_array($this->ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0',
CURLOPT_HEADERFUNCTION => function($curl, $header) use (&$headers) {
$header2 = trim($header);
if($header2 !== '') $headers[] = $header2;
return strlen($header);
}
]);
curl_exec($this->ch);
exit(var_dump($headers));
return $this;
}

Invalid cookies recieved from CURL request and file_get_contents

I am receiving an invalid cookie string when trying capture the cookie using file_get_contents and curl. The cookie received while browsing directly from the browser is valid/active. But, the cookie captured from file_get_contents and curl seems to be invalid.
I am trying to capture from file_get_contents like this
$context = array(
'http' => array(
'method' => 'GET',
'header' => array('Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*\/*;q=0.8', 'User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36'),
)
);
$cxContext = stream_context_create($context);
file_get_contents($url, false, $cxContext);
$cookies = array();
foreach ($http_response_header as $hdr) {
if (preg_match('/^Set-Cookie:\s*([^;]+)/', $hdr, $matches)) {
$cookies = $matches[1];
}
}
return $cookies;
I tried playing around with this, by setting headers, but the cookies returned always is either expired or simply invalid.
But, through a browser the cookie I get is always valid.
Anyone faced a similar problem, don't know how to tackle this issue.
There are several unanswered questions from my above comment, but I'll share this bit of code for example purposes. It's what I've used in the past as a base class for browser emulation using cURL:
<?php
if(!function_exists("curl_init")) { throw new Exception("CurlBrowser requires the cURL extension, which is not enabled!"); }
class CurlBrowser
{
public $userAgent = "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0";
/*
Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1");
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0
*/
public $cookiesFile = null;
public $proxyURL = null;
public $saveLastOutput = "";
public $caBundle = "cacert.pem";
public $httpHeaders = array();
public function __construct($UseCookies = true)
{
if(is_bool($UseCookies) && $UseCookies)
{
$this->cookiesFile = dirname(__FILE__)."/cookies.txt";
}
elseif(is_string($UseCookies) && ($UseCookies != ""))
{
$this->cookiesFile = $UseCookies;
}
}
public function SetCustomHTTPHeaders($arrHeaders)
{
$this->httpHeaders = $arrHeaders;
}
public function SetProxy($proxy)
{
$this->proxyURL = $proxy;
}
public function Get($url)
{
return $this->_request($url);
}
public function Post($url,$data = array())
{
return $this->_request($url,$data);
}
private function _request($form_url,$data = null)
{
$ch = curl_init($form_url);
// CA bundle
$caBundle = $this->caBundle;
if(file_exists($caBundle))
{
// Detect and convert relative path to absolute path
if(basename($caBundle) == $caBundle)
{
$caBundle = getcwd() . DIRECTORY_SEPARATOR . $caBundle;
}
// Set CA bundle
curl_setopt($ch, CURLOPT_CAINFO, $caBundle);
}
// Cookies
if($this->cookiesFile !== null)
{
curl_setopt($ch, CURLOPT_COOKIEFILE, $this->cookiesFile);
curl_setopt($ch, CURLOPT_COOKIEJAR, $this->cookiesFile);
}
// User Agent
curl_setopt($ch, CURLOPT_USERAGENT, $this->userAgent);
// Misc
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_ENCODING, "gzip, deflate");
// Optional proxy
if($this->proxyURL !== null)
{
curl_setopt($ch, CURLOPT_PROXY, $this->proxyURL);
}
// Custom HTTP headers
if(count($this->httpHeaders))
{
curl_setopt($ch, CURLOPT_HTTPHEADER, $this->httpHeaders);
}
// POST data
if($data !== null)
{
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
}
// Run operation
$result = curl_exec($ch);
if($result === false)
{
throw new Exception(curl_error($ch));
}
else
{
if(!empty($this->saveLastOutput))
{
file_put_contents($this->saveLastOutput,$result);
}
return $result;
}
}
}
?>
You'd use it like so:
<?php
$browser = new CurlBrowser();
$html = $browser->Get("https://....");
...etc...
My gut guess is that you're simply missing a cookie jar in your original code, but that's mostly based on gut feeling, since we don't have all your problem code at this time.

PHP Bind Json api Data to table

I am connecting to a API which doesnt really have a lot of documentation: I would like to bind the data returned to a table but at the moment its all not even loading...Im rather new at this... Im just not sure how to actually bind the data...Thanks!
Example response from docs:
{
"service":"1",
"name":"Custom Comments",
"type":"Custom Comments"
}
HTML
<div class="col-sm-9">
<?php $api = new Api(); ?>
<?php $services = $api->services(); # return all services ?>
</div><!-- col-sm-9 -->
API CLASS
<?php
class Api
{
public $api_url = 'http://apiurl'; // API URL
public $api_key = 'key'; // Your API key
public function order($data) { // add order
$post = array_merge(array('key' => $this->api_key, 'action' => 'add'), $data);
return json_decode($this->connect($post));
}
public function status($order_id) { // get order status
return json_decode($this->connect(array(
'key' => $this->api_key,
'action' => 'status',
'id' => $order_id
)));
}
public function services() { // get services
return json_decode($this->connect(array(
'key' => $this->api_key,
'action' => 'services',
)));
}
private function connect($post) {
$_post = Array();
if (is_array($post)) {
foreach ($post as $name => $value) {
$_post[] = $name.'='.urlencode($value);
}
}
$ch = curl_init($this->api_url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
if (is_array($post)) {
curl_setopt($ch, CURLOPT_POSTFIELDS, join('&', $_post));
}
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)');
$result = curl_exec($ch);
if (curl_errno($ch) != 0 && empty($result)) {
$result = false;
}
curl_close($ch);
return $result;
}
}

php fopen silently failing

I'm haveing trouble opening this url useing fopen
http://sim8430.agni.lindenlab.com:12046/cap/ccab53e3-49d8-c4f8-b3ad-51ff22c1fa17/
however it silently fails for some reason, it opens other urls just fine but no urls like the one above
the fact that it doesnt give an error is not the real problem, the not loading is what im trying to fix
$file = fopen("http://sim8430.agni.lindenlab.com:12046/cap/ccab53e3-49d8-c4f8-b3ad-51ff22c1fa17/", "r") or exit("Unable to open file!");
catch it like this instead
// add extensive logging
error_reporting(E_ALL);
$file = fopen("http://sim8430.agni.lindenlab.com:12046/cap/xyz/", "r");
if($file === false) {
// try through socket open
$file = fsockopen("sim8430.agni.lindenlab.com", 12046, $errno, $errstr);
if($file == false && $errno == 0) die("Socket initialization failed");
else if(!$file) {
die($errstr);
} else {
$out = "GET /cap/xyz/ HTTP/1.1\r\n";
$out .= "Host: sim8430.agni.lindenlab.com\r\n";
$out .= "Connection: Close\r\n\r\n";
fwrite($file, $out);
}
}
if($file) {
while (!feof($file)) {
$contents = fgets($file, 128);
}
fclose($file);
}
Always use cURL when reading/parsing a file content that is on the WWW server.
<?php
class CurlTool {
public static $userAgents = array(
'FireFox3' => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9) Gecko/2008052906 Firefox/3.0',
'GoogleBot' => 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
'IE7' => 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)',
'Netscape' => 'Mozilla/4.8 [en] (Windows NT 6.0; U)',
'Opera' => 'Opera/9.25 (Windows NT 6.0; U; en)'
);
public static $options = array(
CURLOPT_USERAGENT => 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)',
CURLOPT_AUTOREFERER => true,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_FRESH_CONNECT => true,
CURLOPT_COOKIEJAR => "cookies.txt",
CURLOPT_COOKIEFILE => "cookies.txt",
CURLOPT_SSL_VERIFYPEER => false,
//CURLOPT_COOKIESESSION => false,
);
private static $proxyServers = array();
private static $proxyCount = 0;
private static $currentProxyIndex = 0;
public static $getinfo;
public static function addProxyServer($url) {
self::$proxyServers[] = $url;
++self::$proxyCount;
}
public static function fetchContent($url, $fields = null, $verbose = false) {
//print '*'.$fields.'*';
if (($curl = curl_init($url)) == false) {
throw new Exception("curl_init error for url $url.");
}
if (self::$proxyCount > 0) {
$proxy = self::$proxyServers[self::$currentProxyIndex++ % self::$proxyCount];
curl_setopt($curl, CURLOPT_PROXY, $proxy);
if ($verbose === true) {
echo "Reading $url [Proxy: $proxy] ... ";
}
} else if ($verbose === true) {
echo "Reading $url ... ";
}
//$verbose=TRUE;
//print_r($fields);
// debug_print_backtrace();
//url-ify the data for the POST
$fields_string = '';
if (is_array($fields))
foreach ($fields as $key => $value) {
if (empty($key))
continue;
$fields_string .= $key . '=' . urlencode($value) . '&';
if ($verbose === true) {
echo $key . ": " . $value;
}
}
rtrim($fields_string, '&');
if (count($fields) > 0) {
curl_setopt($curl, CURLOPT_POST, count($fields));
curl_setopt($curl, CURLOPT_POSTFIELDS, $fields_string);
}
if ($verbose === true) {
echo "Fields string $fields_string ... ";
}
curl_setopt_array($curl, self::$options);
$content = curl_exec($curl);
self::$getinfo = curl_getinfo($curl);
if ($content === false) {
throw new Exception("curl_exec error for url $url " . curl_error($curl));
}
curl_close($curl);
if ($verbose === true) {
echo "Done.\n";
}
$content = preg_replace('#\n+#', ' ', $content);
$content = preg_replace('#\s+#', ' ', $content);
return $content;
}
public static function downloadFile($url, $fileName, $fields = null, $verbose = false) {
if (($curl = curl_init($url)) == false) {
throw new Exception("curl_init error for url $url.");
}
if (self::$proxyCount > 0) {
$proxy = self::$proxyServers[self::$currentProxyIndex++ % self::$proxyCount];
curl_setopt($curl, CURLOPT_PROXY, $proxy);
if ($verbose === true) {
echo "Downloading $url [Proxy: $proxy] ... ";
}
} else if ($verbose === true) {
echo "Downloading $url ... ";
}
//url-ify the data for the POST
$fields_string = '';
if (is_array($fields))
foreach ($fields as $key => $value) {
if (empty($key))
continue;
$fields_string .= $key . '=' . urlencode($value) . '&';
}
rtrim($fields_string, '&');
curl_setopt($curl, CURLOPT_POST, count($fields));
curl_setopt($curl, CURLOPT_POSTFIELDS, $fields_string);
curl_setopt_array($curl, self::$options);
if (is_file($fileName)) {
$contents = file_get_contents($fileName, false, null, -1, 3 * 1024);
$pattern = "__VIEWSTATE";
if (strpos($contents, $pattern) === false) {
return $fileName;
}
}
// if (is_file($fileName)) {
// // make a HEAD request and try to get the file size HEAD
// // if they differ then redownload the file, otherwise no need
// curl_setopt($curl, CURLOPT_NOBODY, true);
// curl_setopt($curl, CURLOPT_HEADER, true);
// $ret = curl_exec($curl);
// //echo $fileName;
// $size = curl_getinfo($curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD);
// if ($size == filesize($fileName)) {
// return $fileName;
// } else {
// unlink($fileName);
// return self::downloadFile($url, $fileName, $fields, $verbose);
// }
// }
if (substr($fileName, -1) == '/') {
$targetDir = $fileName;
$fileName = tempnam(sys_get_temp_dir(), 'c_');
}
if (($fp = fopen($fileName, "w")) === false) {
throw new Exception("fopen error for filename $fileName");
}
curl_setopt($curl, CURLOPT_FILE, $fp);
curl_setopt($curl, CURLOPT_BINARYTRANSFER, true);
$ret = curl_exec($curl);
self::$getinfo = curl_getinfo($curl);
if ($ret === false) {
fclose($fp);
unlink($fileName);
throw new Exception("curl_exec error for url $url.");
} elseif (isset($targetDir)) {
$eurl = curl_getinfo($curl, CURLINFO_EFFECTIVE_URL);
preg_match('#^.*/(.+)$#', $eurl, $match);
fclose($fp);
rename($fileName, "$targetDir{$match[1]}");
$fileName = "$targetDir{$match[1]}";
} else {
fclose($fp);
}
curl_close($curl);
if ($verbose === true) {
echo "Done.\n";
}
return $fileName;
}
}
?>
cURL always send browser headers, and you will not get time out, or be redirected or die effect
download the file or fetch file content, your choice !

Detect location by IP address (I get a blank page. Why?)

Here is an useful code snippet to detect the location of a specific IP address. The function below takes one IP address as a parameter, and returns the location of the IP address. If no location is found, UNKNOWN is returned.
But I get a blank page. Why?
function detect_city($ip) {
$default = 'UNKNOWN';
if (!is_string($ip) ||
strlen($ip) < 1 ||
$ip == '127.0.0.1' ||
$ip == 'localhost')
$ip = '8.8.8.8';
$curlopt_useragent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)';
$url = 'http://ipinfodb.com/ip_locator.php?ip=' . urlencode($ip);
$ch = curl_init();
$curl_opt = array(
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_HEADER => 0,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_USERAGENT => $curlopt_useragent,
CURLOPT_URL => $url,
CURLOPT_TIMEOUT => 1,
CURLOPT_REFERER => 'http://' . $_SERVER['HTTP_HOST'],
);
curl_setopt_array($ch, $curl_opt);
$content = curl_exec($ch);
if (!is_null($curl_info)) {
$curl_info = curl_getinfo($ch);
}
curl_close($ch);
if ( preg_match('{<li>City : ([^<]*)</li>}i', $content, $regs) ) {
$city = $regs[1];
}
if ( preg_match('{<li>State/Province : ([^<]*)</li>}i', $content, $regs) ) {
$state = $regs[1];
}
if ( $city!='' && $state!='' ) {
$location = $city . ', ' . $state;
return $location;
}
else {
return $default;
}
}
You probably get a blank page because there is an error in your script and PHP is not configured to display errors, which is the case on most online webspaces. Try to enable error display by adding this to a .htaccess file in the same directory:
php_flag display_errors on
php_value error_reporting 7
This line:
if (!is_null($curl_info)) {
$curl_info = curl_getinfo($ch);
}
The $curl_info does not exist and cannot be by seeing your parameters and the rest code above. Instead, comment the if out and you should be fine (tested on my IP address):
// if (!is_null($curl_info)) {
$curl_info = curl_getinfo($ch);
// }
Hope this helps :)
function detect_city($ip) {
$default = 'UNKNOWN';
if (!is_string($ip) || strlen($ip) < 1 || $ip == '127.0.0.1' || $ip == 'localhost')
$ip = '8.8.8.8';
$curlopt_useragent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)';
$url = 'http://ipinfodb.com/ip_locator.php?ip=' . urlencode($ip);
$ch = curl_init();
$curl_opt = array(
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_HEADER => 0,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_USERAGENT => $curlopt_useragent,
CURLOPT_URL => $url,
CURLOPT_TIMEOUT => 1,
CURLOPT_REFERER => 'http://' . $_SERVER['HTTP_HOST'],
);
curl_setopt_array($ch, $curl_opt);
$content = curl_exec($ch);
if (!is_null($curl_info)) {
$curl_info = curl_getinfo($ch);
}
curl_close($ch);
if ( preg_match('{<li>City : ([^<]*)</li>}i', $content, $regs) ) {
$city = $regs[1];
}
if ( preg_match('{<li>State/Province : ([^<]*)</li>}i', $content, $regs) ) {
$state = $regs[1];
}
if( $city!='' && $state!='' ){
$location = $city . ', ' . $state;
return $location;
}else{
return $default;
}
}

Categories