how to post variables to file like jquery ajax but with php - php

I'm working on a project where I have to include a third party application in our system. This application transmitts parameters to a PHP file using jQuery ajax when calling the index.php. The exact function looks like this:
$.ajax( {
type : "POST",
cache : false,
crossDomain : false,
async : true,
url : "/somwhere/somefile.php",
data : "msg=37120&hello=true&USRID=1"
)};
I'm am not very experienced with AJAX at all but as far as I can see in the end the parameters "msg=37120&hello=true&USRID=1" get postet on somefile.php. Now I want to post the variables directly onto somefile.php using PHP instead of JS/AJAX.
After some researches I found the following solution to send post data directly to the somefile.php:
$url = "/somwhere/somefile.php";
$ch = curl_init( $url );
curl_setopt( $ch, CURLOPT_POST, 1);
curl_setopt( $ch, CURLOPT_POSTFIELDS, "msg=37120&hello=true&USRID=1");
$response = curl_exec( $ch );
Unfortunately this is not working at all. No data arrives when I'm executing this code block. I've no idea what I'm doing wrong. Can someone tell me whats the mistake here?
Or is there even a better way how to do this?

Edit: Sorry -- I posted the get, below is the code to post.
This is the code that I use that works well:
$cookie="cookie.txt";
$useragent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)";
$data=array(
"field1" => 'data1',
"field2" => 'data2',
"SUBMIT" => 'true'
);
$cr = curl_init($url);
curl_setopt($cr, CURLOPT_RETURNTRANSFER, true); // Get returned value as string (dont put to screen)
curl_setopt($cr, CURLOPT_USERAGENT, $useragent); // Spoof the user-agent to be the browser that the user is on (and accessing the php $
curl_setopt($cr, CURLOPT_COOKIEJAR, $cookie); // Use cookie.txt for STORING cookies
curl_setopt($cr, CURLOPT_COOKIEFILE, $cookie); // Use cookie.txt for STORING cookies
curl_setopt($cr, CURLOPT_POST, true); // Tell curl that we are posting data
curl_setopt($cr, CURLOPT_POSTFIELDS, $data); // Post the data in the array above
$output = curl_exec($cr);

Related

PHP cURL with post data login not working where wget with shell_exec is

I'm trying to access remote files on a website that requires me to log in to it. The current way I'm doing this is with a wget ran from shell_exec. This 'works' but is definitly not the way I want to do it in. I would prefer to do it using cURL, but for some reason this will not work for me.
What happens instead of getting the page I requested is I'm redirected to the login page, which does not happen with the wget method, which should be making the exact same request...
What am I doing wrong?
First of all this is the current (not nice but) working method of accessing the page:
shell_exec("wget --post-data='serviceLoginUser=something&serviceLoginPass=something&qq=login' -p https://internal.website.com/somepage.php?get=request -O return.json");
This is my download function which does not work but I want to replace the old way with: (updated after suggestion from Abkarino)
public function download($url, $postData = NULL){
//Set cURL options
curl_setopt($this->ch, CURLOPT_URL, $url);
curl_setopt($this->ch, CURLOPT_HEADER, 0);
//To act like a normal browser (needed for intranet)
curl_setopt($this->ch, CURLOPT_USERAGENT,'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.52 Safari/537.17');
curl_setopt($this->ch, CURLOPT_AUTOREFERER, true);
curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($this->ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($this->ch, CURLOPT_VERBOSE, 1);
//TODO: remove
curl_setopt($this->ch, CURLOPT_SSL_VERIFYPEER, false);
//If POST data was supplied:
if ($postData != NULL){
curl_setopt($this->ch,CURLOPT_POSTFIELDS, $postData);
}
//execute post
return curl_exec($this->ch);
}
Where the following code will form the request: (currenly this code is only to test if the cURL actually returns the correct data)
include 'Downloader.php';
$dl = new Downloader;
$postData = array(
'serviceLoginUser' => 'something',
'serviceLoginPass' => 'something',
'qq' => 'login'
);
echo $dl->download("https://internal.website.com/somepage.php?get=request", $postData);

I am not able to get a proper response through browser, but through command line php works perfectly

Now running through Command Line, I get expected JSON response and successful authentication.
But through a browser, I get errors. edit I am able to authenticate and get the code. If I put in the code in instead of the variable, I am able to get a response from the command line.
Also have tested and am able to sub vars out for the actual strings and still authenticate through command line but not browser.
function tokenRequest($clientId, $clientSecret, $code){
$url = 'https://www.strava.com/oauth/token?';
$oauthFields = array(
'client_id' => $clientId,
'client_secret' => $clientSecret,
'code' => $code);
$parameters = '&' . http_build_query($oauthFields);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url.$parameters);
curl_setopt($ch, CURLOPT_POST, count($oauthFieldsObject));
curl_setopt($ch, CURLOPT_POSTFIELDS, $parameters);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
$urlResponse = curl_exec($ch);
curl_close($ch);
return (json_decode($urlResponse));
}
Expected is a json response, but on my main page I decode it but nothing is showing up. Below is the code to try to show the json data.
$token = tokenRequest($client_id, $client_secret, $codeUrl);
echo $token;
var_dump($token);
you must look which oauth version they use
your function isnt enough you must request an requestToken then an accessToken and then you must request the user data to get a working
look here: http://strava.github.io/api/v3/oauth/
I've figured it out, I've added:
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, TRUE);
Since it was an https site.

Why url returns JSON data via browser but shows nothing with curl (CURLOPT_RETURNTRANSFER is true)?

I want to get json data from here: JSON url;
Using Chrome I can see all JSON data, but using curl (below code) it seems to redirect and get lost (if CURLOPT_FOLLOWLOCATION is false, it does nothing):
$json_url = 'http://cartolafc.globo.com/mercado/filtrar.json?page=1&order_by=media&status_id=7&posicao_id=1';
$ch = curl_init($json_url);
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
//curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, false );
//even killing the redirect process it does not return JSON data
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
$data = curl_exec( $ch );
$dataDecoded = json_decode($data);
print_r($dataDecoded);
I saw something about simulating a browser with curl but I tought agent would do the trick. Maybe something about server using cookies... I really don't know. I saw other answers here today but they didn't solve my problem. Am I missing something?
Thank you.
Wrong variable use:
$data = curl_exec( $ch );
^^^^^--- data here
$dataDecoded = json_decode($json);
^^^^--- not $data here
So you're trying to decode a variable that doesn't exist.

PHP CURL - Problems storing and using cookies when scraping

I've been trying to write a script that retrieves Google trends results for a given keyword. Please note im not trying to do anything malicious I just want to be able to automate this process and run it a few times every day.
After investigating the Google trends page I discovered that the information is available using the following URL:
http://www.google.com/trends/trendsReport?hl=en-GB&q=keyword&cmpt=q&content=1
You can request that information mutliple times with no issues from a browser, but if you try with "privacy mode" after 4 or 5 requests the following is displayed:
An error has been detected You have reached your quota limit. Please
try again later.
This makes me think that cookies are required. So I have written my script as follows:
$cookiefile = $siteurl . '/wp-content/plugins/' . basename(dirname(__FILE__)) . '/cookies.txt';
$url = 'http://www.google.com/trends/trendsReport?hl=en-GB&q=keyword&cmpt=q&content=1';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$x='error';
while (trim($x) != '' ){
$html=curl_exec($ch);
$x=curl_error($ch);
}
echo "test cookiefile contents = ".file_get_contents($cookiefile)."<br />";
echo $html;
However I just can't get anything written to my cookies file. So I keep on getting the error message. Can anyone see where I'm going wrong with this?
I'm pretty sure your cookie file should exist before you can use it with curl.
Try:
$h = fopen($cookiefile, "x+");

Login to website using PHP and CURL

I'm attempting to autologin to https://www.myicomfort.com/ to retrieve data. Tried some of the examples posted but does not seem to work. Maybe I'm not using the correct field names when passing the username and password. Can someone pls help? Still learning PHP. Thanks!
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"https://www.myicomfort.com/");
curl_setopt($ch, CURLOPT_COOKIEFILE,1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 120);
curl_setopt($ch, CURLOPT_TIMEOUT, 120);
$post_array = array(
'ctl00$RightContent$txtUserName'=>'xxxname',
'ctl00$RightContent$txtPwd'=>'xxxpassword',
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_array);
$output = curl_exec($ch);
curl_close($ch);
`
You are missing the other fields. You can check which fields are being sent by pressing Ctrl+Shift+I in Chrome, then try to login. Click on the page and you will see something like this:
__LASTFOCUS:
__EVENTTARGET:
__EVENTARGUMENT:
__VIEWSTATE:/wEPDwUKMTczNjcxMDc0Mg9kFgJmD2QWAgIDD2QWBAIDD2QWBAI...
__EVENTVALIDATION:/wEWCAKI/qyXDAKSptf/CwKJn6v3AQKdg7/fBwKNoqOVDAK...
ctl00$RightContent$hdnPwd:
ctl00$RightContent$txtUserName:asfasdfa
ctl00$RightContent$txtPwd:dfasdf
ctl00$RightContent$chkRemember:on
ctl00$RightContent$btnLogin:Log in
Try to submit those data too. Once you've done this, you should be able to login.
Make sure you are using all the input fields required, doing a net sniff on information being transmitted from the form results in this
'__EVENTARGUMENT'
'__EVENTTARGET'
'__EVENTVALIDATION' /wEWCAKI/qyXDAKSptf/CwKJng .. etc
'__LASTFOCUS'
'__VIEWSTATE' /wEPDwUKMTczNjcxMDc0Mg9kFgJmD2QWAg .. etc
'ctl00$RightContent$btnLog...' Log in
'ctl00$RightContent$chkRem...' on
'ctl00$RightContent$hdnPwd'
'ctl00$RightContent$txtPwd' vyvuy
'ctl00$RightContent$txtUse...' iuhgiu
See also this question on SO where I believe they suggested to at a missing login button
U can look into the source code of that page, there are many hidden input fields. Just provide your username and password is not enough.
Solutions:
get all these fields and send them out together using PHP
use some library which can simulate browser, then u can only provide username and password when logging in. This would be a little slow, but NOT the bottleneck of your code.

Categories