PHP cURL POST bad output - php

I am getting some info from a https web server using PHP plus cURL. All info got as HTTP GET is ok but when I need to do some HTTP POST I get a no sense output. Web server is ok as If i get the info from a web browser all works ok.
I am using following code:
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1");
if($method == "POST"){
print_r($post_fields);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_fields);
curl_setopt($ch,CURLOPT_HTTPHEADER, array (
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Accept-Language: es-es,es;q=0.8,en-us;q=0.5,en;q=0.3",
"Accept-Encoding: gzip, deflate",
"Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7"
));
}
if ($usecookie) {
curl_setopt($ch, CURLOPT_COOKIEJAR, $usecookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $usecookie);
}
if ($refer != "") {
curl_setopt($ch, CURLOPT_REFERER, $refer );
}
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
Answer header is:
HTTP/1.1 200 OK
Date: Sun, 20 Nov 2011 11:04:39 GMT
Server: Apache
Cache-Control: must-revalidate
Pragma: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
max-age: Thu, 01 Jan 1970 00:00:00 GMT
X-Powered-By: Servlet/2.4 JSP/2.0
idWl: PRO-LOW16_6604
Vary: Accept-Encoding
Content-Encoding: gzip
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
Any idea about where the problem could be?

It clearly shows its gziped...
Content-Encoding: gzip
Transfer-Encoding: chunked
Passing the returned data through the below function will inflate it back to readable content.
function gzdecoder($d){
$f=ord(substr($d,3,1));
$h=10;$e=0;
if($f&4){
$e=unpack('v',substr($d,10,2));
$e=$e[1];$h+=2+$e;
}
if($f&8){
$h=strpos($d,chr(0),$h)+1;
}
if($f&16){
$h=strpos($d,chr(0),$h)+1;
}
if($f&2){
$h+=2;
}
$u = gzinflate(substr($d,$h));
if($u===FALSE){
$u=$d;
}
return $u;
}

Related

Curl/php request moving to localhost after second request

I'm trying to get a json result from a website to analyze in my system.
In the first request of the day I receive Status 200 OK I get the result that I'm expecting.
But if I try to get the result again making a refresh in the page for example, I receive the following error
"close Cache-Control: no-cache Pragma: no-cache "
With the following HTTP HEADER
"HTTP/1.1 302 Found : Moved Temporarily Location: http://localhost Cache-Control: private, no-cache, max-age=0, no-store, must-revalidate Expires: Thu, 01 Jan 1970 00:00:01 +0000 Pragma: no-cache
For some reason the server returns error 302 and move to localhost and I have to wait until the next day to get the result 200 again.
Running var_dump(curl_error($curl));. I get bool(false)
My code
<?php $cookie="files/cookies.txt";
$url = "https://extra.bet365.com/ResultsApi/GetResults?sportName=sport&sportId=146&fixtureId=104716787&competitionId=20700663&fromDate=2021-07-01T04:05:00&toDate=2021-07-01T04:05:00&challengeId=62233643&marketOverride=";
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt ($curl, CURLOPT_COOKIEJAR, $cookie);
curl_setopt ($curl, CURLOPT_COOKIEFILE, $cookie);
$headers = array(
"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:89.0) Gecko/20100101 Firefox/89.0",
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3",
"Connection: keep-alive",
);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
$resp = curl_exec($curl);
curl_close($curl);
var_dump($resp); ?>

cURL + PHP + File Upload issues

Hello fellow programmers.
I am beginner programmer and I have failed attempt into creating a script for uploading image on one specific site.
My boss needs to upload images to specific site. To speed things up I need to create a script for that.
I usually use multi-part form data with cURL, but this site it is different.
Here are the headers
POST /upload-new.php?banner_url=http%3A%2F%2Ftest.com&ad_type=1&banner_size= HTTP/1.1
Host: admin.domain.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: lv,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Content-Type: application/octet-stream
X-File-Name: indonesia.gif
X-File-Size: 15450
X-File-Type: image/gif
X-File-Date: Fri, 29 May 2015 09:48:22 GMT
X-Requested-With: FileDrop-XHR-FileAPI
Referer: https://admin.domain.com/campaigns-edit.php
Content-Length: 15450
Cookie: goals=
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
GIF89a,รบ
HTTP/1.1 200 OK
Server: nginx
Date: Sat, 13 Jun 2015 16:22:08 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Encoding: gzip
----------------------------------------------------------
https://admin.domain.com/data/tmp-uploads/159310_20150613122208_indonesia.gif
GET /data/tmp-uploads/159310_20150613122208_indonesia.gif HTTP/1.1
Host: admin.domain.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0
Accept: image/png,image/*;q=0.8,*/*;q=0.5
Accept-Language: lv,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: https://admin.domain.com/campaigns-edit.php
Cookie: goals=
Connection: keep-alive
HTTP/1.1 200 OK
Server: nginx
Date: Sat, 13 Jun 2015 16:22:08 GMT
Content-Type: image/gif
Content-Length: 15450
Last-Modified: Sat, 13 Jun 2015 16:22:08 GMT
Connection: keep-alive
Etag: "557c58b0-3c5a"
Accept-Ranges: bytes
This is what I tried
$file = realpath($file_name);
$finfo = new finfo(FILEINFO_MIME);
$mimetype = $finfo->file($file);
$cfile = curl_file_create($file, $mimetype);
$PostData = array( '' => $cfile );
//$headers
$headers = array();
$headers[] = 'X-File-Name: indonesia.gif';
$headers[] = 'X-File-Size: 15450';
$headers[] = 'X-File-Type: image/gif';
$headers[] = 'X-File-Date: Fri, 29 May 2015 09:48:22 GMT';
$headers[] = 'X-Requested-With: FileDrop-XHR-FileAPI';
$URL = "https://admin.domain.com/upload-new.php?banner_url=http%3A%2F%2Ftest.com&ad_type=1&banner_size="
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $URL);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_POST, 1);
// curl_setopt($ch, CURLOPT_POSTFIELDS,$PostData);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_REFERER, $reffer);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
$size = filesize($lfile);
$file = fopen($lfile, 'r');
curl_setopt($ch, CURLOPT_POSTFIELDS, "#" . $lfile);
curl_setopt($ch, CURLOPT_INFILE, $file);
curl_setopt($ch, CURLOPT_INFILESIZE, $size);
If (StrLen ($Proxy) > 0)
{
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
curl_setopt($ch, CURLOPT_PROXY,$Proxy);
}
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 120);
curl_setopt($ch, CURLOPT_TIMEOUT, 120);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
$result = curl_exec($ch);
curl_close($ch);
Website answers with link to image which gives me 404. So seems like I am failing to send over the file.
Can anybody please give me some guidance?

get redirect url using curl_getinfo failed

I used below function to get redirected (final) url of these links
http://iprice.my/coupons/zalora/
function curlFileGetContents($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; rv:16.0) Gecko/20100101 Firefox/16.0');
$result = curl_exec($ch);
$info = curl_getinfo($ch);
$effectiveUrl = $info['url'];
curl_close($ch);
return $effectiveUrl;
}
but I couldn't get anything, I wonder why? Eg I did curlFileGetContents('http://iprice.my/coupons/zalora/#007b9a6024d19edec91d04c2e92e143e744c83b6');
The URL isn't being redirected - the links you are referring to (that only spawn on that page) are pop unders. A look at the curl header shows the URL is not being re-directed:
curl --head http://iprice.my/coupons/zalora/#007b9a6024d19edec91d04c2e92e143e744c83b6
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/5.5.9-1ubuntu4.9
Cache-Control: no-cache
Date: Fri, 22 May 2015 15:06:31 GMT
X-iPrice-Cached: 1
X-iPrice-Cached-Type: Response
The approach you currently have won't work for what you want to do.

How request xml via PHP-cURL POST request

I want a PHP-cURL script to do the following request.
http://site5.way2sms.com/QuickContacts
POST /QuickContacts HTTP/1.1
Host: site5.way2sms.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:8.0) Gecko/20100101 Firefox/8.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
X-Requested-With: XMLHttpRequest
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Referer: http://site5.way2sms.com/Main.action?id=0CD36BD332A5C7AE77FDBA1CBDBFFBB6.w809
Content-Length: 16
gads=ID=e2cdae9b764355fa:T=1333862873:S=ALNI_MYvxochQ56ILMvBDr4oyyqCIDVn3w
Pragma: no-cache
Cache-Control: no-cache
folder=DashBoard
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: text/html
Content-Length: 1901
Date: Sun, 08 Apr 2012 06:07:52 GMT
Connection: close
I think the returned content is in XML format. How to handle them?
If you want generate a post request to the URL and catch the returned data using cURL you can use the following function.
function cURL($url, $header=NULL, $cookie=NULL, $p=NULL)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, $header);
curl_setopt($ch, CURLOPT_NOBODY, $header);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
if ($p) {
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $p);
}
$result = curl_exec($ch);
if ($result) {
return $result;
} else {
return curl_error($ch);
}
curl_close($ch);
}
Like,
$data = cURL("http://site5.way2sms.com/QuickContacts", NULL, NULL, array("post_var" => value, "another_post_var" => val2));

Simulating a POST with PHP & cURL

I'm trying to simulate a POST to a website based on what I see coming from Live HTTP headers in Firefox.
Here's a copy/paste of the log from the Firefox plugin:
POST /context?tab=login HTTP/1.1
Host: website User-Agent:
Mozilla/5.0 (X11; U; Linux i686;
en-US; rv:1.9.2.13) Gecko/20101206
Ubuntu/10.10 (maverick)
Firefox/3.6.13 Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset:
ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115 Connection:
keep-alive Referer: referer
Cookie: fontsize=2;
JSESSIONID=0000pXE_BK7TjZFzEyNHqOKzXz2:-1
Content-Type:
application/x-www-form-urlencoded
Content-Length: 46
loginid=password&password=password&login=Login
And the response that follows immediately after the POST:
HTTP/1.1 302 Found Location:
website/context?tab=p00689
Content-Language: en-US
Set-Cookie:
JSESSIONID=0000oaKlIeeDRWkX5YCiJu5v1lM:-1;
Path=/ Transfer-Encoding:
chunked Date: Mon, 07 Feb 2011
14:15:21 GMT Server: WebSphere
Application Server/6.1 Expires:
Thu, 01 Dec 1994 16:00:00 GMT
Cache-Control: no-cache="set-cookie,
set-cookie2"
Based on my testing, a response that redirects to
website/context?tab=p00689
Means that the user was authenticated and everything worked properly.
However, when trying to accomplish this via PHP & cURL, I'm being redirected to a page that informs the user that their session has timed out.
Here's the code:
// Provider only likes Firefox
$agent = "User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13";
ini_set("user_agent", $agent);
// Cookie
$cookie = tempnam("/tmp", "curlcookie");
// Post everything that was posted to me.
$fields = $_POST;
foreach($fields as $key=>$value)
{
$fields_string .= "$key=$value&";
}
$fields_string = substr($fields_string, 0, strlen($fields_string) - 1);
// Custom headers
$headers = array(
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Accept-Language: en-us,en;q=0.5",
"Accept-Encoding: gzip,deflate",
"Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7",
"Keep-Alive: 115",
"Connection: keep-alive");
// cURL options
$ch = curl_init("website");
curl_setopt($ch, CURLOPT_REFERER, "referer");
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields_string);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_MAXREDIRS, 1);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$output = curl_exec($ch);
$header = curl_getinfo($ch);
curl_close($ch);
// Debugging junk
echo nl2br($header["request_header"]);
echo "<br/><br/>Output:<br/><br/>$output";
The output from that script is as follows:
POST /context?tab=login HTTP/1.1
User-Agent: User-Agent: Mozilla/5.0
(X11; U; Linux i686; en-US;
rv:1.9.2.13) Gecko/20101206
Ubuntu/10.10 (maverick)
Firefox/3.6.13 Host: website
Pragma: no-cache Referer:
referer Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset:
ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115 Connection:
keep-alive Content-Length:
46 Content-Type:
application/x-www-form-urlencoded
loginid=username&password=password&login=Login
Output: HTTP/1.1 302
Found
Location:website/context?tab=p00697
Content-Language: en-US Set-Cookie:
JSESSIONID=0000Tl8NL1Hg2dbNv_PEnq-bbvr:-1;
Path=/ Set-Cookie:
JSESSIONID=0000Zue58y1tXg3tt4XjB8exXw6:-1;
Path=/ Transfer-Encoding: chunked
Date: Mon, 07 Feb 2011 19:18:20 GMT
Server: WebSphere Application
Server/6.1 Expires: Thu, 01 Dec 1994
16:00:00 GMT Cache-Control:
no-cache="set-cookie,
set-cookie2"
Based on what I've posted, is there anything obvious that I'm missing? What should I try next? The requests look semantically the same; I'm not sure what I could be doing incorrectly.
The one thing that stands out is the following line of code:
$cookie = tempnam("/tmp", "curlcookie");
Now if this fails to create the file then tempnam would return false, meaning that the following lines of code:
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
are as good as not being set at all, you should keep the cookie file within the same directory as the executing script.
the next thing is:
$fields = $_POST;
foreach($fields as $key=>$value)
{
$fields_string .= "$key=$value&";
}
$fields_string = substr($fields_string, 0, strlen($fields_string) - 1);
You do not need to do this as CURLOPT_POSTFIELDS accepts an array so you should be able to do:
curl_setopt($ch, CURLOPT_POSTFIELDS, $_POST);
This will make sure that the entities are correctly parsed.
I also think you can remove the ini_set as that's for native functions such as file_get_contents and fopen streams etc, so double check the line:
ini_set("user_agent", $agent);
Also I would check to see if there is a cookie already set from the main page, such as index.php as the site may block requests from sources that have come directly to the login page with data.

Categories