I want to open HTTPS file with PHP but this page makes a redirection to another page so fopen function doesn't parse the page I want.
I have this code :
$url = 'myHTMLPageWithParameters';
$file = file($url);
// test
var_dump($file);
And result :
array (size=12)
0 => string '<html>
' (length=7)
1 => string '<head>
' (length=7)
2 => string '<script language="javascript">
' (length=31)
3 => string 'function setTop(){top.location="/index.htm"}
' (length=45)
4 => string '</script>
' (length=10)
5 => string '<title>...</title>
' (length=19)
6 => string '</head>
' (length=8)
7 => string '
' (length=1)
8 => string '<body onLoad="setTop()">
' (length=25)
9 => string '</body>
' (length=8)
10 => string '
' (length=1)
11 => string '</html>
' (length=8)
When I display 'myHTMLPageWithParameters' in a HTML browser, I see the correct page after redirection. I'm just looking for a way to capture HTML code of the second page (after the redirection). Thanks for any help
Possible duplicate of follow redirects with curl in php
In short: it's not doable in a reliable manner.
This is not a redirection done by the server, you are getting the page that you requested. Then, that page redirects to another, but using javascript. Javascript it's interpreted by the browser, not by php, curl or any other library.
The only way I can think of, it's by using regex to find location.href or location.top and then follow those redirects. But again, there are plenty ways to redirect a page, you can't expect to parse them all!
Check out this solution from another SO post:
Will PHPs fopen follow 301 redirects?
Another option would be to use curl instead of fopen, which has an option you can set telling it to follow redirects (CURLOPT_FOLLOWLOCATION).
You can use FOLLOW_LOCATION;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "myHTMLPageWithParameters");
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$html_response = curl_exec($ch);
// We get the content
$html = str_get_html($html_response);
// Get #result div for example
$content = $html->find('#result');
Related
I'm using softtouch API with WordPress, and posting data to API via curl.
But in response I am not able to send the big integer value in function. I'm not getting is that data type range issue or curl.
Below is my code:
//create reservation
$prod_items = array();
$single_item = array('product_uid'=>11449701010101);
$prod_items[] = $single_item;
$res_params = array(
'customer_id' => 1111,
'payment_type' => '',
'invoice_address_id' => 123,
'delivery_address_id' => 142,
'giftlist_id' => '',
'store_id' => '',
'items' => $prod_items
);
$res_url = $base_url . 'reservations';
$res_content = json_encode($res_params);
$res_curl = curl_init();
curl_setopt($res_curl, CURLOPT_HTTPHEADER, array('Authorization: ' . $authToken, 'Content-Type: application/json'));
curl_setopt($res_curl, CURLOPT_POST, true);
curl_setopt($res_curl, CURLOPT_POSTFIELDS, $res_content);
curl_setopt($res_curl, CURLOPT_URL, $res_url);
curl_setopt($res_curl, CURLOPT_RETURNTRANSFER, true);
$res_response = curl_exec($res_curl);
if ($res_response === FALSE)
die(curl_error($res_curl));
curl_close($res_curl);
$res_array = json_decode($res_response);
While sending data to the curl_exec() function it removes the last two digits of the product_uid, as I passed it 11449701010101 it send it as 114497010101.
Is that any integer range issue or something curl function issue?
tl:dr There would seem to be something else (in the script that processes the CURL request) that is truncating the product_uid. (?)
array('product_uid'=>11449701010101)
If you are on a 32 bit system (PHP compiled as 32 bit) then 11449701010101 does exceed the 32 bit integer range. However, in such cases, PHP silently converts the number to a float and nothing is "lost" in this instance.
json_encode($res_params);
The PHP function json_encode() converts the passed array into a string respresentation (JSON string). Nothing is lost. The 11449701010101 value is preserved.
While sending data to the curl_exec() function it removes the last two digits
The (POST) data being transmitted is an ordinary string, so nothing can be lost to the individual properties at this stage.
If the receiving script then decodes the transmitted JSON string, encodes it again and sends it back, the data is returned intact. The product_uid is a float, not an integer (as it is in the original data).
If you specifically force 11449701010101 to an integer (eg. (int)11449701010101) then you get -681801035 - the last 2 digits are not simply truncated. For the last 2 digits to be truncated there would seem to be some kind of string manipulation going on?
So, there would seem to be something else (not shown here) during the processing of this data that is perhaps truncating the value. (?)
this is an issue I've run into before using curl. We do a mass data import at the beginning of the day for and part of that is geocoding some addresses. We use google's API to do this, so a simple curl loop (should) work, at least that's what I thought.
Here are my two functions: Note that the properties variable contains about 100 entries. However, no matter how many times I refresh, the loop always stops calling the curl function after the 5th iteration. Note that the loop does not terminate, only the call to the curl function appears to be lost.
function geocode()
{
$service_url = 'https://maps.googleapis.com/maps/api/geocode/json?key=' . GAPI_KEY . '&address=';
$properties = $this->listing_model->get_non_geocoded();
if ($properties) {
foreach ($properties->result() as $property) {
$service_url .= urlencode($property->address . "," . $property->city . "," . $property->state . " " . $property->zip);
try {
$search = $this->curl($service_url);
} catch (Exception $e) {
var_dump($e);
}
}
}
}
function curl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
curl_close($ch);
unset($ch);
return $result;
}
And of course the var dumps:
...previous (first) 3 entries...
string '14' (length=2)
object(stdClass)[116]
public 'lat' => float #######
public 'lng' => float #######
string '15' (length=2)
object(stdClass)[131]
public 'lat' => float #######
public 'lng' => float #######
string '16' (length=2)
null <-- ? Should be from curl
string '17' (length=2)
null <-- ? Should be from curl
string '18' (length=2)
null <-- ? Should be from curl
string '19' (length=2)
null <-- ? Should be from curl
According to the API's documentation found here: https://developers.google.com/maps/documentation/geocoding/, it looks like you can only do five calls a second with a total of 2500 requests per day.
So now, I think your question is how to properly handle the rate limit and it looks like there are some good ideas here: How to manage request limited API calls
(Keeping track of every request, sleeping between every set of five requests, etc.)
Best of luck!
EDIT: Sorry! Didn't see that you had it solved. Figured that since the limit on the API was five and your loop was failing every five iterations, there was a correlation.
Not an experienced developer and using CodeIgniter for the first time. I'm trying to grab a signed URL for a given MP3 filename stored in S3. This is currently working with the exception of files that contain brackets.
Relevant controller code:
function index ($streamfile) {
// Load S3 client
$this->load->spark('amazon-sdk');
$s3 = $this->awslib->get_s3();
// Define request parameters
$s3bucket = $userdata['s3bucket']; // defined elsewhere
$streamfiletest = ($string)'Crazy_(Remix).mp3';
// Request signed URL
$url = $s3->get_object_url($s3bucket, ***EITHER $streamfiletest or $streamfile***, '5 minutes');
// Fetch status code
$http = new CFRequest($url);
$http->add_header('Content-Type', '');
$http->send_request(true);
$code = $http->get_response_code();
$headers = $http->get_response_header();
// Load the view
$data['filename'] = $url;
$data['debug'] = array(
'file1' => $streamfile,
'file2' => $streamfiletest,
'signed_url' => $url,
'code' => $code,
'headers' => $headers
);
$this->load->view('play', $data);
Relevant view code:
<?php if (isset($debug)) {
echo "DEBUGS:";
echo '<pre>' . print_r($debug, TRUE) . '</pre>';
} ?>
As you can see I either pass $streamfile or $streamfiletest. In the debug I can confirm that both variables are the same string.
When passing $streamfile to the URL request, the URL in the response is incorrect:
DEBUGS:
[file1] => Crazy_(Remix).mp3
[file2] => Crazy_(Remix).mp3
[signed_url] => http://s3-...(removed)/Crazy_%26%2340%3BRemix%26%2341%3B.mp3?AWSAccessKey...
[code] => 404
You can see that the brackets have been strangely encoded %26%2340%3B and therefore I can't find the file in S3.
When passing $streamfiletest however, the response is fine:
DEBUGS:
[file1] => Crazy_(Remix).mp3
[file2] => Crazy_(Remix).mp3
[signed_url] => http://s3-...(removed)/Crazy_%28Remix%29.mp3?AWSAccessKey...
[code] => 200
The brackets are encoded correctly in the signed URL an I get a HTTP 200 from S3.
Any ideas what could be causing this?
In the debug I can confirm that both variables are the same string
Actually, not quite.
If you look closely, it becomes apparent what the url escaped values must mean:
%26%2340%3B %26%2341%3B
& # 40 ; & # 41 ;
Those are numeric html character codes that the browser will display as ( and ) but it does not in fact mean that the two strings have identical content. They only appear to.
The solution, of course, depends on how they are getting transformed that way, and either not doing that, or decoding the numeric character codes.
Try doing the following to decode the url encoded brackets
$data['filename'] = urldecode($url);
This should return the string to its expected format ie with brackets
I tried implementing the following PHP code to POST JSON via PHP: cURL (SOME FORCE.COM WEBSITE is a tag that signifies the URL that I want to POST):
$url = "<SOME FORCE.COM WEBSITE>";
$data =
'application' =>
array
(
'isTest' => FALSE,
key => value,
key => value,
key => value,
...
)
$ch = curl_init($url);
$data_string = json_encode($data);
curl_setopt($ch, CURLOPT_POST, true);
//Send blindly the json-encoded string.
//The server, IMO, expects the body of the HTTP request to be in JSON
curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER,
array
(
'Content-Type:application/json',
'Content-Length: ' . strlen($data_string)
)
);
$result = curl_exec($ch);
curl_close($ch);
print_r($result);
echo '<pre>';
echo $_POST;
$jsonStr = file_get_contents('php://input'); //read the HTTP body.
var_dump($jsonStr);
var_dump(json_decode($jsonStr));
echo '</pre>';
The output of the above is the following:
"Your TEST POST is correct, please set the isTest (Boolean) attribute on the application to FALSE to actually apply."
Arraystring(0) ""
NULL
OK, the above confirms that I formatted the JSON data correctly by using json_encode, and the SOME FORCE.COM WEBSITE acknowledges that the value of 'isTest' is FALSE. However, I am not getting anything from "var_dump($jsonStr)" or "var_dump(json_decode($jsonStr))". I decided to just ignore that fact and set 'isTest' to FALSE, assuming that I am not getting any JSON data because I set 'isTest' to TRUE, but chaos ensues when I set 'isTest' to FALSE:
[{"message":"System.EmailException: SendEmail failed. First exception on row 0; first error: REQUIRED_FIELD_MISSING, Missing body, need at least one of html or plainText: []\n\nClass.careers_RestWebService.sendReceiptEmail: line 165, column 1\nClass.careers_RestWebService.postApplication: line 205, column 1","errorCode":"APEX_ERROR"}]
Arraystring(0) ""
NULL
I still do not get any JSON data, and ultimately, the email was unable to be sent. I believe that the issue is resulting from an empty email body because there is nothing coming from "var_dump($jsonStr)" or "var_dump(json_decode($jsonStr))". Can you help me retrieve the JSON POST? I would really appreciate any hints, suggestions, etc. Thanks.
I solved this question on my own. I was not sure if I was doing this correctly or not, but it turns out that my code was perfect. I kept refreshing my website, from where I am POSTing to SOME FORCE.COM WEBSITE. I believe that the people managing SOME FORCE.COM WEBSITE were having issues on their end. Nothing was wrong with what I did. For some reason, I got a code 202 and some gibberish text to go along with it. I would be glad to show the output, but I do not want to POST again for the sake of the people managing SOME FORCE.COM WEBSITE that I am POSTing to. Thank you guys for your help.
This question already has answers here:
curl POST format for CURLOPT_POSTFIELDS
(10 answers)
Closed 4 years ago.
I have this curl request below, which was successfully troubleshooted in another post. Right now, my PHP seems to work through this code without any errors, moves onto the next part of the IF statement and sends the confirmation email. It just doesn't update the database as it should from the web service. I will have to email the creator of the web service if this does not work but I just want to be sure that the code is fairly solid before I do this. Any one have any ideas? Here is the code:
$url = 'http://127.0.0.1:85/AccountCreator.ashx';
$curl_post_data = array(
'companyName' =>urlencode($companyName),
'mainContact' =>urlencode($mainContact),
'telephone1' =>urlencode($telephone1),
'email' => urlencode($email),
'contact2' => urlencode($contact2),
'telephone2' => urlencode($telephone2),
'email2' => urlencode($email2),
'package' => urlencode($package)
);
foreach($curl_post_data as $key=>$value) {$fields_string .=$key. '=' .$value.'&';
}
rtrim($fields_string, '&');
//die("Test: ".$fields_string);
$ch = curl_init();
curl_setopt ($ch, CURLOPT, $url);
curl_setopt ($ch, CURLOPT_POST, count($curl_post_data));
curl_setopt ($ch, CURLOPT_POSTFIELDS, $fields_string);
$result = curl_exec($ch);
curl_close($ch);
Firstly, what is the problem? It would be easier to troubleshoot it if you explained exactly what the failure in the code was. Secondly, there are a couple of odd things you are doing in this code:
I don't see why you are doing
curl_setopt ($ch, CURLOPT_POST, count($curl_post_data));
CURLOPT_POST requires a boolean (true/false) setting. You should set it to true.
Secondly, you don't need to encode CURLOPT_POSTFIELDS manually. Make an array and let cURL deal with it internally:
$curl_post_data = array(
'companyName' =>$companyName,
'mainContact' =>$mainContact,
'telephone1' =>$telephone),
'email' => $email,
'contact2' => $contact2,
'telephone2' => $telephone2,
'email2' => $email2,
'package' => $package
);
These might not fix the problem, but they may help.
The CURLOPT_POSTFIELDS option accepts an associative array of POST-data. Probably better to use that one rather than to construct the query string yourself so you got someone else to blame when it blows up.
PHP Manual:
The full data to post in a HTTP "POST"
operation. To post a file, prepend a
filename with # and use the full path.
This can either be passed as a
urlencoded string like
'para1=val1¶2=val2&...' or as an
array with the field name as key and
field data as value. If value is an
array, the Content-Type header will be
set to multipart/form-data.