My initial project was of checking whether a certain Apple ID exists or not, I have proceeded doing this in php by navigating to apppleid.apple.com/account/ and pretending to register an account with all the fields blank except the account field, and if I got an error it meant the account existed, otherwise If I got other errors but not an "account exists" error I would return false. However I have encountered a few problems on the way. The first was that you need to preserve all the headers/cookies on the way (which I did) but it still does not work, and apparently always answers with "1". The code can be found here : PASTEBIN. Please follow the link and try to solve this problem, I really need this done. Thank you very much whoever got some time to read this post.
EDIT
code:
<?php
require("simplehtmldom_1_5/simple_html_dom.php");
$input = get_data('https://appleid.apple.com/account');
$html = new simple_html_dom();
$html->load($input);
//echo $input;
$table = array();
foreach($html->find('input') as $inn)
{
$val = "";
try
{
$val = $inn->getAttribute('value');
}
catch (Exception $e)
{
$val = "";
}
//echo $inn->getAttribute('name') . $val . "\n";
if($inn->getAttribute('name') != "" && $inn->getAttribute('name') != "account.name")
{
$table[$inn->getAttribute("name")] = $val;
}
if($inn->getAttribute('name') == "account.name")
{
$table[$inn->getAttribute("name")] = "naclo3samuel#gmail.com";
}
}
$NIX = http_build_query($table);
//set the url, number of POST vars, POST data
$ch = curl_init();
$hs = get_headers("https://appleid.apple.com/account", 0);
$headers = $hs;
curl_setopt($ch,CURLOPT_URL, "https://appleid.apple.com/account/");
curl_setopt($ch,CURLOPT_POST, count($NIX));
curl_setopt($ch,CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch,CURLOPT_POSTFIELDS, $NIX);
//execute post
$result = curl_exec($ch);
echo $result;
//close connection
curl_close($ch);
/* gets the data from a URL */
function get_data($url) {
$ch = curl_init();
$timeout = 5000;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
?>
One of the problems is right here: get_headers("https://appleid.apple.com/account", 0);
This will return something like:
[0] => HTTP/1.1 200 OK
[1] => Date: Sat, 29 May 2004 12:28:13 GMT
[2] => Server: Apache/1.3.27 (Unix) (Red-Hat/Linux)
[3] => Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT
[4] => ETag: "3f80f-1b6-3e1cb03b"
[5] => Accept-Ranges: bytes
[6] => Content-Length: 438
[7] => Connection: close
[8] => Content-Type: text/html
What is cURL supposed to do with that? This is not in a format acceptable by CURLOPT_HTTPHEADER and neither headers a Server would expect from a Client request.
I suppose you are trying to stablish Cookie session. I recommend you do all that without use of get_headers() or putting your finger in the headers at all.
Enable cURL's Cookie support by setting the options CURLOPT_COOKIEJAR and CURLOPT_COOKIEFILE, make a call to https://appleid.apple.com/account to initialize your Cookies, and then do the rest.
Example:
$cookies = tmpfile();
$curl = curl_init();
curl_setopt_array($curl, [
CURLOPT_URL => "https://appleid.apple.com/account/",
CURLOPT_COOKIEJAR => $cookies,
CURLOPT_COOKIEFILE => $cookies,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HEADER => true
]);
curl_exec();
curl_setopt_array($curl, [
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $NIX
]);
$result = curl_exec($ch);
$hsize = curl_getinfo($curl, CURLINFO_HEADER_SIZE);
$headers = explode("\r\n", substr($result, 0, $hsize));
$result = substr($result, $hsize);
Related
I looked for a sample code for php & curl and i found this link http://www.php-guru.in/2013/upload-files-using-php-curl/
I tried using the code with gifs.com API to try to convert gif to mp4 (for speed reasons) then display it on my site. i tried using a giphy url to upload to gifs.com and so i ended up with the code below.
$url = 'https://api.gifs.com/media/upload';
$headers = array("Content-Type:multipart/form-data", "Gifs-API-Key:gifkey"); // cURL headers for file uploading
$postfields = array("file" => "#https://media.giphy.com/media/l378drKbCncSKYbS0/giphy.gif", "title" => 'guineapig');
$ch = curl_init();
$options = array(
CURLOPT_URL => $url,
CURLOPT_HEADER => true,
CURLOPT_POST => 1,
CURLOPT_HTTPHEADER => $headers,
CURLOPT_POSTFIELDS => $postfields,
CURLOPT_RETURNTRANSFER => true
); // cURL options
curl_setopt_array($ch, $options);
$server_output = curl_exec($ch);
if (!curl_errno($ch)) {
$info = curl_getinfo($ch);
echo $info['http_code'];
echo $server_output;
} else {
$errmsg = curl_error($ch);
echo $errmsg;
}
curl_close ($ch);
The problem is, it's always showing a 400 http_code and i don't know what the problems is
here is the full error it displays
HTTP/1.1 100 Continue HTTP/1.1 400 Bad Request Server: nginx Date: Thu, 02 Nov 2017 13:44:05 GMT Content-Type: text/plain; charset=utf-8 Content-Length: 0 Access-Control-Allow-Credentials: false Access-Control-Allow-Headers: Origin, Accept,Content-Type,Gifs-API-Key Access-Control-Allow-Methods: GET,POST,OPTIONS Access-Control-Allow-Origin: * Access-Control-Max-Age: 43200 Request-Id: 9b78a2d3-0f25-4f13-bb2d-a40b75e6fa8f Via: 1.1 google Alt-Svc: clear
I don't understand the error
note: I'm using a localhost xampp server, Is this the cause of it messing up?
turns out i only needed to use their import API since it's a link to a gif rather than a file upload, so i changed the header with a json instead of a multipart-form
$url = 'https://api.gifs.com/media/import';
$headers = array("Gifs-API-Key: gifkey", "Content-Type: application/json"); // cURL headers for file uploading
$postfields = "{\n \"source\": \"https://media.giphy.com/media/l378drKbCncSKYbS0/giphy.gif\",\n \"title\": \"guineapig\",\n \"tags\": [\"crazy\", \"hand drawn\", \"2015\", \"art\"],\n \"attribution\": {\n \"site\": \"vine\",\n \"user\": \"someone\"\n }\n}";
$ch = curl_init();
$options = array(
CURLOPT_URL => $url,
CURLOPT_HEADER => true,
CURLOPT_POST => 1,
CURLOPT_HTTPHEADER => $headers,
CURLOPT_POSTFIELDS => $postfields,
CURLOPT_RETURNTRANSFER => true
); // cURL options
curl_setopt_array($ch, $options);
$server_output = curl_exec($ch);
if (!curl_errno($ch)) {
$info = curl_getinfo($ch);
echo $info['http_code'];
echo $server_output;
} else {
$errmsg = curl_error($ch);
echo $errmsg;
}
curl_close ($ch);
I have the following code:
$options = array(
'http' => array(
'header' => "Content-type: text/html\r\n",
'method' => 'POST'
),
);
$context = stream_context_create($options);
$response = file_get_contents($url, false, $context);
This is not giving the appropriate result (no result at all), since the equivalent curl is:
function curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_POST, 1);
$data = curl_exec($ch);
curl_close($ch);
$httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
echo $httpcode;
return $data;
}
curl($url);
actually gives the actual result.
Since i'd like to avoid curl in production server, i'd really like the file_get_contents to work. To debug this, I have decided to examine the header for both curl and file_get_contents. In the curl code above, you can notice an echo, which prints the header:
HTTP/1.0 411 Length Required Content-Type: text/html; charset=UTF-8
Content-Length: 1564 Date: Thu, 03 Dec 2015 18:00:25 GMT Server:
GFE/2.0
I'd like to do the same for file_get_contents to examine its headers and hope fully see what is wrong. (you could point out what's wrong yourself if you like).
Something like this should work:
<?php
$url = 'http://example.com/';
$data = ''; // empty post
$options = array(
'http' => array(
'header' => "Content-type: text/html\r\nContent-Length: " . strlen($data) . "\r\n",
'method' => 'POST'
),
);
$context = stream_context_create($options);
$fp = fopen($url, 'r', false, $context);
$meta = stream_get_meta_data($fp);
if (!$fp) {
echo "Failed!";
} else {
echo "Success";
$response = stream_get_contents($fp);
}
fclose($fp);
var_dump($meta);
To get the stream meta data, you will need to switch from file_get_contents to fopen. It makes no difference as under the hood, PHP will connect and issue the response in the same way (using the http:// wrapper).
If php.ini is set to disable allow_url_fopen, the both file_get_contents and fopen will be affected and won't be able to open remote URLs. As long as this isn't the case, fopen of a URL will work the same way as file_get_contents; using fopen just gives you access to the stream which you can then call stream_get_meta_data on.
I have a php script that returns just plain text without any html. Now I want to make a cURL request to that script and I get the following response:
HTTP/1.1 200 OK
Date: Mon, 28 Feb 2011 14:21:51 GMT
Server: Apache/2.2.14 (Ubuntu)
X-Powered-By: PHP/5.2.12-nmm2
Vary: Accept-Encoding
Content-Length: 6
Content-Type: text/html
6.8320
The actuall response is just 6.8320 as text without any html. I want to retrieve it from the response above by just removing the header information.
I already minified the script a bit:
$url = $_GET['url'];
if ( !$url ) {
// Passed url not specified.
$contents = 'ERROR: url not specified';
$status = array( 'http_code' => 'ERROR' );
} else if ( !preg_match( $valid_url_regex, $url ) ) {
// Passed url doesn't match $valid_url_regex.
$contents = 'ERROR: invalid url';
$status = array( 'http_code' => 'ERROR' );
} else {
$ch = curl_init( $url );
if ( strtolower($_SERVER['REQUEST_METHOD']) == 'post' ) {
curl_setopt( $ch, CURLOPT_POST, true );
curl_setopt( $ch, CURLOPT_POSTFIELDS, $_POST );
}
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_HEADER, true );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_USERAGENT, $_GET['user_agent'] ? $_GET['user_agent'] : $_SERVER['HTTP_USER_AGENT'] );
list( $header, $contents ) = preg_split( '/([\r\n][\r\n])\\1/', curl_exec( $ch ), 2 );
$status = curl_getinfo( $ch );
curl_close( $ch );
}
// Split header text into an array.
$header_text = preg_split( '/[\r\n]+/', $header );
if ( true ) {
if ( !$enable_native ) {
$contents = 'ERROR: invalid mode';
$status = array( 'http_code' => 'ERROR' );
}
// Propagate headers to response.
foreach ( $header_text as $header ) {
if ( preg_match( '/^(?:Content-Type|Content-Language|Set-Cookie):/i', $header ) ) {
header( $header );
}
}
print $contents;
}
Any idea what I need to change to remove the header information from the response?
Just set CURLOPT_HEADER to false.
Make sure you put set the header flag:
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true );
curl_setopt($ch, CURLOPT_TIMEOUT, Constants::HTTP_TIMEOUT);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, Constants::HTTP_TIMEOUT);
$response = curl_exec($ch);
Do this after your curl call:
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$headerstring = substr($response, 0, $header_size);
$body = substr($response, $header_size);
EDIT:
If you'd like to have header in assoc array, add something like this:
$headerArr = explode(PHP_EOL, $headerstring);
foreach ($headerArr as $headerRow) {
preg_match('/([a-zA-Z\-]+):\s(.+)$/',$headerRow, $matches);
if (!isset($matches[0])) {
continue;
}
$header[$matches[1]] = $matches[2];
}
Result print_r($header):
(
[content-type] => application/json
[content-length] => 2848
[date] => Tue, 06 Oct 2020 10:29:33 GMT
[last-modified] => Tue, 06 Oct 2020 10:17:17 GMT
)
Don't forget to close connection curl_close($ch);
Update the value of CURLOPT_HEADER to 0 for false
curl_setopt($ch, CURLOPT_HEADER, 0);
Just for a later use if anyone else needs. I was into same situation, but just need to remove header text, not content. The response i was getting in the header was (including white space):
HTTP/1.1 200 OK
Cache-Control: private, no-cache, no-store, must-revalidate
Content-Language: en
Content-Type: text/html
Date: Tue, 25 Feb 2014 20:59:29 GMT
Expires: Sat, 01 Jan 2000 00:00:00 GMT
Pragma: no-cache
Server: nginx
Vary: Cookie, Accept-Language, Accept-Encoding
transfer-encoding: chunked
Connection: keep-alive
I wanted to remove starting from HTTP till keep-alive with white space:
$contents = preg_replace('/HTTP(.*)alive/s',"",$contents);
that did for me.
If you are using nuSoap, you can access data without headers with $nsoap->responseData or $nsoap->response, if you want the full headers.
Just in case someone needs that.
If for some reason you have to curl_setopt($ch, CURLOPT_HEADER, 1); to get cookies for example, the following worked for me. Not sure if it's 100% reliable but worth a try
$foo = preg_replace('/HTTP(.*)html/s',"",$curlresult);
$content = null;
$ch = curl_init();
$rs = curl_exec($ch);
if (CURLE_OK == curl_errno($ch)) {
$content = substr($rs, curl_getinfo($ch, CURLINFO_HEADER_SIZE));
}
curl_close($ch);
echo $content;
If someone already saved the curl response to a file (like me) and therefore don't know how big the header was to use substr, try:
$file = '/path/to/file/with/headers';
file_put_contents($file, preg_replace('~.*\r\n\r\n~s', '', file_get_contents($file)));
Just do not set the curl_header in the curl request or set it to z or false
like this
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HEADER, false);
Just don't set CURLOPT_HEADER!
<?php
/* gets the data from a URL */
function get_data($url)
{
$ch = curl_init();
$timeout = 5;
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$paste_data=""; if(isset($_POST["paste_code"])) { $paste_data = $_POST["paste_code"]; }
echo $paste_data;
$returned_content = get_data('http://pastebin.com/api_public.php/paste_code(paste_data)');
echo $returned_content;
?>
This is my php code . where $paste_data contains the data to be pasted in a new page . How do I paste it using the function paste_code(String) ?
The documentation says that you need to submit a POST request to
http://pastebin.com/api_public.php
and the only mandatory parameter is paste_code, of type string is the paste that you want to make.
On success a new pastebin URL will be returned.
Bare bone example:
$ch = curl_init("http://pastebin.com/api_public.php");
curl_setopt ($ch, CURLOPT_POST, true);
// A new paste with the string "hello there SO"
curl_setopt ($ch, CURLOPT_POSTFIELDS, "paste_code=hello there SO");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_NOBODY, 0);
$response = curl_exec($ch);
echo $response;
and on running I get:
> POST http://pastebin.com/api_public.php HTTP/1.1
Host: pastebin.com
Accept: */*
Proxy-Connection: Keep-Alive
Content-Length: 25
Content-Type: application/x-www-form-urlencoded
< HTTP/1.1 200 OK
< Transfer-Encoding: chunked
< Date: Mon, 13 Dec 2010 07:51:12 GMT
< Content-Type: text/plain
< Server: nginx/0.8.52
< Vary: Accept-Encoding
< X-Powered-By: PHP/5.3.4-dev
< Via: 1.1 apac-nc06 (NetCache NetApp/6.0.6)
<
http://pastebin.com/Lc7kAw8Z* Closing connection #0
Clearly the response has the URL http://pastebin.com/Lc7kAw8Z
Visit it and you'll see a new paste containing hello there SO
FYI for others looking at this "post 2013", the api_public.php POST has been discontinued.
For those who stumple upon this thread via seach, here is a code that works in 2013:
<?php
$data = 'Hello World!';
$apiKey = 'xxxxxxx'; // get it from pastebin.com
$postData = array(
'api_dev_key' => $apiKey, // your dev key
'api_option' => 'paste', // action to perform
'api_paste_code' => utf8_decode($data), // the paste text
'api_paste_private' => '1', // 0=public 1=unlisted 2=private
'api_paste_expire_date' => '1D', // paste expires in 1 day
);
$ch = curl_init('http://pastebin.com/api/api_post.php');
curl_setopt_array($ch, array(
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => http_build_query($postData),
CURLOPT_RETURNTRANSFER => 1,
));
$re = curl_exec($ch);
curl_close($ch);
$pasteId = end(explode('/', $re));
echo "Created new paste.\r\n Link:\t{$re}\r\n Raw:\t" . sprintf('http://pastebin.com/raw.php?i=%s', $pasteId) . "\r\n";
Actually I have two questions.
(1) Is there any reduction in processing power or bandwidth used on remote server if I retrieve only headers as opposed to full page retrieval using php and curl?
(2) Since I think, and I might be wrong, that answer to first questions is YES, I am trying to get last modified date or If-Modified-Since header of remote file only in order to compare it with time-date of locally stored data, so I can, in case it has been changed, store it locally. However, my script seems unable to fetch that piece of info, I get NULL, when I run this:
class last_change {
public last_change;
function set_last_change() {
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "http://url/file.xml");
curl_setopt($curl, CURLOPT_HEADER, true);
curl_setopt($curl, CURLOPT_FILETIME, true);
curl_setopt($curl, CURLOPT_NOBODY, true);
// $header = curl_exec($curl);
$this -> last_change = curl_getinfo($header);
curl_close($curl);
}
function get_last_change() {
return $this -> last_change['datetime']; // I have tested with Last-Modified & If-Modified-Since to no avail
}
}
In case $header = curl_exec($curl) is uncomented, header data is displayed, even if I haven't requested it and is as follows:
HTTP/1.1 200 OK
Date: Fri, 04 Sep 2009 12:15:51 GMT
Server: Apache/2.2.8 (Linux/SUSE)
Last-Modified: Thu, 03 Sep 2009 12:46:54 GMT
ETag: "198054-118c-472abc735ab80"
Accept-Ranges: bytes
Content-Length: 4492
Content-Type: text/xml
Based on that, 'Last-Modified' is returned.
So, what am I doing wrong?
You are passing $header to curl_getinfo(). It should be $curl (the curl handle). You can get just the filetime by passing CURLINFO_FILETIME as the second parameter to curl_getinfo(). (Often the filetime is unavailable, in which case it will be reported as -1).
Your class seems to be wasteful, though, throwing away a lot of information that could be useful. Here's another way it might be done:
class URIInfo
{
public $info;
public $header;
private $url;
public function __construct($url)
{
$this->url = $url;
$this->setData();
}
public function setData()
{
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $this->url);
curl_setopt($curl, CURLOPT_FILETIME, true);
curl_setopt($curl, CURLOPT_NOBODY, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HEADER, true);
$this->header = curl_exec($curl);
$this->info = curl_getinfo($curl);
curl_close($curl);
}
public function getFiletime()
{
return $this->info['filetime'];
}
// Other functions can be added to retrieve other information.
}
$uri_info = new URIInfo('http://www.codinghorror.com/blog/');
$filetime = $uri_info->getFiletime();
if ($filetime != -1) {
echo date('Y-m-d H:i:s', $filetime);
} else {
echo 'filetime not available';
}
Yes, the load will be lighter on the server, since it's only returning only the HTTP header (responding, after all, to a HEAD request). How much lighter will vary greatly.
Why use CURL for this? There is a PHP-function for that:
$headers=get_headers("http://www.amazingjokes.com/img/2014/530c9613d29bd_CountvonCount.jpg");
print_r($headers);
returns the following:
Array
(
[0] => HTTP/1.1 200 OK
[1] => Date: Tue, 11 Mar 2014 22:44:38 GMT
[2] => Server: Apache
[3] => Last-Modified: Tue, 25 Feb 2014 14:08:40 GMT
[4] => ETag: "54e35e8-8873-4f33ba00673f4"
[5] => Accept-Ranges: bytes
[6] => Content-Length: 34931
[7] => Connection: close
[8] => Content-Type: image/jpeg
)
Should be easy to get the content-type after this.
You could also add the format=1 to get_headers:
$headers=get_headers("http://www.amazingjokes.com/img/2014/530c9613d29bd_CountvonCount.jpg",1);
print_r($headers);
This will return the following:
Array
(
[0] => HTTP/1.1 200 OK
[Date] => Tue, 11 Mar 2014 22:44:38 GMT
[Server] => Apache
[Last-Modified] => Tue, 25 Feb 2014 14:08:40 GMT
[ETag] => "54e35e8-8873-4f33ba00673f4"
[Accept-Ranges] => bytes
[Content-Length] => 34931
[Connection] => close
[Content-Type] => image/jpeg
)
More reading here (PHP.NET)
(1) Yes. A HEAD request (as you're issuing in this case) is far lighter on the server because it only returns the HTTP headers, as opposed to the headers and content like a standard GET request.
(2) You need to set the CURLOPT_RETURNTRANSFER option to true before you call curl_exec() to have the content returned, as opposed to printed:
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
That should also make your class work correctly.
You can set the default stream context:
stream_context_set_default(
array(
'http' => array(
'method' => 'HEAD'
)
)
);
Then use:
$headers = get_headers($url,1);
get_headers seems to be more efficient than cURL once get_headers skip steps like trigger authentication routines such as log in prompts or cookies.
Here is my implementation using CURLOPT_HEADER, then parsing the output string into a map:
function http_headers($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
$headers = curl_exec($ch);
curl_close($ch);
$data = [];
$headers = explode(PHP_EOL, $headers);
foreach ($headers as $row) {
$parts = explode(':', $row);
if (count($parts) === 2) {
$data[trim($parts[0])] = trim($parts[1]);
}
}
return $data;
};
Sample usage:
$headers = http_headers('https://i.ytimg.com/vi_webp/g-dKXOlsf98/hqdefault.webp');
print_r($headers);
Array
(
['Content-Type'] => 'image/webp'
['ETag'] => '1453807629'
['X-Content-Type-Options'] => 'nosniff'
['Server'] => 'sffe'
['Content-Length'] => 32958
['X-XSS-Protection'] => '1; mode=block'
['Age'] => 11
['Cache-Control'] => 'public, max-age=7200'
)
You need to add
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
to return the header instead of printing it.
Whether returning only the headers is lighter on the server depends on the script that's running, but usually it will be.
I think you also want "filetime" instead of "datetime".