Is there a way for PHP CURL functions to get the contents of the website, but stopped on the characters that we just ask. I think this sort of buffer.
so the script did not call the overall page
So schemes like this:
: curl execution
<html>
->
->
->
-> Title Detected
: curl close
->
->
->
->
</ html>
Please this is not a DOM problem. But how to curl stops when it finds that we ask.
this is my code :
function curl_download($Url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $Url);
curl_setopt($ch, CURLOPT_REFERER, $Url);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.46 Safari/536.5");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
If cURL can't handle this problem, how about fopen? and do you have an example?
Thanks before.Also please give me your example code for me,, thanks
Here is a very simple example using fsockopen(). Extend it to fit your needs.
$host = 'www.site.com';
$port = 80;
$sock = fsockopen($host, $port, $errno, $errstr, 30);
if (!$sock) {
die("Failed to connect. $errno: $errstr");
}
// write http request to socket:
$request = "GET /file.html HTTP/1.0\r\n"
."Host: $host\r\n"
."User-Agent: some-user-agent\r\n"
."Connection: close\r\n"
."\r\n";
fwrite($sock, $request);
$buffer = ''; // buffer for storing response
while (!feof($sock)) {
$buffer .= fgets($sock, 1024); // read 1024 bytes from socket, append to buffer
if (strpos($buffer, '</title>') !== false)) { // title was found
fclose($sock);
break;
}
}
So we connect to the HTTP server on the remote host, issue a simple HTTP/1.0 request, and read the response 1024 bytes at a time until the closing title tag is detected. Once it is found, the connection is closed.
Note, even though we didn't read the entire response from the socket, the underlying system (PHP and the OS socket layer) may have read more (or possibly all depending on size) of the response. In either case, you did prevent PHP from reading most of the response. If the pages are very big, closing the socket early will likely prevent a bulk of the data from actually ever being received.
Hope that helps.
I do not think you can parse the DOM with CURL
I advise you to use the string function like strstr, strtok...
http://www.php.net/manual/en/ref.strings.php
Related
I want to parse a lot of URLs to only get their status codes.
So what I did is:
$handle = curl_init($url -> loc);
curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);
curl_setopt($handle, CURLOPT_HEADER , true); // we want headers
curl_setopt($handle, CURLOPT_NOBODY , true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
$response = curl_exec($handle);
$httpCode = curl_getinfo($handle, CURLINFO_HTTP_CODE);
curl_close($handle);
But as soon as the "nobody"-option is set to true, the returned status codes are incorrect (google.com returns 302, other sites return 303).
Setting this option to false is not possible because of the performance loss.
Any ideas?
The default HTTP request method for curl is GET. If you want only the response headers, you can use the HTTP method HEAD.
curl_setopt($handle, CURLOPT_CUSTOMREQUEST, 'HEAD');
According to #Dai's answer, the NOBODY is already using the HEAD method. So the above method will not work.
Another option would be to use fsockopen to open a connection, write the headers using fwrite. Read the response using fgets until the first occurrence of \r\n\r\n to get the complete header. Since you need only the status code, you just need to read the first 13 characters.
<?php
$fp = fsockopen("www.google.com", 80, $errno, $errstr, 30);
if ($fp) {
$out = "GET / HTTP/1.1\r\n";
$out .= "Host: www.google.com\r\n";
$out .= "Accept-Encoding: gzip, deflate, sdch\r\n";
$out .= "Accept-Language: en-GB,en-US;q=0.8,en;q=0.6\r\n";
$out .= "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36\r\n";
$out .= "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\n";
$out .= "Connection: Close\r\n\r\n";
fwrite($fp, $out);
$tmp = explode(' ', fgets($fp, 13));
echo $tmp[1];
fclose($fp);
}
cURL's nobody option has it use the HEAD HTTP verb, I'd wager the majority of non-static web applications I the wild don't handle this verb correctly, hence the problems you're seeing with different results. I suggest making a normal GET request and discarding the response.
i suggest get_headers() instead:
<?php
$url = 'http://www.example.com';
print_r(get_headers($url));
print_r(get_headers($url, 1));
?>
So I'm struggling with the code below. I have java that is serving a file and php that is trying to grab the file and save it. I can download the file from the servlet just fine in a browser. I can download any file or webpage (SSL or not) with the php code. But FOR THE LIFE OF ME i cant get the php to download from this servlet!!! Im pretty familiar with PHP but not so much with java and i didnt write that servlet if it helps. I can however edit either end of the code. Thank you!!
JAVA:
#Override
public void doGet( HttpServletRequest request, HttpServletResponse response ) throws ServletException, IOException {
try {
String fileName = "/tmp/DongleUpdate.jar";
File f = new File( "/tmp/DongleUpdate.jar" );
response.setContentType( "application/zip" );
response.setHeader( "Content-Disposition", "attachment;filename=\"" +
fileName + "\"" );
byte[] arBytes = new byte[( int ) f.length()];
FileInputStream is = new FileInputStream( f );
is.read( arBytes );
ServletOutputStream op = response.getOutputStream();
op.write( arBytes );
op.flush();
} catch( IOException ioe ) {
ioe.printStackTrace();
}
}
PHP:
$ch = curl_init($file);
curl_setopt($ch, CONNECTTIMEOUT, 10);
curl_setopt($ch, TIMEOUT, 60);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_FRESH_CONNECT, true);
curl_setopt($curl, CURLOPT_GET, true);
curl_setopt($curl, CURLOPT_SSLVERSION, 3);
curl_setopt($cURL,CURLOPT_HTTPHEADER,array(
"Accept: application/zip"
));
$rawdata=curl_exec($ch);
curl_close ($ch);
$fp = fopen($saveloc,'w');
fwrite($fp, $rawdata);
fclose($fp);
I've tried every version of curl options i can think of here. I'm new to stack overflow but i can message someone the actual link im using if it helps. Also if it helps here's a raw connection output to the link from fiddler:
CONNECT XXXX:8443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0
Connection: keep-alive
Connection: keep-alive
Host: XXXX
A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below.
Version: 3.1 (TLS/1.0)
Random: XXXX
SessionID: empty
Extensions:
elliptic_curves XXXX
ec_point_formats 01 00
SessionTicket TLS empty
NextProtocolNegotiation empty
Ciphers:
[00FF] TLS_EMPTY_RENEGOTIATION_INFO_SCSV
[C00A] TLS1_CK_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
[C014] TLS1_CK_ECDHE_RSA_WITH_AES_256_CBC_SHA
[0088] TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA
[0087] TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA
[0039] TLS_DHE_RSA_WITH_AES_256_SHA
[0038] TLS_DHE_DSS_WITH_AES_256_SHA
[C00F] TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
[C005] TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
[0084] TLS_RSA_WITH_CAMELLIA_256_CBC_SHA
[0035] TLS_RSA_AES_256_SHA
[C007] TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
[C009] TLS1_CK_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
[C011] TLS_ECDHE_RSA_WITH_RC4_128_SHA
[C013] TLS1_CK_ECDHE_RSA_WITH_AES_128_CBC_SHA
[0045] TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA
[0044] TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA
[0033] TLS_DHE_RSA_WITH_AES_128_SHA
[0032] TLS_DHE_DSS_WITH_AES_128_SHA
[C00C] TLS_ECDH_RSA_WITH_RC4_128_SHA
[C00E] TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
[C002] TLS_ECDH_ECDSA_WITH_RC4_128_SHA
[C004] TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
[0096] TLS_RSA_WITH_SEED_CBC_SHA
[0041] TLS_RSA_WITH_CAMELLIA_128_CBC_SHA
[0005] SSL_RSA_WITH_RC4_128_SHA
[0004] SSL_RSA_WITH_RC4_128_MD5
[002F] TLS_RSA_AES_128_SHA
[C008] TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA
[C012] TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
[0016] SSL_DHE_RSA_WITH_3DES_EDE_SHA
[0013] SSL_DHE_DSS_WITH_3DES_EDE_SHA
[C00D] TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA
[C003] TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA
[FEFF] SSL_RSA_FIPS_WITH_3DES_EDE_SHA
[000A] SSL_RSA_WITH_3DES_EDE_SHA
Compression:
[00] NO_COMPRESSION
Old post but if anyone cares im just an idiot and did a cut and paste blunder so my curl options have two different variables labeled :/
Perhaps you'd have better luck if the Java side set the Content-length header.
Browsers can usually do without it, but here ... ?
I am using cUrl in PHP to request from some external service.
Interesting enough, the server is responding with raw "multipart/form-data" instead of binary file data.
My website is using a shared hosting, therefore PECL HTTP is not an option.
Is there a way to parse this data with PHP?
Sample code:
$response = curl_exec($cUrl);
/* $response is raw "multipart/form-data" string
--MIMEBoundaryurn_uuid_DDF2A2C71485B8C94C135176149950475371
Content-Type: application/xop+xml; charset=utf-8; type="text/xml"
Content-Transfer-Encoding: binary
(xml data goes here)
--MIMEBoundaryurn_uuid_DDF2A2C71485B8C94C135176149950475371
Content-Type: application/zip
Content-Transfer-Encoding: binary
(binary file data goes here)
*/
EDIT: I tried piping the response to a localhost HTTP request, but the respond data is likely to exceed the allowed memory size in PHP process. Expending mem limit is not very practical, this action also dramatically reduces the server performance dramatically.
If there is no alternatives to the original question, you may suggest a way to handle very large POST requests, along with XML parsing, in terms of streams in PHP.
I know this would be hard, please comment. I am open for discussions.
if you need the zip file from the response I guess you could just write a tmp file to save the curl response to, and stream that as a workaround:
Never tried that with multipart curls, but I guess it should work.
$fh = fopen('/tmp/foo', 'w');
$cUrl = curl_init('http://example.com/foo');
curl_setopt($cUrl, CURLOPT_FILE, $fh); // redirect output to filehandle
curl_exec($cUrl);
curl_close($cUrl);
fclose($fh); // close filehandle or the file will be corrupted
if you do NOT need anything but the xml part of the response you might want to disable headers
curl_setopt($cUrl, CURLOPT_HEADER, FALSE);
and add option to only accept xml as a response
curl_setopt($cUrl, CURLOPT_HTTPHEADER, array('Accept: application/xml'));
//That's a workaround since there is no available curl option to do so but http allows that
[EDIT]
A Shot in the dark...
can you test with these curlopt settings to see if modifiying these help anything
$headers = array (
'Content-Type: multipart/form-data; boundary=' . $boundary,
'Content-Length: ' . strlen($requestBody),
'X-EBAY-API-COMPATIBILITY-LEVEL: ' . $compatLevel, // API version
'X-EBAY-API-DEV-NAME: ' . $devID,
'X-EBAY-API-APP-NAME: ' . $appID,
'X-EBAY-API-CERT-NAME: ' . $certID,
'X-EBAY-API-CALL-NAME: ' . $verb,
'X-EBAY-API-SITEID: ' . $siteID,
);
$cUrl = curl_init();
curl_setopt($cUrl, CURLOPT_URL, $serverUrl);
curl_setopt($cUrl, CURLOPT_TIMEOUT, 30 );
curl_setopt($cUrl, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($cUrl, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($cUrl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($cUrl, CURLOPT_POST, 1);
curl_setopt($cUrl, CURLOPT_POSTFIELDS, $requestBody);
curl_setopt($cUrl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($cUrl, CURLOPT_FAILONERROR, 0 );
curl_setopt($cUrl, CURLOPT_FOLLOWLOCATION, 1 );
curl_setopt($cUrl, CURLOPT_HEADER, 0 );
curl_setopt($cUrl, CURLOPT_USERAGENT, 'ebatns;xmlstyle;1.0' );
curl_setopt($cUrl, CURLOPT_HTTP_VERSION, 1 ); // HTTP version must be 1.0
$response = curl_exec($cUrl);
if ( !$response ) {
print "curl error " . curl_errno($cUrl ) . PHP_EOL;
}
curl_close($cUrl);
[EDIT II]
This is just a try, as mentioned I cannot get my curled pages to respond with a multipart form data. So be gentle with me here ;)
$content_type = ""; //use last know content-type as a trigger
$tmp_cnt_file = "tmp/tmpfile";
$xml_response = ""; // this will hold the "usable" curl response
$hidx = 0; //header index.. counting the number of different headers received
function read_header($cUrl, $string)// this will be called once for every line of each header received
{
global $content_type, $hidx;
$length = strlen($string);
if (preg_match('/Content-Type:(.*)/', $string, $match))
{
$content_type = $match[1];
$hidx++;
}
/*
should set $content_type to 'application/xop+xml; charset=utf-8; type="text/xml"' for the first
and to 'application/zip' for the second response body
echo "Header: $string<br />\n";
*/
return $length;
}
function read_body($cUrl, $string)
{
global $content_header, $xml_response, $tmp_cnt_file, $hidx;
$length = strlen($string);
if(stripos ( $content_type , "xml") !== false)
$xml_response .= $string;
elseif(stripos ($content_type, "zip") !== false)
{
$handle = fopen($tmp_cnt_file."-".$hidx.".zip", "a");
fwrite($handle, $string);
fclose($handle);
}
/*
elseif {...} else{...}
depending on your needs
echo "Received $length bytes<br />\n";
*/
return $length;
}
and of course set the proper curlopts
// Set callback function for header
curl_setopt($cUrl, CURLOPT_HEADERFUNCTION, 'read_header');
// Set callback function for body
curl_setopt($cUrl, CURLOPT_WRITEFUNCTION, 'read_body');
don't forget to NOT save the curl response to a variable because of the memory issues,
hopefully all you need will be in the $xml_response above anyways.
//$response = curl_exec($cUrl);
curl_exec($cUrl);
And for parsing your code you can refer to $xml_response and the temp files you created starting with tmp/tmpfile-2 in this scenario. Again, I have not been able to test the code above in any way. So this might not work (but it should imho ;))
[EDIT III]
Say we want curl to write all incoming data directly to another (outgoing) stream, in this case a socket connection
I'm not sure if it is as easy as this:
$fs = fsockopen($host, $port, $errno, $errstr);
$cUrl = curl_init('http://example.com/foo');
curl_setopt($cUrl, CURLOPT_FILE, $fs); // redirect output to sockethandle
curl_exec($cUrl);
curl_close($cUrl);
fclose($fs); // close handle
else we will have to use our known write and header functions with just a little trick
//first open the socket (before initiating curl)
$fs = fsockopen($host, $port, $errno, $errstr);
// now for the new callback function
function socket_pipe($cUrl, $string)
{
global $fs;
$length = strlen($string);
fputs($fs, $string); // add NOTHING to the received line just send it to $fs; that was easy wasn't it?
return $length;
}
// and of course for the CURLOPT part
// Set callback function for header
curl_setopt($cUrl, CURLOPT_HEADERFUNCTION, 'socket_pipe');
// Set the same callback function for body
curl_setopt($cUrl, CURLOPT_WRITEFUNCTION, 'socket_pipe');
// do not forget to
fclose($fs); //when we're done
The thing is, not editing the result and simply piping it to $fs will make it necessary that apache is listening on a certain port which you then assign your script to.
Or you will need to add ONE header line directly after fsockopen
fputs($fp, "POST $path HTTP/1.0\n"); //where path is your script of course
I'm sorry i can't help much because you did not put much code but i remember i was having a similar issue when i was playing with curl_setopt options.
Did you use CURLOPT_BINARYTRANSFER?
From php documentation -> CURLOPT_BINARYTRANSFER-> TRUE to return the raw output when CURLOPT_RETURNTRANSFER is used.
just set CURLOPT_RETURNTRANSFER CURLOPT_POST
$c = curl_init($url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
curl_setopt($c, CURLOPT_CONNECTTIMEOUT, 1);
curl_setopt($c, CURLOPT_TIMEOUT, 1);
curl_setopt($c, CURLOPT_POST, 1);
curl_setopt($c, CURLOPT_POSTFIELDS,
array());
$rst_str = curl_exec($c);
curl_close($c);
You can re-assemble you Binary data doing something like this, I hope it helps.
$file_array = explode("\n\r", $file, 2);
$header_array = explode("\n", $file_array[0]);
foreach($header_array as $header_value) {
$header_pieces = explode(':', $header_value);
if(count($header_pieces) == 2) {
$headers[$header_pieces[0]] = trim($header_pieces[1]);
}
}
header('Content-type: ' . $headers['Content-Type']);
header('Content-Disposition: ' . $headers['Content-Disposition']);
echo substr($file_array[1], 1);
If you don't need binary data, have you tried below?
curl_setopt($c, CURLOPT_NOBODY, true);
I have some PHP code that was at one point WORKING FINE. It makes a call out to an external API, the API has NOT CHANGED AT ALL. The PHP code has also NOT CHANGED AT ALL. But suddenly, I am getting no results back for this function:
if (!function_exists(setFieldsAndCallURL))
{
function setFieldsAndCallURL($url,$fields)
{
//url-ify the data for the POST
$fields_string='';
foreach($fields as $key=>$value)
{ $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string,'&');
//open connection
$ch = curl_init();
//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//execute the jump
$result = '';
$result = curl_exec($ch);
//close connection
curl_close($ch);
return $result;
}
}
Before, it would return a text GUID when called:
$userURL ='https://api.nottherealendpointurl.net/public/user/authenticate';
$userFields = array(
'username'=>$username,
'lastName'=>$lastname,
'firstName'=>$firstname,
'email'=>$email,
'token'=>urlencode($adminKey),
);
//Login this particular user
$userKey = setFieldsAndCallURL($userURL,$userFields);
But suddenly it has started returning "" (empty string) and I have no idea why.
Is there any way to get more info and spy on the inner workings of this thing? See the call it is making using HTTP header logging software? Or anything?
NOTE: I have already tested the POST manually to the API and it is working as expected, I am still getting back the proper GUID. For some reason doing it through this curl thing just suddenly quit doing it properly. Nobody has any idea what could be different now.
Propably the ip of the server is blocked by now, where your local ip is not?
You might want to add
$headerFile = fopen(filepath_to_header_file);
$errorFile = fopen(filepath_to_error_file);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_WRITEHEADER, $headerFile );
curl_setopt($ch, CURLOPT_STDERR, $errorFile );
to get the header of the response and the errors into files and look their content up.
edit:
To verify if the ip of the server is blocked you could try something like this
$host = "ssl://api.nottherealendpointurl.net/";
$port = 443;
$url = "/public/user/authenticate";
$timeout = 30;
$errno = "";
$errstr= "";
$fp = fsockopen($host, $port, $errno, $errstr, $timeout);
if($fp)
{
$request = "GET ".$url." HTTP/1.1\r\n";
$request.= "Host: ".$host."\r\n";
$request.= "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.12) Gecko/20050919 Firefox/1.0.7\r\n";
$request.= "Connection: Close\r\n\r\n";
fwrite($fp, $request);
while (!feof($fp))
{
$data .= fgets($fp, 128);
}
fclose($fp);
echo $data;
}
else
{
echo "ERROR: ".$errstr;
}
where $data contains the response from the remote server-
Depending on what platform you're on you can look at the raw packets, for linux command line only, that'd be tcpdump for widows/others you can use wireshark.
tcpdump -i eth1 tcp port 80
or
http://www.wireshark.org/download.html
In PHP, how can I replicate the expand/contract feature for Tinyurls as on search.twitter.com?
If you want to find out where a tinyurl is going, use fsockopen to get a connection to tinyurl.com on port 80, and send it an HTTP request like this
GET /dmsfm HTTP/1.0
Host: tinyurl.com
The response you get back will look like
HTTP/1.0 301 Moved Permanently
Connection: close
X-Powered-By: PHP/5.2.6
Location: http://en.wikipedia.org/wiki/TinyURL
Content-type: text/html
Content-Length: 0
Date: Mon, 15 Sep 2008 12:29:04 GMT
Server: TinyURL/1.6
example code...
<?php
$tinyurl="dmsfm";
$fp = fsockopen("tinyurl.com", 80, $errno, $errstr, 30);
if (!$fp) {
echo "$errstr ($errno)<br />\n";
} else {
$out = "GET /$tinyurl HTTP/1.0\r\n";
$out .= "Host: tinyurl.com\r\n";
$out .= "Connection: Close\r\n\r\n";
$response="";
fwrite($fp, $out);
while (!feof($fp)) {
$response.=fgets($fp, 128);
}
fclose($fp);
//now parse the Location: header out of the response
}
?>
And here is how to contract an arbitrary URL using the TinyURL API. The general call pattern goes like this, it's a simple HTTP request with parameters:
http://tinyurl.com/api-create.php?url=http://insertyourstuffhere.com
This will return the corresponding TinyURL for http://insertyourstuffhere.com. In PHP, you can wrap this in an fsockopen() call or, for convenience, just use the file() function to retrieve it:
function make_tinyurl($longurl)
{
return(implode('', file(
'http://tinyurl.com/api-create.php?url='.urlencode($longurl))));
}
// make an example call
print(make_tinyurl('http://www.joelonsoftware.com/items/2008/09/15.html'));
As people have answered programatically how to create and resolve tinyurl.com redirects, I'd like to (strongly) suggest something: caching.
In the twitter example, if every time you clicked the "expand" button, it did an XmlHTTPRequest to, say, /api/resolve_tinyurl/http://tinyurl.com/abcd, then the server created a HTTP connection to tinyurl.com, and inspected the header - it would destroy both twitter and tinyurl's servers..
An infinitely more sensible method would be to do something like this Python'y pseudo-code..
def resolve_tinyurl(url):
key = md5( url.lower_case() )
if cache.has_key(key)
return cache[md5]
else:
resolved = query_tinyurl(url)
cache[key] = resolved
return resolved
Where cache's items magically get backed up into memory, and/or a file, and query_tinyurl() works as Paul Dixon's answer does.
Here is another way to decode short urls via CURL library:
function doShortURLDecode($url) {
$ch = #curl_init($url);
#curl_setopt($ch, CURLOPT_HEADER, TRUE);
#curl_setopt($ch, CURLOPT_NOBODY, TRUE);
#curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
#curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$response = #curl_exec($ch);
preg_match('/Location: (.*)\n/', $response, $a);
if (!isset($a[1])) return $url;
return $a[1];
}
It's described here.
Another simple and easy way:
<?php
function getTinyUrl($url) {
return file_get_contents('http://tinyurl.com/api-create.php?url='.$url);
}
?>
If you just want the location, then do a HEAD request instead of GET.
$tinyurl = 'http://tinyurl.com/3fvbx8';
$context = stream_context_create(array('http' => array('method' => 'HEAD')));
$response = file_get_contents($tinyurl, null, $context);
$location = '';
foreach ($http_response_header as $header) {
if (strpos($header, 'Location:') === 0) {
$location = trim(strrchr($header, ' '));
break;
}
}
echo $location;
// http://www.pingdom.com/reports/vb1395a6sww3/check_overview/?name=twitter.com%2Fhome
In PHP there is also a get_headers function that can be used to decode tiny urls.
The Solution here from #Pons solution, didn't work alone on my php7.3 server reslolving stackexchange URLs like https://stackoverflow.com/q/62317
This solved it:
public function doShortURLDecode($url) {
$ch = #curl_init($url);
#curl_setopt($ch, CURLOPT_HEADER, TRUE);
#curl_setopt($ch, CURLOPT_NOBODY, TRUE);
#curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
#curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$response = #curl_exec($ch);
$cleanresponse= preg_replace('/[^A-Za-z0-9\- _,.:\n\/]/', '', $response);
preg_match('/Location: (.*)[\n\r]/', $cleanresponse, $a);
if (!isset($a[1])) return $url;
return parse_url($url, PHP_URL_SCHEME).'://'.parse_url($url, PHP_URL_HOST).$a[1];
}