So I'm struggling with the code below. I have java that is serving a file and php that is trying to grab the file and save it. I can download the file from the servlet just fine in a browser. I can download any file or webpage (SSL or not) with the php code. But FOR THE LIFE OF ME i cant get the php to download from this servlet!!! Im pretty familiar with PHP but not so much with java and i didnt write that servlet if it helps. I can however edit either end of the code. Thank you!!
JAVA:
#Override
public void doGet( HttpServletRequest request, HttpServletResponse response ) throws ServletException, IOException {
try {
String fileName = "/tmp/DongleUpdate.jar";
File f = new File( "/tmp/DongleUpdate.jar" );
response.setContentType( "application/zip" );
response.setHeader( "Content-Disposition", "attachment;filename=\"" +
fileName + "\"" );
byte[] arBytes = new byte[( int ) f.length()];
FileInputStream is = new FileInputStream( f );
is.read( arBytes );
ServletOutputStream op = response.getOutputStream();
op.write( arBytes );
op.flush();
} catch( IOException ioe ) {
ioe.printStackTrace();
}
}
PHP:
$ch = curl_init($file);
curl_setopt($ch, CONNECTTIMEOUT, 10);
curl_setopt($ch, TIMEOUT, 60);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_FRESH_CONNECT, true);
curl_setopt($curl, CURLOPT_GET, true);
curl_setopt($curl, CURLOPT_SSLVERSION, 3);
curl_setopt($cURL,CURLOPT_HTTPHEADER,array(
"Accept: application/zip"
));
$rawdata=curl_exec($ch);
curl_close ($ch);
$fp = fopen($saveloc,'w');
fwrite($fp, $rawdata);
fclose($fp);
I've tried every version of curl options i can think of here. I'm new to stack overflow but i can message someone the actual link im using if it helps. Also if it helps here's a raw connection output to the link from fiddler:
CONNECT XXXX:8443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0
Connection: keep-alive
Connection: keep-alive
Host: XXXX
A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below.
Version: 3.1 (TLS/1.0)
Random: XXXX
SessionID: empty
Extensions:
elliptic_curves XXXX
ec_point_formats 01 00
SessionTicket TLS empty
NextProtocolNegotiation empty
Ciphers:
[00FF] TLS_EMPTY_RENEGOTIATION_INFO_SCSV
[C00A] TLS1_CK_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
[C014] TLS1_CK_ECDHE_RSA_WITH_AES_256_CBC_SHA
[0088] TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA
[0087] TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA
[0039] TLS_DHE_RSA_WITH_AES_256_SHA
[0038] TLS_DHE_DSS_WITH_AES_256_SHA
[C00F] TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
[C005] TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
[0084] TLS_RSA_WITH_CAMELLIA_256_CBC_SHA
[0035] TLS_RSA_AES_256_SHA
[C007] TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
[C009] TLS1_CK_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
[C011] TLS_ECDHE_RSA_WITH_RC4_128_SHA
[C013] TLS1_CK_ECDHE_RSA_WITH_AES_128_CBC_SHA
[0045] TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA
[0044] TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA
[0033] TLS_DHE_RSA_WITH_AES_128_SHA
[0032] TLS_DHE_DSS_WITH_AES_128_SHA
[C00C] TLS_ECDH_RSA_WITH_RC4_128_SHA
[C00E] TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
[C002] TLS_ECDH_ECDSA_WITH_RC4_128_SHA
[C004] TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
[0096] TLS_RSA_WITH_SEED_CBC_SHA
[0041] TLS_RSA_WITH_CAMELLIA_128_CBC_SHA
[0005] SSL_RSA_WITH_RC4_128_SHA
[0004] SSL_RSA_WITH_RC4_128_MD5
[002F] TLS_RSA_AES_128_SHA
[C008] TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA
[C012] TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
[0016] SSL_DHE_RSA_WITH_3DES_EDE_SHA
[0013] SSL_DHE_DSS_WITH_3DES_EDE_SHA
[C00D] TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA
[C003] TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA
[FEFF] SSL_RSA_FIPS_WITH_3DES_EDE_SHA
[000A] SSL_RSA_WITH_3DES_EDE_SHA
Compression:
[00] NO_COMPRESSION
Old post but if anyone cares im just an idiot and did a cut and paste blunder so my curl options have two different variables labeled :/
Perhaps you'd have better luck if the Java side set the Content-length header.
Browsers can usually do without it, but here ... ?
Related
I'm currently running a personal file upload site, which roughly does the following;
Select files to upload through the website
PHP uploads and zips the files together
The zipped file gets sent via cURL to a remote storage server
A URL is presented to share the files
I'm having an issue with step 3. I currently have a progress bar while the file uploads to the website and I'd like another percentage to show the cURL transfer to the storage server. I've seen that you that can use CURLOPT_PROGRESSFUNCTION - but I can not get it working, after a lot of searching.
My current code is as below;
// Prepare the POST data
$data = array(
'file' => new CurlFile('/path/to/local.zip', 'application/zip', 'uploaded_archive.zip'),
'upload_folder' => 'folder_name'
);
// Send the POST
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://example.com/upload.php");
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0");
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_NOPROGRESS, false);
curl_setopt($ch, CURLOPT_PROGRESSFUNCTION, 'progressCall');
curl_setopt($ch, CURLOPT_BUFFERSIZE, 128);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 160);
curl_setopt($ch, CURLOPT_TIMEOUT, 160);
$res = curl_exec($ch);
curl_close($ch);
This calls "progressCall" with the progress, which looks like the below to test;
// Function to save the output of progress from CURLOPT_PROGRESSFUNCTION
function progressCall($ch, $dltotal, $dlnow, $uptotal, $upnow) {
$progress = "$ch/$dltotal/$dlnow/$uptotal/$upnow";
file_put_contents('tmp/curlupload.txt', 'PERC: ' . $progress . "\n", FILE_APPEND);
}
The problem is, I get the output below;
https://gist.github.com/ialexpw/6160e32495d857cc0d8984373f3808ae
PERC: Resource id #15/0/0/0/65532
PERC: Resource id #15/0/0/0/65532
PERC: Resource id #15/0/0/0/131064
PERC: Resource id #15/0/0/0/131064
PERC: Resource id #15/0/0/0/196596
This seems to show the uploaded amount going up (which is OK), but has 0 for the total upload amount. I've tried a lot of example online, but I can not work out why the total amount to be uploaded is always 0. I tried setting a Content-Length, although it's the same.
Any help would be appreciated if something is wrong with the above.
Thanks!
Hello sorry for the response 2 years after, but just working on it !
Possible for Upload, but as mentionned we don't have in this example the total file size.
The easiest way if possible is to take the file size on the callback
function, with a global for example :
function progressCall($ch, $dltotal, $dlnow, $uptotal, $upnow) {
// You will get this before with | $fsize = filesize($thefile);
global $fsize;
$percent = round($upnow / $fsize * 100);
echo $percent . ' % downloaded';
}
What I'm trying to achieve:
Get request to an API Endpoint, retrieving an XML and subsequently parse the results.
I am sending a file_get_contents request to achieve this.
Issues:
`file_get_Contents` fails, error:
Warning: file_get_contents(https://api.twitter.com/1.1/statuses/mentions_timeline.json):
failed to open stream:
A connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond.
Update 17/08
To consolidate my current understanding:
1. PHP FAILS:
1.a it fails via php (timeout)
1.b it fails via command line (curl -G http://api.eve-central.com/api/quicklook?typeid=34)
1.c file_get_contents
1.d file_get_contents w/ create_stream_context
2. What WORKS:
2.a Pasting the url in a chrome tab
2.b via postman
What has been attempted:
- Check Headers in Postman ,and try to replicate them via php
Postman Headers sent back by eve-central:
Access-Control-Allow-Origin → *
Connection → Keep-Alive
Content-Encoding → gzip
Content-Type → text/xml; charset=UTF-8
Date → Wed, 17 Aug 2016 10:40:24 GMT
Proxy-Connection → Keep-Alive
Server → nginx
Transfer-Encoding → chunked
Vary → Accept-Encoding
Via → HTTP/1.1 proxy10014
Corresponding Code:
$headers = array(
'method' => 'GET',
'header' => 'Connection: Keep-Alive',
'header' => 'Content-Encoding: gzip',
'header' => 'Content-Type: text/xml',
'header' => 'Proxy-Connection: Keep-Alive',
'header' => 'Server: nginx',
'header' => 'Transfer-Encoding: chunked',
'header' => 'Vary: Accept-Encoding',
'header' => 'Via: HTTP/1.1 proxy10014');
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt($curl, CURLOPT_PORT , 8080); // Attempt at changing port in the event it was blocked.
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_POST, false );
curl_setopt($curl, CURLOPT_URL, $url );
$resp = curl_exec($curl);
if(curl_error($curl))
{
echo 'error:' . curl_error($curl);
}
Use Wireshark to capture the GET request to see if changing the port helped
Run cUrl via command line
I'm out of ideas and option.
So the questions are:
If it works in a browser, and in Postman, why does it not work via PHP ?
How can I modify my code so that it mimics what Postman does? ?
Previous Attempts
What I have tried:
Various cURL options from other threads, such as
function curl_get_contents($url) {
$ch = curl_init();
if (!$ch)
{
die("Couldn't initialize a cURL handle");
} else
echo "Curl Handle initialized ";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
$data = curl_exec($ch);
// Check if any error occurred
if (!curl_errno($ch))
{
$info = curl_getinfo($ch);
echo 'Took ', $info['total_time'], ' seconds to send a request to ', $info['url'], "";
displayData($info);
} else
echo "Failed Curl, reason: ".curl_error($ch)." ";
curl_close($ch);
return $data;
}
result: nothing, no data returned.
- Checked php.ini options:
- allow_fopen is On
- allow_url_include = on
- relevant ssl extensions are enabled
- Raised the timeout window
- both via php.ini
- also via explicit declaration within the php file.
- Tried with a different url
- same error, so it doesn't really depends on my particular endpoint
- for example, both twitter/wikipedia/google return the specific error
- tried with:
- file_get_contents on a local xml file (https://msdn.microsoft.com/en-us/library/ms762271(v=vs.85).aspx) --> works
- file_get_contents on a remote xml file (http://www.xmlfiles.com/examples/note.xml) --> fails same error
- Overall, the following is true, so far:
- Curl fails, timeout
- file_get_Contents fails, timeout
- Open XML file url in a browser works
- Make a GET request via Postman, works
Obviously, in all cases where the file_get_contents fails via php, I can easily access the file via any browser.
Tried to work around the issue.
Attempt 1:
Use nitrous.io, create a LAMP stack, perform the deed via the platform
results: file_get_contents works, however, due to the large number of xml files to be retrieved, the operation times-out.
Tentative solution:
- Download XML files from source
- Zip them
- Download xml_file
- Locally parse said xml files
Later on, write a small php scripts that, when invoked, performs the bits above, sends the data to the local directory, which then unpacks it and performs additional work on it.
Another attempt would be to use Google Sheets, with a user function that pulls the data into the sheet, and just dump the excel file / values into mysql.
For my purposes, while an awfully ignorant solution, it does the trick.
Code used for avoiding timeout issue on shared host:
function downloadUrlToFile2($url, $outFileName)
{
//file_put_contents($xmlFileName, fopen($link, 'r'));
//copy($link, $xmlFileName); // download xml file
;
echo "Passing $url into $outFileName ";
// $outFileName = touch();
$fp = fopen($outFileName, "w");
if(is_file($url))
{
copy($url, $outFileName); // download xml file
} else
{
$ch = curl_init();
$options = array(
CURLOPT_TIMEOUT => 28800, // set this to 8 hours so we dont timeout on big files
CURLOPT_URL => $url
);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt_array($ch, $options);
$contents = curl_exec($ch);
fwrite($fp, $contents);
curl_close($ch);
}
}
I have also added this on top of the ini script:
ignore_user_abort(true);
set_time_limit(0);
ini_set('memory_limit', '2048M');
I see some issue with HTTPS url request, for fix issue you have to add below lines in your CURL request
function curl_get_contents($url) {
$ch = curl_init();
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: ";
curl_setopt( $ch, CURLOPT_HTTPHEADER, $header );
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
// I have added below two lines
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
I am not much familiar with php and curl, need to convert an advance PHP cURL POST request to python equivalent.
It's a code from payment gateway site called paygate, and am using their sample php API from developer.paygate.co.za/. The code that I tried to convert into python is below:
<?php
//The PayGate PayXML URL
define( "SERVER_URL", "https://www.paygate.co.za/payxml/process.trans" );
//Construct the XML document header
$XMLHeader = "<?xml version=\"1.0\" encoding=\"UTF-8\"?><!DOCTYPE protocol SYSTEM \"https://www.paygate.co.za/payxml/payxml_v4.dtd\">";
// - Then construct the full transaction XML
$XMLTrans = '<protocol ver="4.0" pgid="10011013800" pwd="test"><authtx cref="ABCqwerty1234" cname="Patel Sunny" cc="5200000000000015" exp="032022" budp="0" amt="10000" cur="ZAR" cvv="123" rurl="http://localhost/pg_payxml_php_final.php" nurl="http://localhost/pg_payxml_php_notify.php" /></protocol>'
// Construct the request XML by combining the XML header and transaction
$Request = $XMLHeader.$XMLTrans;
// Create the POST data header containing the transaction
$header[] = "Content-type: text/xml";
$header[] = "Content-length: ".strlen($Request)."\r\n";
$header[] = $Request;
// Use cURL to post the transaction to PayGate
// - first instantiate cURL; if it fails then quit now.
$ch = curl_init();
if (!$ch) die("ERROR: cURL initialization failed. Check your cURL/PHP configuration.");
// - then set the cURL options; to ignore SSL invalid certificates; set timeouts etc.
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
curl_setopt ($ch, CURLOPT_CUSTOMREQUEST, "POST");
// - then set the PayXML URL and the transaction data
curl_setopt ($ch, CURLOPT_URL, SERVER_URL);
curl_setopt ($ch, CURLOPT_HTTPHEADER, $header);
// Connect to PayGate PayXML and send data
$Response = curl_exec ($ch);
// Checl for any connection errors and then close the connection.
$curlError = curl_errno($ch);
curl_close($ch);
I know about basic requests in python but couldn't pass attributes in that request, I am also confused about passing cURL data in requests.
I am trying it like:
import requests
post_data = {'pgid':'10011013800',
'pwd':'test',
'cref': 'ABCX1yty36858gh',
'cname':'PatelSunny',
'cc':'5200000470000015',
'exp':'032022',
'budp':'0',
'amt':'50000',
'cur':'ZAR',
'cvv':'123',
'rurl':'http://localhost/pg_payxml_php_final.php',
'nurl':'http://localhost/pg_payxml_php_notify.php',
'submit':'Submit'
}
r = requests.get('https://www.paygate.co.za/payxml/process.trans', params=post_data,headers=headers)
# print(r.url)
print r.text
But it shows error
405 - HTTP verb used to access this page is not allowed.
Finally solved it,
import requests
import xml.etree.ElementTree as ET
headers = {'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'Accept-Encoding':'gzip, deflate',
'Accept-Language':'en-US,en;q=0.8',
'Cache-Control':'max-age=0',
'Connection':'keep-alive',
'Content-Length':'112',
'Content-Type':'application/x-www-form-urlencoded',
'User-Agent':"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.81 Safari/537.36",}
xml = """<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE protocol SYSTEM "https://www.paygate.co.za/payxml/payxml_v4.dtd">
<protocol ver="4.0" pgid="10011013800" pwd="test">
<authtx cref="ABCX1j64564" cname="Patel Sunny" cc="5200000000000015" exp="032022" budp="0" amt="10000" cur="ZAR" cvv="123"
rurl="http://localhost/pg_payxml_php_final.php" nurl="http://localhost/pg_payxml_php_notify.php" />
</protocol>
"""
headers = {'Content-Type': 'application/xml'} # set what your server accepts
response = requests.post('https://www.paygate.co.za/payxml/process.trans', data=xml, headers=headers).text
tree = ET.fromstring(response)
for node in tree.iter('authrx'):
sdesc = node.attrib.get('sdesc') # STATUS MESSAGE
tid = node.attrib.get('tid') # TRANSACTION ID
cref = node.attrib.get('cref') # REFERENCE NO. like invoice_no or sale_order_no
auth = node.attrib.get('auth')
rdesc = node.attrib.get('rdesc') # Result Code description.
print sdesc, tid, cref
I am building a web application for my router, it will be my Bachelor's Thesis.
The bad thing is that I can't display my router's informations using my cURL function because I get bad router username and password error. I didn't found any problem at all:
The cURL function:
function myCurl($url, $post="")
{
global $status;
$header = 'Authorization: Basic YWRtaW46YWRtaW4=';
$cookiepath_tmp = "c:/xampp/htdocs/wifi/cookie.txt";
$resp = array();
$ch = curl_init();
curl_setopt($ch,CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1" );
curl_setopt($ch,CURLOPT_URL, trim($url));
curl_setopt($ch,CURLOPT_REFERER, trim($url));
curl_setopt($ch,CURLOPT_COOKIEJAR,$cookiepath_tmp);
curl_setopt($ch,CURLOPT_COOKIEFILE,$cookiepath_tmp);
curl_setopt($ch,CURLOPT_COOKIESESSION, true);
curl_setopt($ch,CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch,CURLOPT_MAXREDIRS, 10);
curl_setopt($ch,CURLOPT_ENCODING, "");
curl_setopt($ch,CURLOPT_RETURNTRANSFER, true);
#curl_setopt($ch,CURLOPT_AUTOREFERER, true);
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT, 15);
curl_setopt($ch,CURLOPT_TIMEOUT, 15);
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false );
curl_setopt($ch,CURLOPT_HEADER, 0);
curl_setopt($ch,CURLOPT_HTTPHEADER, array( 'Expect:' ) );
curl_setopt($ch,CURLOPT_VERBOSE, 1);
#curl_setopt($ch,CURLOPT_FAILONERROR, true);
if($post) { curl_setopt($ch, CURLOPT_POST,1); curl_setopt($ch,CURLOPT_POSTFIELDS,$post); }
$returned = curl_exec($ch);
$resp['returned'] = $returned;
$status=curl_getinfo($ch);
$resp['status'] = $status;
curl_close($ch);
return $resp;
}
I am trying to display the informations using PHP:
The PHP code:
<?php echo $success_msg;
$url = "http://192.168.0.1/session.cgi";
$post = "REPORT_METHOD=xml&ACTION=login_plaintext&USER=admin&PASSWD=admin&CAPTCHA=";
$data = myCurl($url, $post);
#$url = "http://192.168.0.1/st_log.php";
#$data = myCurl($url);
echo $data['returned'];
?>
The error is:
Username or Password is incorrect.
However, The username and password admin are correct.
I have added the following code into myCurl function but still doesn't work:
$header = 'Authorization: Basic YWRtaW46YWRtaW4=';
YWRtaW46YWRtaW4= is the encoded username:password in Base64.
LAST EDIT:
I set the CURLOPT_HEADER to true, and I got this text displayed:
HTTP/1.1 501 Not Implemented Server: Router Webserver Connection: close WWW-Authenticate: Basic realm="TP-LINK Wireless Lite N Router WR740N" Content-Type: text/html
Any solution for this?
I really appreciate your help! Thank you!
I don't known what is your router (vendor / model) but most of them use HTTP basic authentication. And, when the authentication is empty or wrong you get a HTTP 401 error: Unauthorized, which could correspond to your error string.
So you should try to insert a HTTP authorization header in the cURL request:
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
Is there a way for PHP CURL functions to get the contents of the website, but stopped on the characters that we just ask. I think this sort of buffer.
so the script did not call the overall page
So schemes like this:
: curl execution
<html>
->
->
->
-> Title Detected
: curl close
->
->
->
->
</ html>
Please this is not a DOM problem. But how to curl stops when it finds that we ask.
this is my code :
function curl_download($Url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $Url);
curl_setopt($ch, CURLOPT_REFERER, $Url);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.46 Safari/536.5");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
If cURL can't handle this problem, how about fopen? and do you have an example?
Thanks before.Also please give me your example code for me,, thanks
Here is a very simple example using fsockopen(). Extend it to fit your needs.
$host = 'www.site.com';
$port = 80;
$sock = fsockopen($host, $port, $errno, $errstr, 30);
if (!$sock) {
die("Failed to connect. $errno: $errstr");
}
// write http request to socket:
$request = "GET /file.html HTTP/1.0\r\n"
."Host: $host\r\n"
."User-Agent: some-user-agent\r\n"
."Connection: close\r\n"
."\r\n";
fwrite($sock, $request);
$buffer = ''; // buffer for storing response
while (!feof($sock)) {
$buffer .= fgets($sock, 1024); // read 1024 bytes from socket, append to buffer
if (strpos($buffer, '</title>') !== false)) { // title was found
fclose($sock);
break;
}
}
So we connect to the HTTP server on the remote host, issue a simple HTTP/1.0 request, and read the response 1024 bytes at a time until the closing title tag is detected. Once it is found, the connection is closed.
Note, even though we didn't read the entire response from the socket, the underlying system (PHP and the OS socket layer) may have read more (or possibly all depending on size) of the response. In either case, you did prevent PHP from reading most of the response. If the pages are very big, closing the socket early will likely prevent a bulk of the data from actually ever being received.
Hope that helps.
I do not think you can parse the DOM with CURL
I advise you to use the string function like strstr, strtok...
http://www.php.net/manual/en/ref.strings.php