I'm trying to create a script that posts to an URL, using a list of proxies. The script seems to work fine without setting a proxy (CURLOPT_PROXY to null), but with a proxy set it does not seem to work. Does anybody know what I'm doing wrong here? The error I'm getting when posting with a proxy is:
ERROR
The requested URL could not be retrieved
Invalid Request error was encountered while trying to process the request:
POST /###/ HTTP/1.1
Host: ###
Accept: */*
Proxy-Connection: Keep-Alive
Content-Length: 368
Expect: 100-continue
Content-Type: multipart/form-data; boundary=----------------------------9b24e849cac0
Some possible problems are:
Missing or unknown request method.
Missing URL.
Missing HTTP Identifier (HTTP/1.0).
Request is too large.
Content-Length missing for POST or PUT requests.
Illegal character in hostname; underscores are not allowed.
And my code is:
$url = 'urltopost';
$proxy = 'proxy:port';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_POST, true);
$data = array(
'postname1' => 'value1',
'postname2' => 'value2'
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
You are missing the PROXYPORT option... You can't use the proxy:port notation
curl_setopt($ch, CURLOPT_PROXY, "YOUR PROXY HOST");
curl_setopt($ch, CURLOPT_PROXYPORT, "YOUR PROXY PORT");
Related
Setting up a JSON-RPC on my vps which I want to connect via PHP CURL on my website doing a basic request and looking for getmasternodecount.
Tried many scripts and libraries before however none seems to work in my case. Now I try to write some basic php code, but this skill isnt my best.
<?php
error_reporting(E_ALL);
ini_set('display_errors', '1');
function coinFunction () {
$feed = 'http://user:pass#ip/';
$post_string = '{"method": "getmasternodecount", "params": []}';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $feed);
curl_setopt($ch, CURLOPT_PORT, port);
curl_setopt($ch, CURLOPT_USERPWD, "user:pass");
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_string);
//curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-type: application/stratum', 'Content-length: '.strlen($post_string)));
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-type: application/json', 'Content-length: '.strlen($post_string)));
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
$data = coinFunction();
var_dump($data);
echo $data;
?>
And gives me this data dump:
string(127) "HTTP/1.1 403 Forbidden Date: Sun, 24 May 2020 00:06:21 GMT Content-Length: 0 Content-Type: text/html; charset=ISO-8859-1 " HTTP/1.1 403 Forbidden Date: Sun, 24 May 2020 00:06:21 GMT Content-Length: 0 Content-Type: text/html; charset=ISO-8859-1
When i delete all the var dump information etc, it send me a whitepage and sometimes NULL.
Kindly Regards,
Let's work with the first snippet. Since it's a POST request, file_get_contents is rather out of place here. Add the following setopt lines:
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_HEADER, 0);
Without those, the result of curl_exec won't contain the returned content.
It would also be advisable to specify the Content-Type of the request (which is application/json). The server might handle it even without, but just in case:
curl_setopt($curl, CURLOPT_HTTPHEADER, array(
'Content-Type:application/json'));
Authentication is another thing. Credentials in the URL suggest Basic, but the server might expect otherwise... See CURLOPT_HTTPAUTH.
I'm using a local php script that runs curl on a script on a remote server. The remote script is protected with http authentication.
When I run my local script, I receive the following error:
"This server could not verify that you are authorized to access the URL "/script.php". You either supplied the wrong credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required.
In case you are allowed to request the document, please check your user-id and password and try again.
If you think this is a server error, please contact the webmaster.
Error 401"
I've tried the highest-voted answers from here: How do I make a request using HTTP basic authentication with PHP curl?
and I've tried this: https://thisinterestsme.com/php-curl-http-auth/
-> I always receive the same error from above.
I have $username and $password defined. The code I'm using:
$url = 'https://' . $page . '/script.php';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/xml', $additionalHeaders));
print_r($additionalHeaders);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_USERPWD, $username . ":" . $password);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $payloadName);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$return = curl_exec($ch);
print_r($return);
curl_close($ch);
After printing the $additionalHeaders variable I get the following additional information before the error:
"HTTP/1.1 401 Unauthorized Date: Sat, 10 Aug 2019 22:05:26 GMT Server: Apache WWW-Authenticate: Digest realm="Protected", nonce="Xg78fMqPBQA=8d674639fbd19b9ec67b085aa43e90be7a4c57cc", algorithm=MD5, qop="auth" Vary: accept-language,accept-charset Strict-Transport-Security: max-age=16000000 Upgrade: h2 Connection: Upgrade Accept-Ranges: bytes Transfer-Encoding: chunked Content-Type: text/html; charset=utf-8 Content-Language: en"
Printing $return does not show anything ...
your website isn't using HTTP Basic Auth at all, it's using Digest access authentication, google CURLAUTH_DIGEST,
curl_setopt($ch,CURLOPT_HTTPAUTH,CURLAUTH_DIGEST);
The working solution in its entirety (except for $url $username and $password) looks like this:
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_DIGEST);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_USERPWD, $username . ":" . $password);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $payloadName);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$return = curl_exec($ch);
curl_close($ch);
I'm working with this Phalcon PHP API HMAC framework and I have a little question.
I declared a GET route in routes.php and Im trying to make a request from client-connect.php (a simple CURL script to make requests).
That's how the client looks like:
$privateKey = '593fe6ed77014f9502761028801aa376f141916bd26b1b3f0271b5ec3135b989';
$time = time();
$id = 1;
$data = [
'name' => 'bob',
];
$message = buildMessage($time, $id, $data);
$hash = hash_hmac('sha256', $message, $privateKey);
$headers = ['API_ID: ' . $id, 'API_TIME: ' . $time, 'API_HASH: ' . $hash];
$ch = curl_init();
curl_setopt($ch, CURLOPT_VERBOSE, TRUE);
curl_setopt($ch, CURLOPT_URL, $host);
// curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "DELETE");
curl_setopt($ch, CURLOPT_POST, FALSE);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE);
When I make a post request, all works ok, but when I try to make a GET, PUT or DELETE request (I just uncomment the line commented in client), it returns me an error:
Request:
DELETE /nodes/mikrotik HTTP/1.1
Host: vpn.wibee.com
Accept: */*
API_ID: 1
API_TIME: 1446031137
API_HASH: 4d0852239859da5e90270b3f7dfd2167f6e5153ca83a9c5896ee262e41b19674
Content-Length: 508
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------e71b5fd144802749
Response:
HTTP/1.1 100 Continue
HTTP/1.1 401 Unauthorized
Server: nginx/1.6.2
Date: Wed, 28 Oct 2015 11:18:58 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Access denied
I think it's posible im missing something (something related with authentication). Any solution?
Thanks!
I would try it. In original implementation of client app you have such a part:
curl_setopt($ch, CURLOPT_VERBOSE, TRUE);
curl_setopt($ch, CURLOPT_URL, $host);
switch($method) {
case 'POST':
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
break;
case 'GET':
break;
default:
$data = http_build_query($data);
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
break;
}
And your implementation is quite different. Once you are using POST method, functionality surprisingly work even with ($ch, CURLOPT_POST, FALSE). But Once using all other requests, you are running into troubles. It's because your implementation of GET does a few things it should not:
curl_setopt($ch, CURLOPT_POST, FALSE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
this is not correct for making GET requests with CURL. It surprisingly works, but you are receiving access denied. It is probably because authentication data, are not properly iterpreted on server side.
Also other methods will have some troubles to work, because you are mising this part of original code:
$data = http_build_query($data);
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
My best solution wold be to advise you, to try to use existing code sample instead of rewriting it by hand in buggy way. Extending that code for your use should not be complicated, especially if you are running it from shell.
I need to perform an Http Request with the CURL method.
Here is the documentation of the service I have to connect with.
In order to invoke an API method, the client makes an HTTPS POST request on
https://app.kigo.net/api/ra/v1/method.
The request Content-Type header must be set to application/json and the post data (HTTP request body) must hold a JSON representation of input data.
Example of a valid HTTP request:
POST /api/ra/v1/ping HTTP/1.0
Host: app.kigo.net
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
Content-Type: application/json
{
"PING" : "PONG"
}
The question is:
How to do it with CURL?
You need the SSL options for the https connection
followlocation will follow any redirects
return_transfer will save the response in a variable
$url = 'https://app.kigo.net/api/ra/v1/ping';
$headers = array( 'Authorization' => 'Basic dXNlcm5hbWU6cGFzc3dvcmQ=', 'Content-Type' => 'application/json' );
$ch = curl_init();
curl_setopt($ch,CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch,CURLOPT_POSTFIELDS, "PING=PONG");
$result = curl_exec($ch);
curl_close($ch);
print_r( $result );
i want to change xml value in php but don't change ....
<?php
$url = "http://192.168.1.103:8080/ew.xml";
$xml = '<?xml version="1.0" ?>
<DDCConfig:getValue xmlns:DDCConfig="urn:SMUDDCConfiguration">
<DDCConfig:Network>
<DDCConfig:LocalIP>192.168.103.223</DDCConfig:LocalIP>
<DDCConfig:GlobalIP>168.188.127.123</DDCConfig:GlobalIP>
<DDCConfig:RootBridge>Yes</DDCConfig:RootBridge>
</DDCConfig:Network>
</DDCConfig:getValue>';
$ch = curl_init();
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_USERPWD, "lmk:alrud89");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_URL, "$url");
curl_setopt($ch, CURLOPT_PORT, 8080);
curl_setopt($ch, CURLOPT_PUT, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,array($xml));
$http_result = curl_exec($ch);
if($http_result){
echo $http_result;
}else{
echo curl_error($ch);
}
curl_close($ch);
?>
error message is
* About to connect() to 192.168.1.103 port 8080 (#0)
* Trying 192.168.1.103...
* connected
* Server auth using Basic with user 'lmk'
> PUT /ew.xml HTTP/1.1
> Authorization: Basic bG1rOmFscnVkODk=
> Host: 192.168.1.103:8080 Accept: /
> Transfer-Encoding: chunked
> Expect: 100-continue
< HTTP/1.1 401 Unauthorized
< Content-Length: 0
<
> WWW-Authenticate: Digest qop="auth", realm="mydomain.com",
nonce="1366254379"
* HTTP error before end of send, stop sending
* Closing connection #0
what is the problem & solution?
Your credentials are incorrect! (that's what error 401 means)
There's a wonderful plugin for Firefox called poster - install it and use it to double-check that the request is formatted correctly.
By the way, you might want to add the following header:
curl_setopt ($ch, CURLOPT_HTTPHEADER, Array("Content-Type: application/xml"))
Another thing: PUT is not always supported, try POST - sometimes it'll work.