I'm reading the past of the question (Request forbidden while accessing github api on node.js program), but still does not solve the problem.
PHP code
$ch = curl_init();
curl_setopt_array($ch, [
CURLOPT_URL => "https://api.github.com/markdown/raw",
CURLOPT_HEADER => [
"Accept: application/vnd.github.v3+json",
"Content-Type: text/plain",
"User-Agent: mfmfneko"
],
CURLOPT_RETURNTRANSFER => true,
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => "# Hello World!"
]);
$response = curl_exec($ch);
curl_close($ch);
print($response);
Response
HTTP/1.0 403 Forbidden
Cache-Control: no-cache
Connection: close
Content-Type: text/html
Request forbidden by administrative rules. Please make sure your request has a User-Agent header (http://developer.github.com/v3/#user-agent-required). Check https://developer.github.com for other possible causes.
You made a mistake with the installation of headers.
CURLOPT_HEADER TRUE to include the header in the output.
CURLOPT_HTTPHEADER An array of HTTP header fields to set, in the
format array('Content-type: text/plain', 'Content-length: 100')
They should be reported using the CURLOPT_HTTPHEADER
Try this code:
$ch = curl_init();
curl_setopt_array($ch, [
CURLOPT_URL => "https://api.github.com/markdown/raw",
CURLOPT_HTTPHEADER => [
"Accept: application/vnd.github.v3+json",
"Content-Type: text/plain",
"User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 YaBrowser/16.3.0.7146 Yowser/2.5 Safari/537.36"
],
CURLOPT_RETURNTRANSFER => true,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => "# Hello World!"
]);
$response = curl_exec($ch);
curl_close($ch);
print($response);
Response:
<h1>
<a id="user-content-hello-world" class="anchor" href="#hello-world" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Hello World!</h1>
More details can be read here
Related
This question already has an answer here:
curl 1020 error when trying to scrape page using bash script
(1 answer)
Closed yesterday.
`I have sent a GET (Also POST) method to get data from api but the response is error 1020 and at the same time post man tried and its giving response then what is wrong here.
I am not able to share the api url due to security reasons.
<?php
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => 'url_with_parameter',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 0,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'GET',
CURLOPT_HTTPHEADER => array(
'Cookie: PHPSESSID=f1r22a2srpgamnidkflgqsuem2; csrf_token=c31d6332c0066fa652b79948c43e5dd7_d5ec9637ea9c6914f3cebeab7fa4ad4d',
'Host' => "localhost",
'User-Agent' => 'PostmanRuntime/7.31.0',
'Accept' => '*/*',
'Accept-Encoding' => 'gzip, deflate, br',
'Connection' => 'keep-alive'
),
));
$response = curl_exec($curl);
curl_close($curl);
echo $response;
When 1020 error is returned by curl, it means typically that the request was blocked by security settings, which can include challenges like CAPTCHAs or browser checks.
Try to add UA header into your curl request, like
$headers = [
'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36',
];
curl_setopt($curl_handle, CURLOPT_HTTPHEADER, $headers);
I was trying to get html response using PHP but always get something like this
screenshot [picture hidden from post due to binary characters]
<?php
$header_request = array (
"accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-encoding: gzip",
"accept-language: id-ID,id;q=0.9,en-US;q=0.8,en;q=0.7",
"cookie: csrf_cookie_name=9316014c9d7860019da66a78edfaf926; _data_pop=115-1_274-1; ci_session=607f0be4e56b8b08ee2398b892f115c9e660192e; _data_cpc=1-2_15-2_190-4",
"referer: https://ptc4btc.com/",
"user-agent: Mozilla/5.0 (Linux; Android 7.0; Moto C Plus) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.56 Mobile Safari/537.36",
"upgrade-insecure-requests: 1"
);
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL => "https://ptc4btc.com/dashboard",
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_HTTPHEADER => $header_request,
CURLOPT_SSL_VERIFYPEER => 0,
CURLOPT_SSL_VERIFYHOST => 2,
));
$exec = curl_exec($ch);
echo($exec);
curl_close($exec);
You said you were willing to accept gzip data with accept-encoding: gzip in the headers. So decode it:
echo gzdecode($exec);
I wrote this code for getting https pages content but i couldnt succesfull.
<?php
function bot($url)
{
$header ="Host: tr-tr.facebook.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:48.0) Gecko/20100101 Firefox/48.0\r\n
Accept: */*
Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate, br";
$options = array(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_PORT => 443,
CURLOPT_SSL_VERIFYPEER => true,
CURLOPT_SSL_VERIFYHOST => 2,
CURLOPT_CAINFO => "C:\\xampp\\htdocs\\curl-ca-bundle.crt",
CURLOPT_HTTPHEADER => explode("\r\n",$header)
);
$ch = curl_init();
curl_setopt_array($ch, $options);
$response = curl_exec($ch);
curl_close($ch);
return $response;
}
echo bot("https://tr-tr.facebook.com");
?>
When i run that codes it is returning that. "��0#a�jȌ#�#.3�j�##u�.����/#cw#,�q=ߓ���K"<�˞#%#����t[�d�:|��s##$!� ��(��M��ߛ#w'#u" Where is my mistake? Why is returning these characters?
I dont want to use CURLOPT_SSL_VERIFYPEER = false. I want to https handshake with curl..
Maybe you should remove CURLOPT_RETURNTRANSFER or handle it properly.
From the manual
http://php.net/manual/en/function.curl-setopt.php
Do you really want this?!
> CURLOPT_RETURNTRANSFER
> TRUE to return the transfer as a string of the
> return value of curl_exec() instead of outputting it out directly.
I am making an HTTP API for use in Roblox. I am using PHP and cURL to do so.
However, there is one method on which I am stuck. That is sending messages through roblox in PHP. I have captured the message sending request using Fiddler. I have put all the headers into my cURL request.
I also am fetching the .ROBLOSECURITY cookie, the X-CSRF Token, and the RBXSessionTracker and __RequestVerification cookies.
Here is the messaging code for those of you want to help:
$RecipientId=$_POST['RecipientId'];
$Subject=$_POST['Subject'];
$Message=$_POST['Message'];
$AuthCookie=decodeJSON($_POST['Login']);
$AuthCookie=$AuthCookie[0];
$TokenCurl=curl_init('roblox.com/build/upload?groupId=1');
curl_setopt_array($TokenCurl,array(
CURLOPT_HTTPGET => 1,
CURLOPT_FRESH_CONNECT => 1,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_HEADER => 1,
CURLOPT_HTTPHEADER => array('Cookie: .ROBLOSECURITY='.$AuthCookie),
));
$TokenResponse=curl_exec($TokenCurl);
$TokenHeaderSize=curl_getinfo($TokenCurl,CURLINFO_HEADER_SIZE);
$TokenCurlError=curl_error($TokenCurl);
$Cookies=GetCookies(substr($TokenResponse,0,$TokenHeaderSize));
$SessionTracker=$Cookies['RBXSessionTracker'];
$RequestVerification=$Cookies['__RequestVerificationToken'];
$AuthCookie='.ROBLOSECURITY='.$AuthCookie.'; RBXSessionTracker='.$SessionTracker.'; __RequestVerificationToken='.$RequestVerification.';';
curl_close($TokenCurl);
if($TokenCurlError!==''){
die($TokenCurlError);
};
$Cache=time();
preg_match("\`<script\stype=\"text/javascript\">\s*?Roblox\.XsrfToken\.setToken\('(.*?)'\);\s*?</script>\`",$TokenResponse,$Matches);
$Token=$Matches[1];
if(empty($Token)){
die("Could not get X-CSRF Token");
};
$PostData=json_encode(array(subject=>$Subject,body=>$Message,recipientid=>$RecipientId,cacheBuster=>$Cache));
$Curl=curl_init('roblox.com/messages/send');
curl_setopt_array($Curl,array(
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0',
CURLOPT_POST => 1,
CURLOPT_FRESH_CONNECT => 1,
CURLOPT_HEADER => 1,
CURLOPT_POSTFIELDS => $PostData,
CURLOPT_HTTPHEADER => array('Content-Length: '.strlen($PostData),'X-CSRF-TOKEN: '.$Token,'Referer: www.roblox.com/messages/compose?recipientId='.$RecipientId,'X-Requested-With: XMLHttpRequest','Cookie: '.$AuthCookie,'Accept: application/json, text/javascript, */*; q=0.01','Accept-Language: en-US,en;q=0.5','Accept-Encoding: gzip, deflate','Connection: keep-alive','Pragma: no-cache','Cache-Control: no-cache'),
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_VERBOSE => 1
));
$ResponseBody=curl_exec($Curl);
echo $ResponseBody;
$HTTPResponseCode=curl_getinfo($Curl,CURLINFO_HTTP_CODE);
$HTTPHeaderSize=curl_getinfo($Curl,CURLINFO_HEADER_SIZE);
$ResponseHeader=substr($ResponseBody,0,$HTTPHeaderSize);
$CurlError=curl_error($Curl);
curl_close($Curl);
if($CurlError!==''){
die($CurlError);
};
$MessageResult=decodeJSON(substr($ResponseBody,$HTTPHeaderSize));
echo json_encode(array(Message=>$MessageResult['message']));
I removed the 'http://' from the links because of reputation.
Everything works, it just will not send the message. Does anyone know what exactly you need to send a message?
When I try to login to page with curl I get 500 Internal Server Error. Page is on https.
Here is Login function:
function login ($login, $pass) {
$loginURL = "https://bitomat.pl/Account/LogOn";
$loginPage = $this->curl->getPage($loginURL);
$numStart = strpos(htmlentities($loginPage['content']), "type="hidden" value="");
$numEnd = strpos(htmlentities($loginPage['content']), "<fieldset>");
$verNum = substr(htmlentities($loginPage['content']), $numStart+36, 64);
echo $numStart." - ".$numEnd." - ".$verNum."<br>";
$page = $this->curl->post($loginURL, "__RequestVerificationToken=".urlencode($verNum)."&UserName=$login&Password=".urlencode($pass)."&RememberMe=false");
echo nl2br(htmlentities($page['content']));
}
In response I get:
HTTP/1.1 500 Internal Server Error
Cache-Control: private
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.0
X-AspNetMvc-Version: 2.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Sat, 25 Jun 2011 16:32:18 GMT
Content-Length: 3974
CUrl post function:
function post($url, $params) {
$rnd = rand(0, 10000000000);
$options = array(
CURLOPT_COOKIESESSION => true,
CURLOPT_COOKIEFILE => "cookie/cookieBitomat",// . $rnd,
CURLOPT_COOKIEJAR => "cookie/cookieBitomat",// . $rnd,
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => true, // don't return headers
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $params,
CURLOPT_USERAGENT => "Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1", // who am i
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_CONNECTTIMEOUT => 60, // timeout on connect
CURLOPT_TIMEOUT => 60, // timeout on response
CURLOPT_MAXREDIRS => 20, // stop after 10 redirects
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_VERBOSE => TRUE,
CURLOPT_HTTPHEADER => array("Content-Type: application/x-www-form-urlencoded",
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Host: bitomat.pl",
"Accept-Language: pl,en-us;q=0.7,en;q=0.3",
"Accept-Encoding: gzip, deflate",
"Accept-Charset: ISO-8859-2,utf-8;q=0.7,*;q=0.7",
"Keep-Alive: 115",
"Connection: keep-alive",
"Referer: https://bitomat.pl/Account/LogOn"
)
);
$ch = curl_init($url);
curl_setopt_array($ch, $options);
$content = curl_exec($ch);
$err = curl_errno($ch);
$errmsg = curl_error($ch);
$header = curl_getinfo($ch);
curl_close($ch);
$header['errno'] = $err;
$header['errmsg'] = $errmsg;
$header['content'] = $content;
return $header;
}
Usually you get 500 in cases like this because your HTTP request differs somewhat from one that a browser would do in this situation and the server-side script assumes something about it.
Make sure you inspect a "live" request and mimic that as closely as possible.