I wrote a very simple script to just see what a server was sending like this:
<?php
$html1 = <<<EOT
<!doctype html>
<html lang=en>
<head>
<meta charset = 'utf-8'>
<title>TradingView Test</title>
</head>
<body>
<pre>
EOT;
$html2 = <<<EOT
</pre>
</body>
</html>
EOT;
date_default_timezone_set('America/Phoenix');
$file = date('Y/m/d h:i:sa') . "\n";
$url = (isset($_SERVER['HTTPS']) && $_SERVER['HTTPS'] === 'on' ? "https" : "http")
. "://$_SERVER[HTTP_HOST]$_SERVER[REQUEST_URI]";
$file .= "\nURL: $url\n\n";
$html = $file;
if (!empty($_REQUEST)) {
$r = "\$_REQUEST:\n" . var_export($_REQUEST, true) . "\n\n";
$file .= $r;
$html .= htmlspecialchars($r);
}
$headers = apache_request_headers();
if (!empty($headers)) {
$h = "HEADERS:\n" . var_export($headers, true) . "\n\n";
$file .= $h;
$html .= htmlspecialchars($h);
}
// if (!empty($_SERVER)) {
// $s = "\$_SERVER:\n" . var_export($_SERVER, true) . "\n\n";
// $file .= $s;
// $html .= htmlspecialchars($s);
// }
$file .= str_repeat('-', 40) . "\n";
file_put_contents('./get-post.log', $file, FILE_APPEND);
# echo $html1 . $html . $html2;
?>
In the example for what the server sends they had this example:
#!/bin/sh
curl -H 'Content-Type: text/plain; charset=utf-8' -d '1111111111,data,more-data' -X POST https://www.testtest.com/
I thought with $_REQUEST I would see all that was sent but I don't see anything and what the script outputs is:
2022/03/30 06:15:22pm
URL: https://www.testtest.com/
HEADERS:
array (
'X-Https' => '1',
'Connection' => 'close',
'Accept-Encoding' => 'gzip',
'Content-Type' => 'text/plain; charset=utf-8',
'Content-Length' => '45',
'User-Agent' => 'Go-http-client/1.1',
'Host' => 'www.testtest.com',
)
I suspect the issue is that they are not sending the data as a proper POST by naming a field and I can't change how the data is being sent because it's not my server.
Does anyone know what I can look at in PHP to see the data being sent? Or perhaps this is an Apache problem as PHP gets its data through Apache and maybe in that format it's just not getting passed through?
You will note that I have displaying of $_SERVER commented out because even when it wasn't it didn't give me anything that helped.
Found it. PHP won't fill $_POST without named arguments so to get the string the way it's being sent this needs to be used:
<?php
$str = file_get_contents('php://input');
Related
The first parameter of the curl_file_create function is the filename. Is there a way to directly use the content of the file?
I do not want to write content in a temporary file and use that filename in the function.
More expand:
I get an image from URL with $image = file_get_contents('http://example.com/image.jpg'); function, and so I have a $image variable with binary image content for now, After get it I need to post this image as part of form to another URL instantly without use temporary file on disk.
to anyone thinking about doing this: it's almost certainly not worth the hassle. it's possible, sure, but you'd almost certainly want to use the tmpfile() trick instead, for example if you have a variable called $file_content that you want to upload as a file, but you don't have a file, do
<?php
$file_content = "the content of the *file* i want to upload, which does not exist on disk";
$file_name = "example.bin";
$tmph = tmpfile(); // will create a file in the appropriate temp folder when an unique filename
fwrite($tmph, $file_content);
$tmpf = stream_get_meta_data($tmph)['uri'];
$ch = curl_init('http://target_url/');
curl_setopt_array($ch, array(
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => array(
'file' => new CURLFile($tmpf, 'application/octet-stream', $file_name)
)
));
curl_exec($ch);
fclose($tmph); // thanks to tmpfile() magic, the file is automatically deleted when fclosed()'d
unset($tmph, $tmpf); // unset() is not needed, but this is the end of the tmpfile()-trick.
.. and with a bit of luck, as long as the upload is quick and we don't call fflush($tmph);, the file may never touch the actual disk anyway, it will just be created in the IO cache, scheduled to be written to disk later, then deleted (from the io cache) - also this code is protected by PHP's garbage collector, if there's an uncaught exception/error in the code which stops execution, PHP's garbage collector will delete the file for us..
However, here's how to actually upload a file in the Multipart/form-data-format without having nor creating an on-disk file, by creating the Multipart/form-data-request with userland PHP code:
usage example:
<?php
// normal php-way:
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL=>'https://example.com/',
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => array(
'sample_variable' => 'anything',
'file' => new CURLFile("path/to/file.txt")
)
));
curl_exec($ch);
curl_close($ch);
// shitty_multipart-way:
$post_data = array();
$tmp = new shitty_multipart_variable();
$tmp->post_name = 'sample_variable';
$tmp->content = 'anything';
$post_data[] = $tmp;
$tmp = new shitty_multipart_file();
$tmp->post_name = 'file';
$tmp->post_file_name = "file.ext";
$tmp->content = 'contents of file.ext';
$post_data[] = $tmp;
$content_type_header="";
$post_body=shitty_multipart_form_data_generator($post_data,$content_type_header);
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL=>'https://example.com/',
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => $post_body,
CURLOPT_HTTPHEADER=>array(
$content_type_header
)
));
curl_exec($ch);
curl_close($ch);
implementation:
class shitty_multipart_file
{
public $post_name = "";
public $file_name = "";
public $content_type = "application/octet-stream";
public $content = "";
public $additional_headers = [];
}
class shitty_multipart_variable
{
public $post_name = "";
public $content = "";
public $additional_headers = [];
}
function shitty_multipart_form_data_generator(array $postfields, string &$out_content_type_header): string
{
// Content-Type: multipart/form-data; boundary=------------------------7b5b9abe8c56fd67
// same boundary format as used by curl
$boundary = "------------------------" . strtolower(bin2hex(random_bytes(8)));
$out_content_type_header = 'Content-Type: multipart/form-data; boundary=' . $boundary;
$body = "";
foreach ($postfields as $unused => $post) {
$body .= $boundary . "\r\n";
if (is_a($post, 'shitty_multipart_variable')) {
$body .= "Content-Disposition: form-data; name=\"{$post->post_name}\"\r\n";
foreach ($post->additional_headers as $header) {
$body .= $header . "\r\n";
}
$body .= "\r\n";
$body .= $post->content . "\r\n";
} elseif (is_a($post, 'shitty_multipart_file')) {
$body .= "Content-Disposition: form-data; name=\"{$post->post_name}\"; filename=\"{$post->file_name}\"\r\n";
$body .= "Content-Type: " . $post->content_type . "\r\n";
foreach ($post->additional_headers as $header) {
$body .= $header . "\r\n";
}
$body .= "\r\n";
$body .= $post->content . "\r\n";
} else {
throw new \InvalidArgumentException("postfields key {$unused} is not an instance of shitty_multipart_variable NOR an instance of shitty_multipart_file!");
}
}
$body .= $boundary . "--\r\n";
return $body;
}
I am getting an error when using the ConnectWise API to upload a document to an existing ticket, and I believe the issue is with how I performing the curl request.
I have tried messing around with the curl options based on what I've found on Google, but nothing seems to work.
Here is the POST request
function post_ticket_attachment ( $ticket_id, $file_attachment ) {
$url = "https://api-na.myconnectwise.net/v4_6_release/apis/3.0/system/documents";
$filename = $file_attachment['name'];
$filedata = $file_attachment['tmp_name'];
$filesize = $file_attachment['size'];
$fields = array (
"recordId" => $ticket_id,
"recordType" => "Ticket",
"title" => $file_attachment['name']
);
$filenames = array($filedata);
$files = array();
foreach( $filenames as $f ){
$files[$f] = file_get_contents($f);
}
$ch = curl_init();
$boundary = uniqid();
$delimiter = '-------------' . $boundary;
$post_data = build_data_files($boundary, $fields, $files);
curl_setopt_array($ch, array(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_CUSTOMREQUEST => "POST",
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => $post_data,
CURLOPT_HTTPHEADER => array(
"Authorization: basic " . $connect_wise_auth,
"Content-Type: multipart/form-data; boundary=" . $delimiter,
"Content-Length: " . strlen($post_data)
),
));
$response = curl_exec($ch);
console_log ($response);
}
And here is how I'm building the post data:
function build_data_files($boundary, $fields, $files){
$data = '';
$eol = "\r\n";
$delimiter = '-------------' . $boundary;
foreach ($fields as $name => $content) {
$data .= "--" . $delimiter . $eol
. 'Content-Disposition: form-data; name="' . $name . "\"".$eol.$eol
. $content . $eol;
}
foreach ($files as $name => $content) {
$data .= "--" . $delimiter . $eol
. 'Content-Disposition: form-data; name="' . $name . '"; filename="' . $name . '"' . $eol
. 'Content-Transfer-Encoding: binary'.$eol
;
$data .= $eol;
$data .= $content . $eol;
}
$data .= "--" . $delimiter . "--".$eol;
return $data;
}
I'm expecting the request to post correctly, but instead I'm getting a response from ConnectWise stating, "Cannot route. Codebase/company is invalid."
I just had this problem and was because I forget to add the username and password to the basic authorization. Looks like that you are adding it but I can't see the what is containing so, just to clarify, the username is the company+publickey and the password is the private key, while encoded in base64.
From the official documentation (you may be to login in to be able to see it)
It is recommended to create API Members versus using API Keys tied to
a specific member. Authorization: Basic
base64(companyid+publickey:privatekey) (Authorization: Basic
Y29tcGFueWlkK3B1YmxpY2tleTpwcml2YXRla2V5)
Once you add that information properly the API will be able to check that your user is correct and you belong to that company.
Please note that this is a very specific problem with a very specific API, so probably if you find more problems in the future you will have more luck asking directly on their forums.
Sorry not a full answer (I cannot comment yet)
I was getting this same error when trying to lookup a company by its identifier. It turned out that I was not using the correct URI.
try using fiddler or something to examine exactly what you are posting to see if there is something not 100% right.
My issue was I had a forward slash here: company/companies/?conditions
It should have been company/companies?conditions
I didn't notice at first as I was using variables to build the URI so it was not very clear.
I also find that using this https://github.com/christaylorcodes/ConnectWiseManageAPI is helpful as I can see how the request should be formatted (by watching with fiddler when I post). Then you can make sure your post is behaving the same.
I have just a PHP script for HTML parsing and it works on simple web sites, but now I need to parse the cinema program from this website. I am using the file_get_contents function, which returns just 4 new line delimiters \n and I just can't figure out why.
The website itself will be more difficult to parse with DOMDocument a XPath because the program itself is just pop-up window and it doesn't seem to change the URL address but I will try to handle this problem after retrieving the HTML code of the site.
Here is the shortened version of my script:
<?php
$url = "http://www.cinemacity.cz/";
$content = file_get_contents($url);
$dom = new DomDocument;
$dom->loadHTML($content);
if ($dom == FALSE) {
echo "FAAAAIL\n";
}
$xpath = new DOMXPath($dom);
$tags = $xpath->query("/html");
foreach ($tags as $tag) {
var_dump(trim($tag->nodeValue));
}
?>
EDIT:
So, following the advice by WBAR (thank you), I was looking for a way how to change the header in file_get_contents() function a this is the answer I've found elsewhere. Now I am able to obtain the HTML of the site, hopefully I will manage parsing of this mess :D
<?php
libxml_use_internal_errors(true);
// Create a stream
$opts = array(
'http'=>array(
'user_agent' => 'PHP libxml agent', //Wget 1.13.4
'method'=>"GET",
'header'=>"Accept-language: en\r\n" .
"Cookie: foo=bar\r\n"
)
);
$context = stream_context_create($opts);
// Open the file using the HTTP headers set above
$content = file_get_contents('http://www.cinemacity.cz/', false, $context);
$dom = new DomDocument;
$dom->loadHTML($content);
if ($dom == FALSE) {
echo "FAAAAIL\n";
}
$xpath = new DOMXPath($dom);
$tags = $xpath->query("/html");
foreach ($tags as $tag) {
var_dump(trim($tag->nodeValue));
}
?>
The problem is not in PHP but in target host. It detects client's User-Agent header. Look at this:
wget http://www.cinemacity.cz/
2012-10-07 13:54:39 (1,44 MB/s) - saved `index.html.1' [234908]
but when remove User-Agent headers:
wget --user-agent="" http://www.cinemacity.cz/
2012-10-07 13:55:41 (262 KB/s) - saved `index.html.2' [4/4]
Only 4 bytes were returned by the server
Try to get the contents this way:
function get2url($url, $timeout = 30, $port = 80, $buffer = 128) {
$arr = parse_url($url);
if(count($arr) < 3) return "URL ERROR";
$ssl = "";
if($arr['scheme'] == "https") $ssl = "ssl://";
$header = "GET " . $arr['path'] . "?" . $arr['query'] . " HTTP/1.0\r\n";
$header .= "Host: " . $arr['host'] . "\r\n";
$header .= "\r\n";
$f = #fsockopen($ssl . $arr['host'], $port, $errno, $errstr, $timeout);
if(!$f)
return $errstr . " (" . $errno . ")";
else{
#fputs($f, $header . $arr['query']);
$echo = "";
while(!feof($f)) { $echo .= #fgets($f, $buffer); }
#fclose($f);
return $echo;
}
}
You will have to remove the headers though.
I'm trying to figure out how to get this working with PHP. I have a working IPN for paypal that is less than 20 lines of code to get the data I need. I have tried reading the Google docs but they are either way too specific or way too general. There is some sample code, which is about 1300 lines in 5 files and I can't make sense of it. I just need a handful of vars back from a completed transaction, nothing more. Is it possible to do this with a few lines of code (and I mean without 1300 lines worth of "include" files) or is Google Checkout's process really that bulky?
Here's a bit of code I started. Not yet finished.
It works perfectly.
All you need to do is take the data Google sends back and this code writes to a file and use it to insert into your sales table send notification of payment received to customer and so on.
The trick is that when Google sends you a post you must call back with and Authorization header or it will not take it in consideration.
function post2google($url, $timeout = 30, $port = 80, $buffer = 128) {
$mid = "123456789";
$mky = "qwertyuiop";
$aut = base64_encode($mid . ":" . $mky);
$arr = parse_url($url);
$ssl = "";
if($arr['scheme'] == "https") $ssl = "ssl://";
$post = "POST " . $arr['path'] . " HTTP/1.1\r\n";
$post .= "Host: " . $arr['host'] . "\r\n";
$post .= "Authorization: Basic " . $aut . "\r\n";
$post .= "Content-Type: application/xml; charset=UTF-8\r\n";
$post .= "Accept: application/xml; charset=UTF-8\r\n";
$post .= "Content-Length: " . strlen($arr['query']) . "\r\n";
$post .= "Connection: Close\r\n";
$post .= "\r\n";
$post .= $arr['query'];
$f = fsockopen($ssl . $arr['host'], $port, $errno, $errstr, $timeout);
if(!$f)
return $errstr . " (" . $errno . ")";
else{
fputs($f, $post);
while(!feof($f)) { $echo .= #fgets($f, $buffer); }
fclose($f);
return $echo;
}
}
$re = post2google("https://checkout.google.com/api/checkout/v2/reportsForm/Merchant/123456789?_type=notification-history-request&serial-number=" . $_REQUEST['serial-number'], 3, 443);
$re = str_replace("&", "\n", $re) . "\n\n--\n\n";
file_put_contents("gpn.txt", $re, FILE_APPEND);
I've gotten it to work, and here's the skeleton of my code that can be used to handle HTTP notifications/responses. This was obviously derived from tntu's example above. (Thanks!)
//incoming data is in the var $_POST['serial-number']
//"send" the response to acknowledge the serial number that google talks about all over but never explains how
echo "_type=notification-acknowledgment&serial-number=".$_POST['serial-number'];
//now we need to call google's server and ask for this transaction's data:
//you'll need to change your merchant id in the $url and $mid vars, and your merchant key in the $mky var
$url = "https://sandbox.google.com/checkout/api/checkout/v2/reportsForm/Merchant/1234567890?_type=notification-history-request&serial-number=" . $_REQUEST['serial-number'];
$mid = "1234567890";
$mky = "ABCDEFGHIJK";
$aut = base64_encode($mid . ":" . $mky);
$arr = parse_url($url);
$ssl = "";
if($arr['scheme'] == "https") $ssl = "ssl://";
$post = "POST " . $arr['path'] . " HTTP/1.1\r\n";
$post .= "Host: " . $arr['host'] . "\r\n";
$post .= "Authorization: Basic " . $aut . "\r\n";
$post .= "Content-Type: application/xml; charset=UTF-8\r\n";
$post .= "Accept: application/xml; charset=UTF-8\r\n";
$post .= "Content-Length: " . strlen($arr['query']) . "\r\n";
$post .= "Connection: Close\r\n";
$post .= "\r\n";
$post .= $arr['query'];
//now we actually make the request by opening a socket and calling Google's server
$f = fsockopen($ssl . $arr['host'], 443, $errno, $errstr, 30);
if(!$f){
//something failed in the opening of the socket, we didn't contact google at all, you can do whatever you want here such as emailing yourself about it and what you were trying to send, etc
#mail("troubleshooting#yourdomain.com","Google IPN - HTTP ERROR ",$errstr . " (" . $errno . ")\n\n\n".$arr['query']);
}else{
//the socket was opened, send the request for the order data:
fputs($f, $post); // you're sending
while(!feof($f)) { $response .= #fgets($f, 128); } //google replies and you store it in $response
fclose($f); //close the socket, we're done talking to google's server
$spl=strpos($response,"_type="); //parse the type because parse_str won't catch it
if ($spl!==false){
$spl2=strpos($response,"&",$spl);
$ordertype=substr($response,($spl+6),($spl2-$spl)-6);
}//$ordertype will tell you which type of notification is being sent, new-order-notification, risk-information-notification, etc
$subresponse=substr($response,$spl2+1); //put the rest of it into an array for easy access
parse_str($subresponse,$order);//you can now access google's response in $order[] vars
//IMPORTANT: dots in Google's field names are replaced by underscore, for example:
// $order['google-order-number'] and $order['buyer-billing-address_address1'] NOT $order['buyer-billing-address.address1']
//order field names are shown here:
//https://developers.google.com/checkout/developer/Google_Checkout_HTML_API_Notification_API#order_summary
//this is the point where you will want to use the data contained in $order[] to create a new record in your database or whatever.
//NOTE: be sure to store and check for duplicates using the google-order-number because you will get multiple notifications from google regarding the same order
if (strtoupper($order['order-summary_financial-order-state']) == "CHARGEABLE"){
//CHARGEABLE is what indicates it is safe to create a login for the user (if you are delivering digital goods)
// insert into db, and/or email user with key or download url
}
}
i have been trying for the last several days to fetch a request from a website but no success.
I keep getting error 301.
Is anyone able to help me grab the content of this page: https://pre.corrupt-net.org/search.php?search=Lasse_Stefanz-Bara_Du-SE-CD-FLAC-1995-LoKET
I am looking forward to your reply.
EDIT:
This is the php function I've used:
function http_request(
$verb = 'GET', /* HTTP Request Method (GET and POST supported) */
$ip, /* Target IP/Hostname */
$port = 80, /* Target TCP port */
$uri = '/', /* Target URI */
$getdata = array(), /* HTTP GET Data ie. array('var1' => 'val1', 'var2' => 'val2') */
$postdata = array(), /* HTTP POST Data ie. array('var1' => 'val1', 'var2' => 'val2') */
$cookie = array(), /* HTTP Cookie Data ie. array('var1' => 'val1', 'var2' => 'val2') */
$custom_headers = array(), /* Custom HTTP headers ie. array('Referer: http://localhost/ */
$timeout = 1000, /* Socket timeout in milliseconds */
$req_hdr = false, /* Include HTTP request headers */
$res_hdr = false /* Include HTTP response headers */
)
{
$ret = '';
$verb = strtoupper($verb);
$cookie_str = '';
$getdata_str = count($getdata) ? '?' : '';
$postdata_str = '';
foreach ($getdata as $k => $v)
$getdata_str .= urlencode($k) .'='. urlencode($v);
foreach ($postdata as $k => $v)
$postdata_str .= urlencode($k) .'='. urlencode($v) .'&';
foreach ($cookie as $k => $v)
$cookie_str .= urlencode($k) .'='. urlencode($v) .'; ';
$crlf = "\r\n";
$req = $verb .' '. $uri . $getdata_str .' HTTP/1.1' . $crlf;
$req .= 'Host: '. $ip . $crlf;
$req .= 'User-Agent: Mozilla/5.0 Firefox/3.6.12' . $crlf;
$req .= 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' . $crlf;
$req .= 'Accept-Language: en-us,en;q=0.5' . $crlf;
$req .= 'Accept-Encoding: deflate' . $crlf;
$req .= 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7' . $crlf;
foreach ($custom_headers as $k => $v)
$req .= $k .': '. $v . $crlf;
if (!empty($cookie_str))
$req .= 'Cookie: '. substr($cookie_str, 0, -2) . $crlf;
if ($verb == 'POST' && !empty($postdata_str)){
$postdata_str = substr($postdata_str, 0, -1);
$req .= 'Content-Type: application/x-www-form-urlencoded' . $crlf;
$req .= 'Content-Length: '. strlen($postdata_str) . $crlf . $crlf;
$req .= $postdata_str;
}
else $req .= $crlf;
if ($req_hdr)
$ret .= $req;
if (($fp = #fsockopen($ip, $port, $errno, $errstr)) == false)
return "Error $errno: $errstr\n";
stream_set_timeout($fp, 0, $timeout * 1000);
fputs($fp, $req);
while ($line = fgets($fp)) $ret .= $line;
fclose($fp);
if (!$res_hdr)
$ret = substr($ret, strpos($ret, "\r\n\r\n") + 4);
return $ret;
}
Firstly, 301 is not an "error" as such, it indicates that you are being redirected. You need to parse the response headers, take the value of the Location: header (which the HTTP protocol specification requires be present in a redirect response) and request that URI as well.
Secondly, the function above does not appear to provide any support for accessing HTTPS URLs. You need the OpenSSL extension installed for your PHP instance to do this, and you also need to actually call it some how. You could use the above function to do so by passing ssl:// or tls:// in front of the address in the $ip parameter, but you cannot simply pass the IP.
Thirdly, the usual way to do things like this is with the cURL extension. You would do something like this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://pre.corrupt-net.org/search.php?search=Lasse_Stefanz-Bara_Du-SE-CD-FLAC-1995-LoKET'); // Set the URL
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Get the result from the execution
if (($result = curl_exec($ch)) === FALSE) { // Execute the request
echo "cURL failed! Error: ".curl_error($ch);
} else {
echo "Success! Result: $result";
}
curl_close($ch);
Alternatively, if cURL is not available or you don't want to use it for some reason, you could have go with my HTTPRequest class, which is PHP4 compliant and requires no extensions (apart from OpenSSL for HTTPS requests). Documented(ish) in comments at the top of the script. You would do something like this:
$request = new httprequest(); // Create an object
// Set the request URL
if (!$request->setRequestURL('https://pre.corrupt-net.org/search.php?search=Lasse_Stefanz-Bara_Du-SE-CD-FLAC-1995-LoKET')) echo "Failed! Error: ".$request->getLastErrorStr()."<br>\r\n";
// Send the request
if (!$request->sendRequest()) echo "Failed! Error: ".$request->getLastErrorStr()."<br>\r\n";
echo "Success! Result: ".$request->getResponseBodyData(TRUE);
On a side note, a lot of the Scene PreDB managers/providers are not too keen on automated scraping, and you may get yourself banned...