at the moment I am coding an article/customer/billing software with PHP and this is my last step before letting my partner test it.
I'm really stuck at the moment. We want to upload our articles automatically to eBay. The generation of the CSV file for the File Exchange Program works fine, manual uploading also works and the articles will be listed.
Now we want the software to do the upload by itself. We reference to the eBay File Exchange Guide here:
and here:
.
This is my present code:
$SOCKETPROC = fsockopen('bulksell.ebay.de', 80, $errno, $errstr, 4);
fputs($SOCKETPROC, "POST https://bulksell.ebay.de/ws/eBayISAPI.dll?FileExchangeUploadForm HTTP/1.0\r\n");
fputs($SOCKETPROC, "Connection: Keep Alive\r\n");
fputs($SOCKETPROC, "User-Agent: App v1.0\r\n");
fputs($SOCKETPROC, "Host: https://bulksell.ebay.de/ws/eBayISAPI.dll?FileExchangeUpload\r\n");
fputs($SOCKETPROC, "Content-Type: multipart/form-data; boundary=THIS_STRING_SEPARATES\r\n");
fputs($SOCKETPROC, "Content-Length: " . filesize('export/ebay/items-' . date('Y-m-d') . '.csv') + filesize('export/ebay/token') . "\r\n");
fputs($SOCKETPROC, "--THIS_STRING_SEPARATES\r\n");
fputs($SOCKETPROC, "Content-Disposition: form-data; name=\"token\"\r\n");
fputs($SOCKETPROC, file_get_contents('export/ebay/token'));
fputs($SOCKETPROC, "\r\n--THIS_STRING_SEPARATES\r\n");
fputs($SOCKETPROC, "Content-Disposition: form-data; name=\"file\"; filename=\"items-" . date('Y-m-d') . ".csv\"\r\n");
fputs($SOCKETPROC, "Content-Type: text/csv\r\n\r\n");
fputs($SOCKETPROC, file_get_contents('export/ebay/items-' . date('Y-m-d') . '.csv'));
fputs($SOCKETPROC, "\r\n--THIS_STRING_SEPARATES\r\n");
fputs($SOCKETPROC, "Connection: Close\r\n\r\n"); // Not sure if this line is relevant
$RESULT = fgets($SOCKETPROC);
fclose($SOCKETPROC);
But the file just won't appear in the eBay list of uploaded files. When I change the https:// to http:// and checkout the result with fgets($SOCKETPROC); I'm getting a HTTP/1.1 200 OK, otherwise I don't get any reaction.
Note: .de is being used on purpose. So no typo or mistake there.
Try connect and upload by cUrl:
$token = "your_token";
$ebay_url = "https://bulksell.ebay.de/ws/eBayISAPI.dll?FileExchangeUpload";
$sendheaders = array(
"User-Agent: MyClient v1.6",
);
$fields = array(
"token" => $token,
"file" => "#file.csv"
);
$ch = curl_init($ebay_url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0); // set to 0 to eliminate header info from response
curl_setopt($ch, CURLOPT_NOBODY, 0); // set to 1 to eliminate body info from response
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0); // use HTTP/1.0 instead of 1.1
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // Returns response data instead of TRUE(1)
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // uncomment this line if you get no gateway response. ###
curl_setopt($ch, CURLOPT_HTTPHEADER, $sendheaders);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields); // use HTTP POST to send form data
$resp = curl_exec($ch); //execute post and get results
curl_close ($ch);
Related
I had been using PHP curl to get the contents of a file, hosted on a different server. The file can easily be opened on a browser like Chrome etc., but with cURL, it always returns a blank page.
The file is hosted on an Nginx server and even miniproxy.php fails to get contents. Instead, it returns 406 not acceptable. I tried using the HTTP spy extension to monitor the request sent and found the following header:
Upgrade-Insecure-Requests:1
I tried sending the same header along With other headers, but in vain. Still, I couldn't rectify my mistake. On the Internet, I found the zalmos proxy which was able to get the contents of the file. The curl code I wrote is attached below.
$url = "http://smumcdnems01.cdnsrv.jio.com/jiotv.live.cdn.jio.com/" . $ch . "/" . $ch . "_" . $q . ".m3u8" . $tok;
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$headers = array(
"User-Agent: agent",
"lbcookie: 300",
"devicetype: 1",
"os: android",
"appkey: 1111111",
"deviceId: device id",
"uniqueId: unique id",
"ssotoken: any token",
"Upgrade-Insecure-Requests: 1",
"Host: example.com",
"Connection: keep-alive",
"X-Chrome-offline: persist=0 reason=reload",
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"Accept-Encoding: gzip, deflate, sdch",
"Accept-Language: en-GB,en-US;q=0.9,en;q=0.8",
"subscriberId: any id",
);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
//for debug only!
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
$resp = curl_exec($curl);
curl_close($curl);
echo $url;
echo $resp;
I believe that any part is missing in my code which is posing a problem. How can this be rectified?
Check your URL. Curl must give you the response. If it's hit the target URL, either the target URL is not responding to anything when sending the request.
You may be trying to access a websocket. Try to simulate with Postman to get more information.
What I'm trying to achieve:
Get request to an API Endpoint, retrieving an XML and subsequently parse the results.
I am sending a file_get_contents request to achieve this.
Issues:
`file_get_Contents` fails, error:
Warning: file_get_contents(https://api.twitter.com/1.1/statuses/mentions_timeline.json):
failed to open stream:
A connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond.
Update 17/08
To consolidate my current understanding:
1. PHP FAILS:
1.a it fails via php (timeout)
1.b it fails via command line (curl -G http://api.eve-central.com/api/quicklook?typeid=34)
1.c file_get_contents
1.d file_get_contents w/ create_stream_context
2. What WORKS:
2.a Pasting the url in a chrome tab
2.b via postman
What has been attempted:
- Check Headers in Postman ,and try to replicate them via php
Postman Headers sent back by eve-central:
Access-Control-Allow-Origin → *
Connection → Keep-Alive
Content-Encoding → gzip
Content-Type → text/xml; charset=UTF-8
Date → Wed, 17 Aug 2016 10:40:24 GMT
Proxy-Connection → Keep-Alive
Server → nginx
Transfer-Encoding → chunked
Vary → Accept-Encoding
Via → HTTP/1.1 proxy10014
Corresponding Code:
$headers = array(
'method' => 'GET',
'header' => 'Connection: Keep-Alive',
'header' => 'Content-Encoding: gzip',
'header' => 'Content-Type: text/xml',
'header' => 'Proxy-Connection: Keep-Alive',
'header' => 'Server: nginx',
'header' => 'Transfer-Encoding: chunked',
'header' => 'Vary: Accept-Encoding',
'header' => 'Via: HTTP/1.1 proxy10014');
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt($curl, CURLOPT_PORT , 8080); // Attempt at changing port in the event it was blocked.
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_POST, false );
curl_setopt($curl, CURLOPT_URL, $url );
$resp = curl_exec($curl);
if(curl_error($curl))
{
echo 'error:' . curl_error($curl);
}
Use Wireshark to capture the GET request to see if changing the port helped
Run cUrl via command line
I'm out of ideas and option.
So the questions are:
If it works in a browser, and in Postman, why does it not work via PHP ?
How can I modify my code so that it mimics what Postman does? ?
Previous Attempts
What I have tried:
Various cURL options from other threads, such as
function curl_get_contents($url) {
$ch = curl_init();
if (!$ch)
{
die("Couldn't initialize a cURL handle");
} else
echo "Curl Handle initialized ";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
$data = curl_exec($ch);
// Check if any error occurred
if (!curl_errno($ch))
{
$info = curl_getinfo($ch);
echo 'Took ', $info['total_time'], ' seconds to send a request to ', $info['url'], "";
displayData($info);
} else
echo "Failed Curl, reason: ".curl_error($ch)." ";
curl_close($ch);
return $data;
}
result: nothing, no data returned.
- Checked php.ini options:
- allow_fopen is On
- allow_url_include = on
- relevant ssl extensions are enabled
- Raised the timeout window
- both via php.ini
- also via explicit declaration within the php file.
- Tried with a different url
- same error, so it doesn't really depends on my particular endpoint
- for example, both twitter/wikipedia/google return the specific error
- tried with:
- file_get_contents on a local xml file (https://msdn.microsoft.com/en-us/library/ms762271(v=vs.85).aspx) --> works
- file_get_contents on a remote xml file (http://www.xmlfiles.com/examples/note.xml) --> fails same error
- Overall, the following is true, so far:
- Curl fails, timeout
- file_get_Contents fails, timeout
- Open XML file url in a browser works
- Make a GET request via Postman, works
Obviously, in all cases where the file_get_contents fails via php, I can easily access the file via any browser.
Tried to work around the issue.
Attempt 1:
Use nitrous.io, create a LAMP stack, perform the deed via the platform
results: file_get_contents works, however, due to the large number of xml files to be retrieved, the operation times-out.
Tentative solution:
- Download XML files from source
- Zip them
- Download xml_file
- Locally parse said xml files
Later on, write a small php scripts that, when invoked, performs the bits above, sends the data to the local directory, which then unpacks it and performs additional work on it.
Another attempt would be to use Google Sheets, with a user function that pulls the data into the sheet, and just dump the excel file / values into mysql.
For my purposes, while an awfully ignorant solution, it does the trick.
Code used for avoiding timeout issue on shared host:
function downloadUrlToFile2($url, $outFileName)
{
//file_put_contents($xmlFileName, fopen($link, 'r'));
//copy($link, $xmlFileName); // download xml file
;
echo "Passing $url into $outFileName ";
// $outFileName = touch();
$fp = fopen($outFileName, "w");
if(is_file($url))
{
copy($url, $outFileName); // download xml file
} else
{
$ch = curl_init();
$options = array(
CURLOPT_TIMEOUT => 28800, // set this to 8 hours so we dont timeout on big files
CURLOPT_URL => $url
);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt_array($ch, $options);
$contents = curl_exec($ch);
fwrite($fp, $contents);
curl_close($ch);
}
}
I have also added this on top of the ini script:
ignore_user_abort(true);
set_time_limit(0);
ini_set('memory_limit', '2048M');
I see some issue with HTTPS url request, for fix issue you have to add below lines in your CURL request
function curl_get_contents($url) {
$ch = curl_init();
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: ";
curl_setopt( $ch, CURLOPT_HTTPHEADER, $header );
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
// I have added below two lines
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
I have read and tried thousands of solutions in different posts and none of them seems to work with me. This three are example of that.
cURL is unable to use client certificate , in local server
php openssl_get_publickey() and curl - unable to use client certificate (no key found or wrong pass phrase?)
Getting (58) unable to use client certificate (no key found or wrong pass phrase?) from curl
I received a .p12 certificate which I converted to .pem file in https://www.sslshopper.com/ssl-converter.html
The password is correct otherwise it wouldn't convert it.
$xml = 'my xml here';
$url = 'https://qly.mbway.pt/Merchant/requestFinancialOperationWS';
$headers = array(
'Content-Type: text/xml; charset="utf-8"',
'Content-Length: ' . strlen($xml),
'Accept: text/xml',
'Cache-Control: no-cache',
'Pragma: no-cache'
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_SSLCERT, base_url() . 'public/cert.pem');
curl_setopt($ch, CURLOPT_SSLCERTPASSWD, 'my password here');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml);
$data = curl_exec($ch);
if(!$data)
print_r('ERROR: ' . curl_error($ch));
else
print_r('SUCCESS: ' . curl_error($ch));
I have tried with SoapUI application and works fine but with cURL I'm receiving the error:
unable to use client certificate (no key found or wrong pass phrase?)
I have tried without success:
Disable CURLOPT_SSL_VERIFYPEER and/or CURLOPT_SSL_VERIFYHOST
Add CURLOPT_SSLKEYTYPE and/or CURLOPT_SSLKEY fields
EDIT 1:
I have been trying around with SOAPClient besides cURL and it seems that the error might be the headers.
My headers after print_r($soapClient) are the following:
Host: qly.mbway.pt
Connection: Keep-Alive
User-Agent: PHP-SOAP/5.5.9-1ubuntu4.14
Content-Type: application/soap+xml; charset=utf-8; action=""
Content-Length: 1750
I would like to know how can I remove the action=""? I tried to extend the original class without success in terms of changing the header.
class MySoapClient extends SoapClient
{
public function __construct($wsdl, $options = array())
{
$ctx_opts = array('http' => array('header' => array('Content-Type' => 'application/soapyyyyyml')));
$ctx = stream_context_create($ctx_opts);
parent::__construct($wsdl, array('stream_context' => $ctx));
}
}
Solved with cURL.
The problem was the path of the pem file.
I was using base_url() . 'public/cert.pem' but that's not possible. Instead I need to use a relative path such as ./public/cert.pem.
I need to HTTP PUT a csv file and some POST fields using multipart POST with PHP and Curl to a REST API endpoint.
The contents of the file upload is stored in a variable $list. The other end point is $url.
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_PUT, true);
$post = array(
//Other Post fields array
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
$fh = fopen('php://memory', 'rw');
fwrite($fh, $list);
rewind($fh);
curl_setopt($ch, CURLOPT_INFILE, $fh);
curl_setopt($ch, CURLOPT_INFILESIZE, strlen($list));
$response = curl_exec($ch);
The above code seems to work the only problem is that the other end point requires a specific fieldname for the file upload. How do i set a filename ?
Am i doing something wrong ?
This is the PUT format they have mentioned on API
Content-Disposition: form-data; name="list[csv]"; filename="RackMultipart20110923-63966-hfpyg"
Content-Length: 33
Content-Type: text/csv
Content-Transfer-Encoding: binary
xxxx
yyyy
zzzz
-------------MultipartPost
Content-Disposition: form-data; name="list[list_type]"
Blacklist
-------------MultipartPost--
FYI that is multipart/form-data. You will need to build the body yourself I think, I don't think cURL could be made to build that sort of request with a PUT request. However, this is not a serious problem:
<?php
function recursive_array_mpfd ($array, $separator, &$output, $prefix = '') {
// Recurses through a multidimensional array and populates $output with a
// multipart/form-data string representing the data
foreach ($array as $key => $val) {
$name = ($prefix) ? $prefix."[".$key."]" : $key;
if (is_array($val)) {
recursive_array_mpfd($val, $separator, $output, $name);
} else {
$output .= "--$separator\r\n"
. "Content-Disposition: form-data; name=\"$name\"\r\n"
. "\r\n"
. "$val\r\n";
}
}
}
// This will hold the request body string
$requestBody = '';
// We'll need a separator
$separator = '-----'.md5(microtime()).'-----';
// First add the postfields
$post = array(
//Other Post fields array
);
recursive_array_mpfd($post, $separator, $requestBody);
// Now add the file
$list = "this,is,some,csv,data"; // The content of the file
$filename = "data.csv"; // The name of the file
$requestBody .= "--$separator\r\n"
. "Content-Disposition: form-data; name=\"list[list_type]\"; filename=\"$filename\"\r\n"
. "Content-Length: ".strlen($list)."\r\n"
. "Content-Type: text/csv\r\n"
. "Content-Transfer-Encoding: binary\r\n"
. "\r\n"
. "$list\r\n";
// Terminate the body
$requestBody .= "--$separator--";
// Let's go cURLing...
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_PUT, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $requestBody);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: multipart/form-data; boundary="'.$separator.'"'
));
$response = curl_exec($ch);
If you have any problems with this, try echo $requestBody; before the cURL request and make sure it looks like you expect it to.
I want to create a proxy like application from which I send the header to the server and the response goes right to the client and doesn't use all of the server bandwidth.
The only way I can think of is using PHP cURL for this, but that doesn't work since it downloads the file and the sends it to client. I want to know is there a way to remove or minimize the used bandwidth.
What I want to do:
Clients opens the page, presses the download button, then MY server requests to the file server for the file (using a header) and sends its directly to the client or MY server redirects to client.
Clients opens the page presses the download button
MY server requests to the file server the file and sends to the client 8k at time (in the following example).
This using CURLOPT_BUFFERSIZE, CURLOPT_HEADERFUNCTION and CURLOPT_WRITEFUNCTION.
<?php
/*
* curl-pass-through-proxy.php
*
* propose: php curl pass through proxy handle: big file, https, autentication
* example: curl-pass-through-proxy.php?url=precise/ubuntu-12.04.4-desktop-i386.iso
* limitation: don't work on binary if is enabled in php.ini the ;output_handler = ob_gzhandler
* licence: BSD
*
* Copyright 2014 Gabriel Rota <gabriel.rota#gmail.com>
*
*/
$url = "http://releases.ubuntu.com/" . $_GET["url"]; // NOTE: this example don't use https
$credentials = "user:pwd";
$headers = array(
"GET ".$url." HTTP/1.1",
"Content-type: text/xml",
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Cache-Control: no-cache",
"Pragma: no-cache",
"Authorization: Basic " . base64_encode($credentials)
);
global $filename; // used in fn_CURLOPT_HEADERFUNCTION setting download filename
$filename = substr($url, strrpos($url, "/")+1); // find last /
function fn_CURLOPT_WRITEFUNCTION($ch, $str){
$len = strlen($str);
echo( $str );
return $len;
}
function fn_CURLOPT_HEADERFUNCTION($ch, $str){
global $filename;
$len = strlen($str);
header( $str );
//~ error_log("curl-pass-through-proxy:fn_CURLOPT_HEADERFUNCTION:str:".$str.PHP_EOL, 3, "/tmp/curl-pass-through-proxy.log");
if ( strpos($str, "application/x-iso9660-image") !== false ) {
header( "Content-Disposition: attachment; filename=\"$filename\"" ); // set download filename
}
return $len;
}
$ch = curl_init(); // init curl resource
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, false); // a true curl_exec return content
curl_setopt($ch, CURLOPT_TIMEOUT, 600); // 60 second
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); // login $url
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false); // don't check certificate
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // don't check certificate
curl_setopt($ch, CURLOPT_HEADER, false); // true Return the HTTP headers in string, no good with CURLOPT_HEADERFUNCTION
curl_setopt($ch, CURLOPT_BUFFERSIZE, 8192); // 8192 8k
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_HEADERFUNCTION, "fn_CURLOPT_HEADERFUNCTION"); // handle received headers
curl_setopt($ch, CURLOPT_WRITEFUNCTION, 'fn_CURLOPT_WRITEFUNCTION'); // callad every CURLOPT_BUFFERSIZE
if ( ! curl_exec($ch) ) {
error_log( "curl-pass-through-proxy:Error:".curl_error($ch).PHP_EOL, 3, "/tmp/curl-pass-through-proxy.log" );
}
curl_close($ch); // close curl resource
?>