Fetching Data from mca.gov.in using curl and PHP - php

I am trying to fetch the data from http://mca.gov.in. I am using CURL and PHP for extracting the data.
Here is the form : http://www.mca.gov.in/mcafoportal/viewSignatoryDetails.do
When I do the HTML form submit directly through this HTML form to ( "http://www.mca.gov.in/mcafoportal/viewSignatoryDetails.do"), it works.
I developed my own form and submitted directly to the above link and it works too.
However, when I pass the same parameters to retrieve the data using curl, it is not giving any data.
Here is the code I am using to fetch the data from http://mca.gov.in.
$data = http_build_query($post_data);
curl_setopt($ch, CURLOPT_URL, "http://www.mca.gov.in/mcafoportal/viewSignatoryDetailsAction.do");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
echo $result;
Any idea, what could be wrong or If I have missed anything ?

Your problem is that the url you are trying to fetch uses a captcha and you cannot bypass this by code.
You need to use a this url "http://www.mca.gov.in/mcafoportal/popupSignatories.do" that does not require a captcha.
So change your code to:
curl_setopt($ch, CURLOPT_URL, "http://www.mca.gov.in/mcafoportal/popupSignatories.do");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, array(
"companyid" => "AAA-4416"
));
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
echo $result;
Additionally if you want the company details, use "http://www.mca.gov.in/mcafoportal/companyMasterDataPopup.do"
curl_setopt($ch, CURLOPT_URL, "http://www.mca.gov.in/mcafoportal/companyMasterDataPopup.do");
curl_setopt($ch, CURLOPT_POSTFIELDS, array(
"method" => "ajaxGetCompanyMasterData",
"companyid" => 'AAA-4416'
));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
echo $result;
Update
There are some new APIs that can be found at company master data API from https://data.gov.in

Related

cURL does not send postfields

I have been reading documentation and queries for several hours and I think I have a correct POST made with cURL.
It communicates correctly, because I receive the auth key and OK is validated, but then it doesn't receive any parameters from the post. I think some parameter must be missing, but I don't see anything in the documentation.
My code:
$data = "email=name#xxxx.com&template=3";
$url = "https://xxxx.net/xxx/aaa/bbb";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
"keyname: keyvalue"
));
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data );
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
$result = curl_exec($ch);
Does anyone know if I should stick some more configuration??

Get some data from external website via cURL

I want to pull some html pice of code with some data from an external website with cURL, but a recived a null response. If I type the url in browser i get the data that i want but i need to do that from my app via cURL or file_get_contents();
This is my code :
$_awb = '888000074599';
$url = 'http://89.45.196.45';
$post_fields = ':8080/?id=8IM*8J*9K&NT='.$_awb;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_fields);
$result = curl_exec($ch);
curl_close($ch);
You should not be using CURLOPT_POSTFIELDS. You are trying to do a GET request with a query string, but postfields is intended for making POST requests.
So setting CURLOPT_URL to the proper (absolute) URL should work.
$url = "http://89.45.196.45:8080/?id=8IM*8J*9K&NT="
$nt = "888000074599";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url . $nt);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
curl_close ($ch);

Form submit not working via curl

I am trying to login to a site using curl but for some reason when I use the print_r($login) to view the output, I always end up seeing the original login page without it being processed (it was supposed to return something like login success or fail), I think curl is not able to submit the form, any suggestion?
$user="something"; $pass="something";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, "username={$user}&password={$pass}");
curl_setopt($ch, CURLOPT_USERAGENT, "Something");
$login = curl_exec($ch);
print_r($login);
You need to add
curl_setopt($ch, CURLOPT_POST, TRUE);
To send POST content. Without it, your CURLOPT_POSTFIELDS will be ignored.

logging into twitter with curl

I'm trying to login into twitter with php + curl, but I think there's something wrong with my request because I get this as response:
Something is technically wrong.
Thanks for noticing—we\'re going to fix it up and have things back to
normal soon.
The php code I'm using is:
<?php
$ch = curl_init($sTarget);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $sPost);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $_CKS);
curl_setopt($ch, CURLOPT_COOKIEFILE, $_CKS);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-type: multipart/form-data"));
curl_setopt($ch, CURLOPT_REFERER, "https://twitter.com");
?>
$sPost looks like this:
session[username_or_email]=user&session[password]=password&remember_me=1&scribe_log=&redirect_after_login=&authenticity_token=6e706165609354bd7a92a99cf94e09140ea86c6f
The code first fetches the login form fields so that the post variables do have the proper values(auth token). I just tried about everything and yes I know there's an api for this but I'd rather find out how to do it manual for learning's sake.
Thanks in advance.
The CURLOPT_COOKIESESSION is used to indicate a new session. That's not what you want, since you need to send the session cookie in the second post.
I got the twitter login to work with this code:
<?php
# First call gets hidden form field authenticity_token
# and session cookie
$ch = curl_init();
$sTarget = "https://twitter.com/";
curl_setopt($ch, CURLOPT_URL, $sTarget);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/cookie.txt");
curl_setopt($ch, CURLOPT_REFERER, "https://twitter.com/");
$html = curl_exec($ch);
# parse authenticity_token out of html response
preg_match('/<input type="hidden" value="([a-zA-Z0-9]*)" name="authenticity_token"\/>/', $html, $match);
$authenticity_token = $match[1];
$username = "your#email.com";
$password = "password";
# set post data
$sPost = "session[username_or_email]=$username&session[password]=$password&return_to_ssl=true&scribe_log=&redirect_after_login=%2F&authenticity_token=$authenticity_token";
# second call is a post and performs login
$sTarget = "https://twitter.com/sessions";
curl_setopt($ch, CURLOPT_URL, $sTarget);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $sPost);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-type: application/x-www-form-urlencoded"));
# display server response
curl_exec($ch);
curl_close($ch);
?>
PS: Sorry for not reading your post properly the first time.
I noticed two things:
1) Try to URL encode your POST data
such as:
session%5Busername_or_email%5D=user&session%5Bpassword%5D=password...
instead of:
session[username_or_email]=user&session[password]=password...
2) twitter has a hidden field named authenticity_token in the login form. It is bound to the session. Thus you cannot use a static authenticity_token, you have to read the login form first and use the authenticity_token field from there.
Hope that helps.

POST JSON with PHP cURL

I have the following php code
curl_setopt($ch, CURLOPT_URL, $URL);
curl_setopt($ch, CURLOPT_USERAGENT, $this->_agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $this->_headers);
curl_setopt($ch, CURLOPT_ENCODING , "gzip");
curl_setopt($ch, CURLOPT_VERBOSE, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 120);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, $this->_cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEJAR, $this->_cookie_file_path);
curl_setopt($ch, CURLOPT_POSTFIELDS, '{"folderId":"1","parameters":{"amount":3,"ascending":false,"offset":0,"sort":"date"}}');
curl_setopt($ch, CURLOPT_POST, 1);
But I don't understand why is not working . The API that I'm posting the JSON to says that the parameters were not received . Is there anything wrong in my code ? I think the whole trick is on the JSON parameters... I'm not sure how to send them as I couldn't see any "nave->value" pair with the http analyzer as it usually appears in simple forms ... just that JSON code without any "name".
You can try as follows.
Basically if we have a array of data then it should be json encoded using php json_encode. And you should also add content-type in header which can be defined in curl as curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
$data = array("country"=>"US","states"=>array("MHASASAS"=>"MH","XYZABABA"=>"XYZ"));
$postdata = json_encode($data);
$ch = curl_init($URL);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
$result = curl_exec($ch);
curl_close($ch);
you can use this and replace with
curl_setopt($ch, CURLOPT_POSTFIELDS, '{"folderId":"1","parameters":{"amount":3,"ascending":false,"offset":0,"sort":"date"}}');
use this
$Parameters = array(
'MerchantCode' => $MerchantCode,
'PriceValue' => $amount,
'ReturnUrl' => $callback,
'InvoiceNumber' => $resnum,
);
curl_setopt($curl, CURLOPT_POSTFIELDS, json_encode($Parameters));
If:you use post method,you should know that:
CURLOPT_POST TRUE to do a regular HTTP POST. This POST is the normal application/x-www-form-urlencoded kind, most commonly used by HTML forms.
#see http://php.net/manual/en/function.curl-setopt.php
So:you can do like this:
$ar_form = array('name'=>'PHPJungle','age'=>66,'gender'=>'male');
$poststr = http_build_query($ar_form ); # important
$options[CURLOPT_HTTPGET] = false;
$options[CURLOPT_POST] = true;
$options[CURLOPT_POSTFIELDS] = $poststr ; //default type:application/x-www-from-urlencoded
curl_setopt_array ( $ch, $options );
# #todo your other codes
This is my class I have used for a long time.The class is based on PHP cURL.
It supports GET/POST,HTTP/HTTPS.
#see https://github.com/phpjungle/iHttp/
You can post a json data with curl like so:
Using Command Prompt:
curl -X POST -H "Content-Type: application/json" -d '{"folderId":"1","parameters":{"amount":3,"ascending":false,"offset":0,"sort":"date"}}' $url
Using PHP:
$data = array("folderId"=>"1","parameters"=>array("amount"=>3,"ascending"=>false,"offset"=>0,"sort"=>"date"));
$postdata = json_encode($data);
OR
$postdata = '{"folderId":"1","parameters":{"amount":3,"ascending":false,"offset":0,"sort":"date"}}';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
$result = curl_exec($ch);
curl_close($ch);
print_r($result);
You haven't set the content type, so the post data is being sent as form data. Try setting the content type to application/json.
If that doesn't work, try wrapping the json string with an array.
$link = "http://test.domain/myquery";
$json = '{"folderId":"1","parameters":{"amount":3,"ascending":false,"offset":0,"sort":"date"}}';
$postdata = json_decode($json);
echo openurl($link, $postdata);
This works as json decode converts a json string into array.
function openurl($url, $postvars = "") {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postvars);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, '3');
$content = trim(curl_exec($ch));
curl_close($ch);
return $content;
}
if your API endpoint using body for send request using json data may be you can use Guzzle the doc is here doc.
use GuzzleHttp\Client;
$client = new Client();
$request = $this->client->post($url,array(
'content-type' => 'application/json'
),array());
$request->setBody($json_data);
$response = $request->send();
return $response;
hope this work.
You can do it make by steps:
$data = array(
'folderId'=>"1","parameters"=>array(
"amount"=>"3","ascending"=>false,"offset"=>0,"sort"=>"date"
)
);
$data_string = http_build_query($data);
$ch = curl_init();
curl_setopt($ch,CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_POST, count($data));
curl_setopt($ch,CURLOPT_POSTFIELDS, $data_string);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
curl_close($ch);
$result = json_decode($result,true);
I don't know if you need the header. I think that by default it is already application/x-www-form-urlencode
If it doesn't work, try changing the $data values in array. Think it helps. :)
I'm not sure, that this is the solution but this works for me when posting json, change the json from
'{"folderId":"1","parameters":{"amount":3,"ascending":false,"offset":0,"sort":"date"}}'
to
"{'folderId':'1','parameters':{'amount':3,'ascending':false,'offset':0,'sort':'date'}}"
The only change i made was the double quotes are now on the outside, that works for me but I'm obviously posting to a different server
Only other help I could offer is to download a network debugging tool such as Fiddler or Charles proxy and monitor the requests sent/received, it could be a case that something else is wrong in your code.
Hope i helped :)
first of all
please check the curl http status code
$http_code= curl_get_info($exec_res,CURL_HTTP_CODE);
then modify this request header set with post header API server add a recorder to log those request info.

Categories