I want a data from other website which url looks like "https://www.example.com/quote/results" after fill all data. I not understand how to pass data to get correct data.I try this code but it show only blank page.
<?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
include('simple_html_dom.php');
$czip=$_POST['czip'];
$dest=$_POST['dest'];
$weight=$_POST['weight'];
$url = 'https://www.example.com/quote/results';
$data = array('collection-zip' =>$czip,
'destination'=>$dest,
'weight' =>$weight);
$ch = curl_init();
$timeout=5;
$ch = curl_init($url);
$data=http_build_query($data);
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
$data = curl_exec($ch);
curl_close($ch);
echo "<pre>";
print_r($data);
exit;
if($headers['http_code'] == 200){
echo $content;
exit;
}
?>
I am not familiar with curl and web scraping.
if any soultion please share.
Related
I am trying to Scrape dropdown data from this https://www.equibase.com/premium/eqbRaceChartCalendar.cfm?SAP=TN
I use curl to create dom from Url
<?php
include('simple_html_dom.php');
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$returned_content = get_data('https://www.equibase.com/premium/eqbRaceChartCalendar.cfm?SAP=TN');
echo $returned_content;
?>
When i get content like that it's not return the hole webpage. it's not returnning middle part content(search aria and calander aria) of this page.
so i cant Scrape from Track,month,Year dropdown data because that part not return with curl. How return hole web page with curl with middle content
scraping perticular data from website using php without using any tools,i have tried this code but it is not sufficient-
<?php
$url = 'http://www.google.com';
$output = file_get_contents($url);
echo $output;
?>
you can used curl in php
<?
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.example.com');
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$data = curl_exec();
curl_close($ch);
?>
where $data contain html of given url
My PHP code (free.php) on http://techmentry.com/free.php is
<?php
{
//Variables to POST
$access_token = "b34480a685e7d638d9ee3e53cXXXXX";
$message = "hi";
$send_to = "existing_contacts";
//Initialize CURL data to send via POST to the API
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://freesmsgateway.com/api_send");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,
array('access_token' => $access_token,
'message' => urlencode('hi'),
'send_to' => $send_to,)
);
//Execute CURL command and return into variable $result
$result = curl_exec($ch);
//Do stuff
echo "$result";
}
?>
I am getting this error: THE MESSAGE WAS BLANK
This error means: "The message field was blank or was not properly URL encoded" (as told by my SMS gateway). But as you can see that my message field isn't blank.
I believe you can't send an array to CURLOPT_POSTFIELDS, you would need to replace the line with the following
curl_setopt($ch, CURLOPT_POSTFIELDS, "access_token=".$accesstoken."&message=".urlencode('hi')."&send_to=".$send_to);
I hope this solves it
Use http_build_query():
<?php
{
$postdata = array();
//Variables to POST
$postdata['access_token'] = "b34480a685e7d638d9ee3e53cXXXXX";
$postdata['message'] = "hi";
$postdata['send_to'] = "existing_contacts";
//Initialize CURL data to send via POST to the API
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://freesmsgateway.com/api_send");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($postdata) );
//Execute CURL command and return into variable $result
$result = curl_exec($ch);
//Do stuff
echo "$result";
}
?>
I have made a script that generates an IMDB API link for a movie in XML.
Once this link is generated it will save to an XML file with its contents. The only issue is that the contents aren't saving.
Link generated:
http://imdbapi.org/?title=One+Piece&type=xml&plot=simple&mt=none&episode=0&aka=simple&release=simple
PHP script:
$url="http://imdbapi.org/?title=One+Piece&type=xml&plot=simple&mt=none&episode=0&aka=simple&release=simple";
$curl = curl_init();
$data = fopen("text.xml", "w");
curl_setopt ($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_FILE, $data);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_exec ($curl);
if ( !$data ) {
echo "No";
} else {
$contents = curl_exec($curl);
fwrite($data, $contents);
}
curl_close($curl);
fclose($data);
Instead of using file_get_contents, you can use CURL
$ch = curl_init('http://imdbapi.org/?title=One+Piece&type=xml&plot=simple&mt=none&episode=0&aka=simple&release=simple');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
curl_close($ch);
Now $response shall contains your XML. And you can do something like
file_put_contents('filename.xml', $response);
make sure that filename.xml is writable
I have built a custom api using php, its a simple api that works by posting xml data.
The code that I have working to post to the api is:
<?php
$xml_data = '<document>
<first>'.$first.'</first>
<last>'.$last.'</last>
<email>'.$email.'</email>
<phone>'.$phone.'</phone>
<body>TEST</body>
</document>';
$URL = "url";
$ch = curl_init($URL);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
curl_setopt($ch, CURLOPT_POSTFIELDS, "$xml_data");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
$Response = curl_exec($ch);
curl_close($ch);
echo "Responce= ".$responce;
?>
On the other side, the code the above posts to:
<?php
$postdata = file_get_contents("php://input");
$xml = simplexml_load_string($postdata);
$first = $xml->first;
$last = $xml->last;
$email = $xml->email;
$phone = $xml->phone;
?>
Then I take those php variables and send to a database.. ALL THIS CODE IS WORKING!!
But my questions is: How do I send a response back to the posting side?
How do I use curl_init to send to curl_exec?
Any help would be great! Thank you
Jason
I think you want:
echo "Responce= ".$Response;
^^^
To return a response you'd do the same as you would for any other content, set a header and echo your output. For example, to return an xml response, from the script handling the post data do the following
<?php
$postdata = file_get_contents("php://input");
$xml = simplexml_load_string($postdata);
$first = $xml->first;
$last = $xml->last;
$email = $xml->email;
$phone = $xml->phone;
// do your db stuff
// format response
$response = '<response>
<success>Hello World</success>
</response>';
// set header
header('Content-type: text/xml');
// echo xml identifier and response back
echo chr(60).chr(63).'xml version="1.0" encoding="utf-8" '.chr(63).chr(62);
echo $response;
exit;
?>
You should see the response returned from curl_exec()