I am building a PHP script that needs to use content loaded from an external webpage, but I don't know how to send/receive data.
The external webpage is http://packer.50x.eu/ .
Basically, I want to send a script (which is manually done in the first form) and receive the output (from the second form).
I want to learn how to do it because it can surely be an useful thing in the future, but I have no clue where to start.
Can anyone help me? Thanks.
You can use curl to receive data from external page. Look this example:
$url = "http://packer.50x.eu/";
$ch = curl_init($url);
// curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // you can use some options for this request
// curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5); // or not to use them
// you can set many others options. read about them in manual
$data = curl_exec($ch);
curl_close($ch);
var_dump($data); // < -- here is received page data
curl_setopt manual
Hope, this will help you.
You may want to look at file_get_contents($url) as it is very simple to use, simpler than CURL (more limited though), so your code could look like:
$url = "http://packer.50x.eu/";
$url_content=file_get_contents($url);
echo $url_content;
Look at the documentation as you could use offset and other tricks.
Related
I am working on a small project to get information from several webpages based on the HTML Markup of the page, and I do not know where to start at all.
The basic idea is of getting the Title from <h1></h1>s, and content from the <p></p>s tags and other important information that is required.
I would have to setup each case from each source for it to work the way it needs. I believe the right method is using $_GET method with PHP. The goal of the project is to build a database of information.
What is the best method to grab the information which I need?
First of all: PHP's $_GET is not a method. As you can see in the documentation $_GET is simply an array initialized with the GET's parameters your web server received during the current query. As such it is not what you want to use for this kind of things.
What you should look into is cURL that allows you to compose even fairly complex query, send to the destination server and retrieve the response. For example for a POST request you could do something like:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,"http://www.mysite.com/tester.phtml");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,
"postvar1=value1&postvar2=value2&postvar3=value3");
// in real life you should use something like:
// curl_setopt($ch, CURLOPT_POSTFIELDS,
// http_build_query(array('postvar1' => 'value1')));
// receive server response ...
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output = curl_exec ($ch);
curl_close ($ch);
Source
Of course if you don't have to do any complex query but simple GET requests you can go with the PHP function file_get_contents
After you received the web page content you have to parse it. IMHO the best way to do this is by using PHP's DOM functions. How to use them should really be another question, but you can find tons of example without much effort.
<?php
$remote = file_get_contents('http://www.remote_website.html');
$doc = new DomDocument();
$file = #$doc->loadHTML($remote);
$cells = #$doc->getElementsByTagName('h1');
foreach($cells AS $cell)
{
$titles[] = $cell->nodeValue ;
}
$cells = #$doc->getElementsByTagName('p');
foreach($cells AS $cell)
{
$content[] = $cell->nodeValue ;
}
?>
You can get the HTML source of a page with:
<?php
$html= file_get_contents('http://www.example.com/');
echo $html;
?>
Then once you ahve the structure of the page you get the request tag with substr() and strpos()
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Fetching data from another website
I want to create a webpage, that would display another webpage on it at user's request. User enters URL and sees webpage he wants on my website. Request to another page has to come from my server, not from user. Otherwise I could just use iframe.
I'm willing to write it on php because I know some of it. Can anyone tell me what subjects one must know to do this ?
You need some kind of "PHP Proxy" for this, that means get the website contents via curl or file_get_contents(). Have a look at this here: http://davidwalsh.name/curl-download
Your proxy script that may look like this:
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
echo get_data($_GET["url"]);
Please note that you may have to pay attention to headers for images etc. and there may also be some security flaws, but that is the basic idea.
Now you have to parse the contents of the initial website you just got and change all links from this format:
http://example.com/thecss.css
to
http://yoursite.com/proxy.php?url=http://example.com/thecss.css
Some regexes or PHP HTML parser may work here.
You could just use
echo file_get_contents('http://google.com')
But why not just download a php webproxy package like http://sourceforge.net/projects/poxy/
I am searching 3 days for an answer and I cannot find one because I always find some obstacles.
I need to load a web page (the reason for this is to accept a cookie) and then at the same time read the source code of the new page without hitting it again. The reason for this is that the page is dynamic so the content will change.
I have tried to do this using iFrame(document.body.innerHTML) but the fact that these pages run on different servers I hit cross-site scripting issues.
I have also tried writing a php script using get_contents but this doesn't allow the cookie to be stored in my local.
This is driving me crazy.... Any suggestion will be helful! Need to use PHP or Javascript for this and any other suggestion will be useful as well.
When you are on the page document.body.innerHTML will give you the page source.
Edit: I didn't realize you were loading it like that. See this SO question.
It can be done using cURL in PHP.
A rough implementation:
$ch = curl_init('http://www.google.com/');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$data = curl_exec($ch);
preg_match('/^Set-Cookie: (.*?);/m', $data, $cookies);
var_dump($cookies);
var_dump($data);
$data will contain the entire response, so we need to parse out the cookie headers ourselves.
If available on your system, HttpRequest would make this easier.
I would like to be able to send variables to another website without actually going to the website using php.
I am building an ecommerce website where the shipping warehouse is being outsourced. After the person checks out with their products, I would like to send some variables over to the shipper's website using $_GET['vars']. This is a completely different URL. The problem is, I don't want the person actually going to the shipper's webpage. I just want to ping that info over there.
Is is possible to send Variables via URL to another site without leaving yours?
Yes, you can. the simplest way:
$contents = file_get_contents("http://example.com/some/page.php?var=abcd");
For more advanced features see Curl.
You should be storing all the relevant order info within your database then using cron to trigger a script that will process the unprocessed, this way systematic checks can be made on orders before any request to your outsource site. Dont rely on your users browser to hit a curtain point in the order process to trigger the API call or trust them not to triple click or inject values before submitting.
And I advise to use curl todo your actual request as its faster. Something as simple as:
<?php
function curl_do_api($url){
if (!function_exists('curl_init')){
die('Sorry cURL is not installed!');
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
?>
actually there is a more simpler way of solving this...using ajax
include the jquery header first
refer here
http://api.jquery.com/jQuery.ajax/
Both are right and you should make your choice based on security you're looking for.
cURL will be more secure and you should use it if you do not want to pass some argument in query string. At the same time when you pass data using file_get_cotents("URL?data=value"); you will have limit of about 2k for data being passed.
On the other side cURL will be secure if you use it with https it's much more secure. With cURL you will also be able to post files and emulate form post.
How do I get the page source of the html page using php codes and then save all all of those codes into a database?
Is it possible?
Please help. Thanks.
Database is MySQL
The answer lies within file_get_contents() or cURL.
$string = file_get_contents('http://www.example.com/');
echo $string;
// create a new cURL resource
$ch = curl_init();
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, "http://www.example.com/");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
// grab URL and pass it to the browser
$string = curl_exec($ch);
echo $string;
// close cURL resource, and free up system resources
curl_close($ch);
As far as storing in a database, presumably MySQL, you could use mysql_query(). I'd suggest using prepared statements but it seems as though that might be overwhelming at first glance for a PHP beginner.
Here's a very simple example of database connection and insertion:
$db = mysql_connect('DB_IP_OR_HOST','DB_USER','DB_PASS') or die("DB error");
mysql_select_db('YOUR_DB_NAME', $db);
$result = mysql_query('INSERT INTO MY_TABLE_NAME SET html = "' . mysql_real_escape_string($string) . '"');
mysql_close($db);
If it lies on an external server, you can not view the PHP source, as it is a server-side language. You can only view the client-side source (generally HTML). If you would like to learn PHP to understand how the page was created, and make one similar, I suggest using Tizag's PHP tutorial as they are very easy to understand.
However, otherwise you can use fopen() or file_get_contents()