I would like to download content of certain page and get one number from it (still not sure how, probably using PHP DOM interface). I opened the page, started Firefox's debugging, picked the element with number and found out that is in <div id="lblOptimizePercent" class="wod-dpsval">98.4%</div> (98.4% is what I am looking for). So I opened its source code, Ctrl - F for lblOptimizePercent and all I found is this <div id="lblOptimizePercent" class="wod-dpsval"></div> without any content. What I've done wrong? Or is it some site's protection not to steal contents?
Link to the original site
Normally, to scrape the page from PHP, you would have to
save the page
extract the value you want from HTML via a regular expression
alternatives include using SimpleXML for DOM querying...
The piece of HTML we are look at is:
<div id="lblOptimizePercent" class="wod-dpsval">DATA</div>
<?php
$text = file_get_contents('http://www.askmrrobot.com/wow/optimize/eu/drak%27thul/Ecclesiastic');
$regexp = '^<div id=\"lblOptimizePercent\" class=\"wod-dpsval\">(.*)<\/div>^';
preg_match($regexp, $text, $matches);
$percentage = $matches[1];
echo $percentage;
This should give you DATA - the percentage value. But this doesn't happen! Why:
The data is dynamically inserted by a Javascript on client-side.
The id or class selector is used for DOM querying (element selection), then the data value is added.
http://api.jquery.com/id-selector/ - http://api.jquery.com/class-selector/
jQuery example
On this site they deliver <div id="lblOptimizePercent" class="wod-dpsval"></div>to the client and then they use an update query like this: $("#lblOptimizePercent").text("100%"); to update the percentage value.
If you want to query it on client-side, you might use $("#lblOptimizePercent").text();**
Try this in your console. It returns the percentage value.
How to scrape this page?
If you want to scrape this page with dynamic data, you need something like a Browser Environment for scraping: PhantomJS or SlimerJS are your friend.
Open the page with PhantomJS, launch the jQuery cmd from above and done.
This snippet should get you pretty close. You might save it as scrape.js then execute it with Phantom.
var page = require('webpage').create();
page.open('http://www.askmrrobot.com/wow/optimize/eu/drak%27thul/Ecclesiastic', function() {
page.includeJs("http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js", function() {
page.evaluate(function() {
alert(
$("#lblOptimizePercent").text()
);
});
phantom.exit()
});
});
You can also save the "evaluated page" (now with data) and do the extract with PHP.
That's exactly like: Save Page in your browser and working on the saved HTML file.
In Firebug or another webdeveloper tools you see the generated content, in Source code there is a blank element only.
First time, blank element is shown (during rendering site) and than using JS the content is filled.
Googlebot etc. can´t see this JS-generated content, but it´s no problem in this case.
Code:
document.getElementById('lblOptimizePercent').innerHTML = '94%';
Or similarly using jQuery:
$('#lblOptimizePercent').html('94%');
// need to load jQuery before, of course
Related
Basically, at the click of a button I need to refresh a div that contains embedded PHP. I tried using .html() in jQuery to reload the contents of the div, but it didn't change anything so I'm assuming it's not recalling the PHP (because the PHP should be changed at this point). It's probably just rewriting the HTML that was outputted by the PHP when the page loaded. I also tried appending something to the new HTML load so I could see it was at least refreshing the HTML code, and it was. It looked something like (if "updateObject" is a variable that contains the location of the div):
updateObject.html(updateObject.html() + " random text");
I also fiddled around with the .load() jQuery method, but it seemed to be for external PHP files (and at this point I can't change my embedded PHP to external PHP).
Any thoughts? Thanks!
EDIT: The biggest problem I've had is that, with my limited knowledge of web dev, I need to have the PHP embedded. If I make it a separate file, I'd have a very difficult time finding it because I honestly don't understand how the files get put together through the framework we're using (Drupal). I did try using an external PHP file but I couldn't figure out how to find it. Even if I put it in the same directory as this HTML file, it doesn't seem to be easy to find.
You are to have 2 files:
PHP
This contains the data you want to show.
HTML
This is the one showing the php content.
In PHP you can put whatever you want. But the HTML once is loaded is loaded and the only way to have some more content into it is to make what is called an asynchronous call, which means a call after the page has been loaded.
To do this you are to use jQuery and call the $.post (or $.ajax), using this syntax:
$.post('filename.php', function(dataFromPhp){
//actions to do once the php has been read
})
So in your case you can make a function that reads the php every time the click is done on a div/button/other DOM object; like so:
function updateDivContent(whatDiv){
$.post('filename.php', function(dataFromPhp){
if(dataFromPhp){
$(whatDiv).html(dataFromPhp)
}
})
}
So if you want the div to be refreshed (and reload the php) you are to connect that function to an event and specify the div you want to show the data:
$('#myBtn').click(function(){
updateDivContent('#myDiv');
})
Refer to this documentation for further informations: https://api.jquery.com/jQuery.post/
I’m trying to store the content of a div to a variable.
Example:
<div class="anything">
<p>We don't know the content of this div</p>
</div>
I want to search for <div class="anything"> and store everything between opening and the end tag.
We also want to avoid using absolute pathnames, so that it only searches the current HTML/PHP file for this div where the code is present.
Is this possible with PHP, or is this only possible with JavaScript ?
PHP is not that intelligent. He doesn't even know what he says.
PHP is a server-side language. It has absolutely NO clue about what the DOM (ie. what is displayed in your browser's window) is when it delivers a page. Yeah I know, PHP rendered the DOM, so how could it not know what's in there?
Simply put, let's say that PHP doesn't have a memory of what he renders. He just knows that at one particular moment, he is delivering strings of characters, but that's all. He kind of doesn't get the big picture. The big picture goes to the client and is called the DOM. The server (PHP) forgets it immediately as he's rendering it.
Like a red fish.
To do that, you need JavaScript (which is on the client's computer, and therefore has complete access to the rendered DOM), or if you want PHP to do this, you have to retrieve an full-rendered page first.
So the only way to do what you want to do in PHP is to get your page printed, and only then you can retrieve it with an http request and parse it with, in your case, a library such as simpleHtmlDom.
Quick example on how to parse a rendered page with simpleHtmlDom:
Let's say you know that your page will be available at http://mypage.com/mypage.php
$html = file_get_html('http://mypage.com/mypage.php');
foreach($html->find('div.anything') as $element)
echo $element->src . '<br>';
you probably need a combination of those.
In your Javascript:
var content = document.getElementsByClassName("anything")[0].innerHTML();
document.getElementByID('formfield').value(content);
document.getElementByID('hiddenForm').submit();
In your HTML/PHP File:
<form id="hiddenForm" action="path/to/your/script">
<input type="hidden" name="formfield" value="" />
</form>
In the script you defined in the form action:
if(!empty($_POST)){
$content = $_POST['formfield'];
// DO something with the content;
}
Alternatively you could send the data via AJAX but I guess you are new to this stuff so you should start slowly :)
Cheers!
steve
You could use JS to take the .innerHTML from the elements you wan and store them in .value of some input fields of a form and then use a submit button to run the PHP form handling as normal. Use .readOnly to make the input fields uneditle.
When a user selects a word in a text on my website (PHP), and then right clicks, i want a jQuery context menu to come up, this can be done by using one of the already existing jQuery context menu plugins.
But besides the options like copy / paste / cut, etc. I also want something to be done with the selected word using PHP. Which, i think, is a little harder.
For example using this script:
$selection = //the selected word or text
$target = //fetch from MYSQL database
$output = array();
while ($row = //fetch $target) {
If ($selection == $row->input) { array_push($output,$row->output); }
}
echo '//menu '.print_r($output).''; // of course not print_r! Just for the example's sake.
Databse example:
(Sorry for the oversized image)
Ok so selecting the word 'lazy' in the example text, and then right clicking, the jQuery box should pop up showing the results from the database extracted by PHP.
Example:
Ok, so i know you can't just combine javascript with PHP and it can only be parsed, but i thought loading an iframe withing the menu, which does the database extraction would do the job by using javascript to set the iframe src containing the selected word in the url.
However, iFrames are not really a nice way to solve this.
The question: How can i do this effectively? Execute this script on right-click and show the database-related content in the menu?
I would need to know the plugin you're using to give you some code examples but, general, I would go about this like this:
There has to be a click handler on the items in the jQuery context menu. Use it to submit an AJAX request to the server when the "selection" term is clicked.
Make sure to give the user some feedback (a loader or spinner)
Put the results into an array server-side.
JSON encode the array and send it as the response (e.g. echo json_encode($output)
JSON.parse(response) on client-side and you now have a JS object with the results
Put those results in the context menu (again, how depends on the plugin you're using)
AJAX is a great way to do what you want.
Here is a simple AJAX example. Note that in the 2nd .PHP file, that is where you put your database lookup etc.
Whatever you echo from the 2nd script is received by the calling javascript (first script again) and can be inserted into your context menu on-the-fly. Here is another example with a very detailed, step-by-step explanation of the process at the bottom of the answer.
I think you have to use Ajax to get JSON from a PHP file, which you would process on the actual page.
I you create a PHP file called test.php, with the following in it:
<?php
echo json_encode(array('time' => time(), 'hour', date('H')));
?>
Then the Javascript:
<script>
$('#curr_menu_entry').click(function() {
$.getJSON('test.php', function(data) {
$.each(data, function(key, val) {
$('#curr_menu_entry').append('<li id="' + key + '">' + val + '</li>');
});
});
});
</script>
Would that work?
I have a page called test.html. This HTML page will contain 2 JavaScripts file. I need to pass a variable value to a PHP file using the first JavaScript file. The user can copy this javascript on as many pages as he wants for displaying output.
The PHP file that receives the variable from the JavaScript file retrieves some data from database depending upon the variable's value. This retrieved value can contain HTML content. This PHP file will always reside on my server.
All of the retrieved content (from the PHP file) needs to be passed to the second JavaScript file so that the data can be displayed in browser. This JS file will need to stay together with the first JS file in order for the data to be displayed.
So I have this:
JavaScript File
<script type= "text/javascript" src="http://www.myserver.com/custom_script.php?unique_id=12"></script>
PHP FILE
//custom_script.php
<?php
$unique_id= (int)$_GET['unique_id'];
$res = db_res(" SELECT col1, col2, col3, .... col10 FROM table WHERE unique_id = $unique_id
LIMIT 1 ");
$rows = mysql_fetch_array($res);
?>
<div id="1"><?php echo $rows['col1']; ?></div>
<div id="2"><?php echo $rows['col2']; ?></div>
<div id="3"><?php echo $rows['col3']; ?></div>
.
.
.
<div id="10"><?php echo $rows['col10']; ?></div>
I need to send all the HTML above from the PHP file to the second JavaScript file so that the output can be displayed. Please note that the CSS styling is also applied using Div ID, so I am expecting those styles would show up too. Please note that there may be more than 10 columns, so an efficient way of passing data is highly appreciated.
So what would be the easiest and the best way to send all the HTML data from the PHP file in one go to the 2nd javascript file residing in test.html page, so that the HTML data can be displayed in test.html file?
EDIT 1:
I apologize to everyone if my question has been confusing. I just thought of an example and hence wanted to add it my edit. I hope you are all aware of what Google Analytics (GA)(or any other Website Visits Stats Tracker) does. Right? You register for a Analytics account and Google gives you a piece of JS code that you copy and paste in your website. And after couple of days, you can login into your GA account to see the stats. Correct? What I am trying to do here is just the same.
Users come to MY WEBSITE and register for an account and I give them JS files that they can paste in their website. The only difference between GA and my website is that GA is personal to you and no one else, but you, the account holder can see it. Whereas in my case, your data can BE SEEN by others as well, as long as you include the JS file on your website. Because users can't just take my PHP file and run it on their server, I am trying to access MY PHP file by giving the full path to it in the JS file.
For example:
<script type= "text/javascript" src="http://www.myserver.com/custom_script.php?unique_id=12"></script>
This is not an actual JS file, rather it is just a medium for my custom_script.php script to receive the unique_id of the user, query MY database and send back the HTML data related to this requesting user. And I am stuck with this part. Hope this clairifies what I am trying to do.
The jQuery documentation actually gives an example almost identical to your problem:
http://api.jquery.com/jQuery.get/
$.get('ajax/test.html', function(data) {
$('.result').html(data);
alert('Load was performed.');
});
This will get an html page and insert it under an html element with class 'result'.
You should be able to replace your php script url and specify where the output should be displayed.
If youre worried about conflicts with other jQuery instances, have a read of this:
http://api.jquery.com/jQuery.noConflict/
If you want to write your own AJAX handler check out this page :
https://developer.mozilla.org/en-US/docs/DOM/XMLHttpRequest/Using_XMLHttpRequest
Essentially you will do something like this (taken from the link):
function reqListener () {
console.log(this.responseText);
};
var oReq = new XMLHttpRequest();
oReq.onload = reqListener;
oReq.open("get", "yourFile.txt", true);
oReq.send();
I think its worth doing if you've never had to do it before, but you will open yourself up to lots of potential problems that have been solved for you.
Ideally if I were doing this I would return json from the handler and allow the user to decide how to display it, but thats your call.
I tried to understand - is it any method to ask browser not to refresh entire page when user clicks onto . Hash adding method is seen - I need another method, working with links without hashes.
May be any headers should be sent ? Or something another ?
I want to process GET queries returning only the part of HTML (or special js commands), not all page, and process it in AJAX-style.
You can ajaxify your links through jquery. Something like this:
$('a.ajax').click(function(ev){
ev.preventDefault();
var target=$(this).attr('data-target');
var url=$(this).attr('href');
$(target).load(url+' '+target);
}
This can be used in conduction with the following HTML:
<div id="output">
Hello World
<div>
and inside world.html you would need to have:
<div id="output">
Foo bar baz boo
</div>
In theory this should load content of the dif from "world" file into the div inside the first file, but I haven't tried it. I think it's what you need, because the regular link is still there, google will properly index this bypassing ajax and your users will be happy to see part of the page change.
you could make it 'fake' links doing something like this:
<span style="cursor:pointer;" onclick="loadPage('mypagename');">My Link</span>
The function then would be:
function loadPage(pageName){
// do ajax call here using pageName var
}
You cannot prevent navigation when a user clicks a hyperlink with a URL. Using a hash value to navigate or having your hyperlinks invoke JavaScript is generally the way to add navigation inside of a single page.
However, if you're trying to convert an existing page that's not designed this way, you would have to use JavaScript to modify hyperlinks so they invoke Ajax. For example:
var links = document.getElementsByTagName('a');
for (var i = 0; i < links.length; i++) {
var oldUrl = links[i].getAttribute('href');
links[i].setAttribute('href', 'javascript:void(0)');
links[i].onclick = (function(url) {
return function() {
// Then you can do your AJAX code here
}
})(oldUrl);
}
I recommend that you don't do something like this, though, and rather create your page with an AJAX design in mind.
Changing the url without using hashes can be achieved with the Html5 History API: http://html5demos.com/history
Not supported by older browser though, so you must always have a fallback.