I'm using urlread in MATLAB to load the content of websites and then search that for further use.
However, I came across a site on which the content I need is located in a specific internal frame which is embedded in the index.php through <iframe>.
When I use urlread with the url of the file which is seemingly embedded in the main file, I still only get the content of that index.php instead of that of the targetframecontent.php.
I guess urlread might not be the correct tool for this or I might be overlooking other things which make my attempts unsuccessful.
Is there a way to get the content of such an internal frame into MATLAB?
EDIT: A bit more precisely:
I would usually use the following to read in the content of a website to my MATLAB workspace:
data = urlread('http://[...]index.php')
But doing that I only get the content of the parent index.php and not that of the embedded targetframecontent.php. I figured that out when going through the websites source code, where the part of the website which includes the content I'm interested in gets loaded into a frame with:
<iframe src="http://[...]targetframecontent.php" width="850px" height="1000px" border="0" frameborder="0"></iframe>
Because of that I tried to use urlread directly with the url which is loaded in the specific target frame.
However, when I did this I still got the same content as if I was still loading the index.php.
So it seems to redirect urlread back to index.php whenever I try to use the direct url of targetframecontent.php. That is also what happened when I tried to load targetframecontent.php in a browser.
Thanks for any help.
It looks like the site in question checks the Referrer of the HTTP request. Using wget, if you pass the --referer option, you can download the <iframe>:
$ wget --referer="http://www.sf.tv/sfmeteo/lokalprognosen/index.php?q=Gen%C3%A8ve" -O test_ref "http://www.sf.tv/sfmeteo/lokalprognosen/detailprognose.php?id=&q=Gen%C3%A8ve&max=&drs=0&kiosk=&js=&deeplink=&f="
$ cat test_ref
[...]
<body class="sf">
<!-- referrer:http://www.sf.tv/sfmeteo/lokal -->
<script type="text/javascript"> <!--
var keineprognose = '0';
// -->
</script>
<p class="inv">
zum Inhalt<br />
</p>
However, if you wget without sending a referrer, you get:
wget -O test_noref "http://www.sf.tv/sfmeteo/lokalprognosen/detailprognose.php?id=&q=Gen%C3%A8ve&max=&drs=0&kiosk=&js=&deeplink=&f="
<body class="sf" role="application">
<p class="inv">
zum Inhalt<br />
</p>
<!--googleoff: all--> <div id="HEADWRAP" class="sf-header">
<div class="INNERWRAP">
<h1 class="inv">SF Schweizer Fernsehen - Navigation</h1>
Which is the original, parent page.
Therefore, to suck this into MATLAB, you'll need to set the Referer HTTP header. Unfortunately I don't have MATLAB in front of me right now, but this page looks like it gives a good introduction to how to send headers with MATLAB HTTP requests: http://undocumentedmatlab.com/blog/expanding-urlreads-capabilities/
Edit: Because it's not super-obvious from that page, here's a link to the urlread2 function, which also includes some examples: http://www.mathworks.com/matlabcentral/fileexchange/35693-urlread2/content/urlread2.m
Related
I know how simple this probably seems to you gurus, but I have been searching for over an hour to no avail...
Goal: Use a single footer file and menu file for all my webpages. Taking into account blocking, speed, etc. The content of my menu is pure html/css and the content of my footer is pure html/css. Would the optimal solution change based on the content being injected? e.g. If videos, jscript, etc. were involved.
Two part question:
1) Which method is optimal? Some kind of php include, using the tag, using jscript, etc.
2) How precisely is this achieved keeping HTML 5 standards? i.e. For the php method to work, does my calling webpage need to be .php and then does that make the HTML5 standard a moot point? e.g. If I want to inject footer.php into index.html, does my index file also have to be .php? Similarly for the tag, can the external file be an .html file(I don't like the idea of reloading all the header information with .css calls) or should it be .php?
Within the index.html file I have tried the following:
<object id="footerArea" width="100%" height="20%"
type="text/html" data="footer.html">
</object>
and
<?php include 'footer.php' ?>
Neither of these seem to work for me.
In case you are wondering... Here is the code for my footer I am trying to inject with sample data to make it shorter and easier to read:
<div class="footer box">
<p class="f-right t-right">
www.mysite.com<br />
Address: Medford, OR<br />
Phone: (541) 555-5555
</p>
<p class="f-left">
Copyright © 2011 My Name<br />
</p>
<p class="f-left" style="margin-left:20px;">
<a href="http://sampleurl.com" target="_blank">
<img style="border:0;width:88px;height:31px"
src="http://sampleurl.com"
alt="Valid CSS3!" />
</a>
</p>
<p class="f-left" style="margin-left:20px;">
<a href="http://sampleurl" target="_blank">
<img src="http://sample.png" width="228" height="50" alt="sample alt" title="sample title">
</a>
</p>
</div>
Please excuse my formatting. I am still new to posting code in forums. I tried my best :)
The extension of a filename you seen in a url has absolutely NOTHING with how that file will be treated by a browser when it's downloaded. It all comes down to the Content-type header that accompanies the file. A webmaster can trivially configure their server to treat all .exe files as plain HTML pages. They can also tell the webserver to run .html pages through the PHP parser. In fact, with "modern" SEO-optimized urls, you rarely see a file extension at all. It'll all be things like example.com/some/wonky/path, not example.com/page.php?id=wonky.
The fact that PHP has built and output a page also has nothing to do with HTML compliance. It comes down to whether the page the browser receives conforms to the standards. Are all tags properly closed? Attributes properly defined? Tags properly nested? Blah blah blah.
If you've built your code properly, the html that's output will be properly structured and be valid html. If it's not valid html, that's not PHP's fault - that's your fault for putting together code that doesn't produce the proper output.
The only time a file extension in a URL MIGHT be relevant is if the webserver outputs a generic content-type, e.g. "application/octet-stream". The browser MAY use a detectable file extension to guess at the content's type and try to treat it as such. But this is not guaranteed nor reliable.
This is what a PHP include should look like:
<?php include 'footer.php'?>
As far as I can see the code you have in your question is assigning the string "footer.php" to the variable include. However, rather than rolling your own template system, have you considered using something like Smarty?
If the called code like for a footer that is canned, you might want to create the simple footer like you want then include:
<?php
readfile("yourfile.htm");
?>
Something like the following should do what you want:
index.php
<html>
<head>
<link rel="stylesheet" type="text/css" href="main.css" />
<!-- other css, scripts etc -->
</head>
<body>
<!-- your main content here -->
<?php include 'footer.html' ?>
</body>
</html>
footer.html
<div class="footer box">
<!-- your footer content here -->
</div>
when you load index.php in the browser, and "view source", you should see the contents shown above, but the <?php include "footer.html" ?> should be replaced with the content from the file "footer.html".
You should not see the php code itself when you view source through the web-browser. If you do see php code in "view source", this indicates that your server isn't configured to run php properly.
For an alternate approach, which loads the content from the browser, and which doesn't use php, I'll point you to this related question and answer.
If you visit this page in Chrome:
http://www.immigrationconsult.org/contact.php
And Inspect Element on the page, go to Console you will see this error:
GET htt...cms/contact/images/ajax-loader.gif 404 (Not Found) jquery.min.js:4
I followed the instructions here to create a:
http://css-tricks.com/weird-file-requests-and-easing-server-stress-with-htaccess/
I tested to make sure it works, and it does, but not on this specific request. jquery.min.js is the jQuery minified from the makers, I did not change it at all. I used Agent Ransack to deep search for any reference of this in any of my files, the search yielded no results.
I have no idea what to do or how to prevent this issue from happening, this seems like such a small issue, but I can not locate the cause of the problem! Please help.
So, it was really a misdirection by Chrome. The plugin at fault was jQuery Coda-Slider. Because it was JS compressed, Agent Ransack couldn't find the string... lesson learned...
You clearly don't have the ajax-loader.gif file in the correct directory, or at least your ajax script is looking the wrong place.
you can either modify the java script to point to the right location or you can just put the file in the correct location, i would choose the latter.
I have had these errors my self with JQuery, where some icons didn't get loaded, just had to put them in the correct folder and it worked.
But if you use googles jquery libs, all resources should get loaded automagic as well. Like this...
<script type="text/javascript"
src="http://www.google.com/jsapi">
</script>
<script type="text/javascript">
google.load("jquery", "1.7.2");
google.load(etc..);
</script>
You need to download the entire zip file from their website: http://jquerymobile.com/download/. After you unzip it, you will find the js file, the CSS and a folder called "images", all of which need to be present locally.
My issue is following: My Partner's have websites and I want that partners can include a script from my server that dynamically generates a link and an image.
i.e.
myscript.php (on my server) generates following output:
<a href="http://www.myserver.com/apples.php?12345">
<img src="http://www.myserver.com/images/12345.png" />
</a>
I don't want my partners to be bothered with any php scripting, hence the best solution for now is to supply them with an iframe, where the src is the link to my script on my server that generates the output above.
However I clearly don't have any control over my partner's website and i.e. opening a lightbox out of the iframe certainly won't work.
Because of that I tried a different approach. I used Ajax to dynamically load my script, however this also doesn't work due to ajax security features (you cannot call an external script via ajax - only if you do a php workaround).
So basically, my question is: Is there any good solution to call my script on my server on the partner's website and display the generated code on my partner's website without the use of php?
Have your PHP generate JavaScript (don't forget the content-type header). Then they can include it with a <script> element.
Make a PHP file, like
<?php
$url = "http://www.myserver.com/apples.php?12345";
$img = "http://www.myserver.com/images/12345.png";
echo "document.getElementById('divthing').innerHTML = '<img src=\"" . $img . "\" /> '";
?>
Your partner's page would be like:
<html>
<body>
Hey, check this site out: <div id="divthing"></div>
<script type="text/javascript" src="http://yoursite.com/script.php"></script>
</body>
</html>
(I know, not really clean code (innerHTML etc), but you should get the idea :))
Could you make it with javascript file which replace/creates that anchor where ever you put the javascript link.
picture.js:
$('#image').ready(function(){
var image = 'http://host/url/to/image.jpg';
$('#image').load(image);
});
on you partners site:
<div id="image">
<script type="text/javascript" src="http://yourhost/picture.js"></script>
</div>
I don't know if it possible, but.. :) and this needs jQuery. And im slow.
I am having difficulty trying to get modernizer to run with codeigniter.
this is what I've done so far.
downloaded Modernizr and renamed the file modernizr-1.5.min.txt to modernizr-1.5.min.js
put the JavaScript file in the same directory as my header file is: apppath/views/tops/
included the file in my headings <script src="modernizr-1.5.min.js"></script>
included this in my HTML element <html class="no-js">
just to preserve my sanity I put the JavaScript file modernizr-1.5.min.js in my views directory and in the application directory.
I am getting absolutely zero response when I read my page source to see if the has been replaced with the elements that my browser covers the wave modernizr is supposed to work. I have tried this using Firefox and chrome as far as reading the source.
any suggestions? Thank you in advance
why would you put javascript files ito the view folder of your app? i would rather use this path "/media/javascripts/modernizr.js" and then <script src="/media/javascripts/modernizr.js"></script> link it like that. the view folder is only for templates. other thing is that you will never see the changes that javascript does to your page in the pagesource. because it only shows you what html the browser received. and javascript starts to work after the browser received the html. you need to install firebug to see the "live" dom.
I'm a beginner in PHP and Javascript..
I found a link from http://cmichaelis.whsites.net/whblog/jquery-extjs-1/example2
Inside it there is a code saying :
function addPanel(location)
{
tabpanel.add({
autoLoad: {url: location},
title: 'More Information...',
closable:true,
autoScroll:true
}).show();
}
how to use :
<a href="javascript:void(0);"
onclick="addPanel('loadpage.php?a=http://www.google.com')">
head over to Google
</a>
What I want to ask is.. what is the code for loadpage.php?
The PHP page does not echo out the contents of google.com as suggested in the other answer. It outputs an iframe that points to Google:
<iframe src="http://www.google.com" width="100%" height="100%" frameborder="no"></iframe>
It looks like loadpage.php could be in use to echo out the contents of www.google.com, using file_get_contents.
loadpage.php:
<?php
// Simplified output - should sanitise $_REQUEST params etc first..
echo file_get_contents($_REQUEST['a']);
?>
loadpage is effectively acting as a proxy, allowing your javascript to call pages which are not on your own domain.
As #annakata points out in the comments, the code above is obscenely dangerous as-is. The code is an illustration of the basic idea behind a proxy file - in production, this file would need to make sure that the $_REQUEST parameters were sanitised, e.g. only accept values from a whitelist.
The same origin policy is a security element of javascript that stops you from pulling content from outside your domain on to your page using javascript.
Some sites get around this by calling a proxy page on their own server (loadpage in this instance) which effectively just prints out the content of a target url. As this proxy page is on your server, this by-passes the same origin security issue, and still makes available the content of a page from another domain - here www.google.com
Oops, I somewhat foolishly didn't RTFA, but just the code in the question and hypothesised at what it could be doing. #andynormancx is right in his answer as to what the page linked in the q is actually doing.