I am using php to develop a twitter search api which is able to search twitter, and save posted images from tweets.
It all works fine etc, but for facebook, instead of the image being loaded with the webpage, its loaded after in a response. Using firebug, going to the Net tag, I can see the html source code I need under the response tab for a getphoto(). I am looking to grab an img src from this html text, but
Facebook seems to load the basic stucture, then reload the page with the image on it.
My question is: How can I get this 'response body'?
I have used get_headers() before, but I dont think it will work in this situation, and I have trawlled the net looking for an answer to this, but none have appeared.
Any help would be much appreciated, thx in adv.
Dont think my code will help explaining, but willing to put some up
EDIT:
example facebook url: /https://www.facebook.com/photo.php?pid=1258064&l=acb54aab14&id=110298935669685
that would take you to the page containing the image
This is the image tag:
img class="fbPhotoImage img" id="fbPhotoImage" src="https://fbcdn-sphotos-a.akamaihd.net/hphotos-ak-ash3/522357_398602740172635_110298935669685_1258064_1425533517_n.jpg" alt=""
But this does not show up until the response comes through.
I have a get_header funciton in to expand shortened URL's, due to twitters love for them, and this can get and image from other 3rd party photo sites with multiple shortens/redirects.
Have not used cURL before, is it the best/only way?
Thanks again
instead of the image being loaded with the webpage, its loaded after in a response
I don't know what this means.
I can only guess that the URL you are trying to fetch the html from, which your code is expected to parse to extract an image URL, is actually issuing a redirect.
Use curl for your transfers and tell it to follow redirects - NB this will only work with header redirects - not meta equiv redirects, meta refresh redirects nor javascript location redirects.
(maybe Faceboo0k don't want you to leech their content?)
Related
The url I want to scrape is https://www.tokopedia.com/juraganlim/info
and I just want to get the number of transaction like in this image (inside box is what I need to take):
I really am confused with ajax because I don't know the url which comes or goes.
When I inspect using Firefox it produces so many links.
Please anyone can give me a clue? Or directly the script?
I would like to get the source of one of a page with a flash/html5 video player. If I use the normal curl request, I only get the flash code, but I want to get the html5 video code.
Is it possible to change the header (I already tried to change the x-flash-version in the header, but it doesn't work) or something to say the javascript,which checks, whether Iam using flash, that Iam not using flash?
Thank you in advance,
Noro
Without a URL of the page that you are trying to get the HTML for it is hard to know how the page is deciding whether to render HTML video code or flash code.
It is very likely that JavaScript on the page is changing the DOM at the time to support the Video tags.
If you have a browser that renders with video tags then look at the headers being sent in the POST / GET request and match those headers in your curl call. You can experiment by using the -H flag to the curl command line to send different headers until you get the result you need.
I run ads on my site which are usually YouTube videos. I would like to find out if a user clicked these videos. I'm basically trying to use this data in a statistics plugin that I'm creating.
I've tried searching many different keywords but I can't really find anything.
I would appreciate it if I could be pointed in the right direction. Any ideas, hints or links are welcomed.
As you mention YouTube I doubt you have control over the URL.
But, if you can control the URL embedded in the video, then I would suggest building a simple redirect script on your server. Make sure the URL contains either the redirect URL or a code representing the URL which would be held in your Data Base. Add a logging function to the redirect script and hey presto.
If you have no control over the URL, then off the top of my head maybe you could implement some client side JS to add an overlay to the video which becomes the click-able element and implement the URL redirect script above. Some methods to get the URL from the video: https://www.google.com/search?q=jquery+youtube+URL
I've tried a bunch of techniques to crawl this url (see below), and for some reason the title comes back incorrect. If I look at the source of the page with firebug I can see the correct title tag, however, if I view the page source it's different.
Using several php techniques I get the same result. Digg is able to crawl the page and parse the correct title.
Here's the link: http://lifehacker.com/#!5772420/how-to-make-ios-more-like-android
The correct title is "How to Make Your iPhone (or Other iOS Device) More Like Android"
The parsed title is "Lifehacker, tips and downloads for getting things done"
Is this normal? How are they doing this? Is there a way to get the correct title?
That's because when you request it using PHP (without any JS support) you're getting the main page of lifehacker - which is lifehacker.com.
Lifehacker switched their CMS recently so that all requests go to an initial page and then everything after the hashbang is read by a JS script in the main page to figure out which page needs to be served. You need to modify your program to take this into account
EDIT
Have a gander at these links
http://code.google.com/web/ajaxcrawling/docs/getting-started.html
http://www.tbray.org/ongoing/When/201x/2011/02/09/Hash-Blecch
Found the answer:
http://lifehacker.com/#!5772420/how-to-make-ios-more-like-android
becomes:
http://lifehacker.com/?_escaped_fragment_=5772420/how-to-make-ios-more-like-android
How to hide an iframe url in HTML source code.I have two applications one applications get an url of another application into its iFrame,so that it displays in its source code.I dont want to display another application url in the source code.
I think you would need to set the IFRAME URL via JavaScript. The Javascript could then be Obfuscated, so that the URL would not be in plain text... Please see the following link for the obfuscator:
http://www.javascriptobfuscator.com/Default.aspx
i.e. if it was jQuery...
$("#myiFrame").attr('src','http://www.google.com');
becomes:
var _0xc1cb=["\x73\x72\x63","\x68\x74\x74\x70\x3A\x2F\x2F\x77\x77\x77\x2E\x67\x6F\x6F\x67\x6C\x65\x2E\x63\x6F\x6D","\x61\x74\x74\x72","\x23\x6D\x79\x69\x46\x72\x61\x6D\x65"];$(_0xc1cb[3])[_0xc1cb[2]](_0xc1cb[0],_0xc1cb[1]);
You can't hide it per say, but you can run it through something like TinyURL so that anyone interested would need to go an extra step. Anyway, that's the only thing I can think of. However, if you are displaying that page in a frame, what's the harm in having the URL in the source code? There really isn't a good, foolproof way to prevent someone determined from finding out the location of that iframe page.
You can create a php script which uses curl to call the url through localhost, then use this script as your iframe source.
If you have an issue with relative links and sub-directories, you can put your curl script inside the sub-directory.