I am trying to convert the search results page of the following website into an RSS feed by using feed43.com or Google Apps Script:
https://www.zvg-portal.de/index.php?button=Termine%20suchen&land_abk=by
If I click on search (Suchen), I am forwarded to https://www.zvg-portal.de/index.php?button=Suchen but I can access this search result page neither with feed43.com nor with Google Apps Script. Is there a way to access all search results automatically?
you will have to send a POST request to that page. This basically means, that all the data you filled out in the search form is not transported to the search result page via the URL. therefore, if you copy the URL and paste it anywhere else, all information on what you were actually searching has been lost.
Check if the two tools you mentioned also support POST (instead of GET). if so, you can use chromes or firefox web developer tools (network tab) to show the values that were submitted to the search page and copy them there.
Related
I want to parse data of a website like this: (website is a dictionary - search website)
When I search on website, the website does not refresh, the result will be shown, and the URL does not change.
Now I want to know is it possible to using this or cURL or anything else for parsing the website ?
There are two options:
Check the ajax calls that are triggered in the console and try to fetch these urls instead using the correct method and data.
Disable javascript and see if it still works. If it does, you will see what you need to get.
However, if the site does not want you to get the data, they can prevent it and / or block you if you make too many requests.
how to get usernames and ids from this type of search? All the information is public here in this link. So how can i get a json data of the complete set of users.
https://www.facebook.com/search/108424279189115/residents/present/104057239629661/home-residents/intersect
In the link above, i have used 2 locations,
Hometown:Guntur
current location: Newyork
I have written some script for downloading ID's for targeting purpose. I have used chrome dev tools to grab ID's of users.
JAVASCRIPT in Browser console helped me to achieve this.
Bookmarklet the javascript code attached here and open that link and click the bookmarklet. Thats it, Page scrolls for some time and a file gets downloaded which contains ID's of the user.
Its just a work around as we dont have an API for this.
I have implemented google custom search engine on my website http://www.finalyearondesk.com/
What I want is that when user enters his query into my own defined search bar, then the results [By result I mean the result that is displayed right after just entering the query and hitting enter, for example "ubuntu"] must be displayed in this page... http://www.finalyearondesk.com/search.php . At now the result are just displayed where I have inserted the code that google has given me.
The Google custom search is a JavaScript Ajax widget. The browser talks to Google independent of your site.
If you want results in your search.php page, either put the search-box on the search.php page, or query Google server side, then display the formatted results any way you like. This is very not ideal because Google limits the number of queries you can do.
Long story short, you have to live with the ads and limited placement options google gives you.
You can always attempt to write your own web search engine if you don't like it [/sarcasm]
I'm new to YQL, and just trying to learn how to do some fairly simple tasks.
Let's say I have a list of URLs and I want to get their HTML source as a string in javascript (so I can later insert it to a database via ajax). How would I go about getting this info back in Javascript? Or would I have to do it in PHP? I'm fine with either, really - whatever can work.
Here's the example queries I'd run on their console:
select * from html where url="http://en.wikipedia.org/wiki/Baroque_music"
And the goal is to essentially save the HTML or maybe just the text or something, as a string.
How would I go about doing this? I somewhat understand how the querying works, but not really how to integrate with javascript and/or php (say I have a list of URLs and I want to loop through them, getting the html at each one and saving it somewhere).
Thanks.
You can't read other pages with Javascript due to a built-in security feature in web browsers. It is called the Same origin policy.
The usual method is to scrape the content of these sites from the server using PHP.
There is an other option with javascript called a bookmarklet.
You can add the bookmarklet in your bookmarks bar, and each time you want the content of a site click the bookmark.
A script will be loaded in the host page, it can read the content and post it back to your server.
Oddly enough, the same origin policy, does not prevent you to POST data from this host page to your domain. You need to POST a FORM to an IFRAME that has a source hosted on your domain.
You won't be able to read the response you get back from the POST.
But you can poll with a setInterval making a JSONP call to your domain to know if the POST was successful.
I'm trying to enter a list of items into Google Base via an XML feed so that, when a user searches for one of these items and then clicks the search result link in Google Base (or plain Google), the user is directed to a dynamic Web page on my Web site. I'm assuming that the only way to specify a specific link (either static or dynamic) is through the attribute in the XML feed. Is that correct? So, for example, if my attribute is:
http://www.example.com/product1-info.html
the user will be directed to the product1-info.html page.
But if, instead of a static product page, I want to have the user redirected to a dynamic page that generates search results from my local database (on my Web site) for all products containing the keyword "product1", would I be able to do something like this?:
http://www.example.com/products.php?productID=product1
Finally, and most importantly, is there any way to specify this landing page (or any specific landing page) from a "regular" Google search? Or is it only possible via Google Base and the attribute? In other words, if I put a bunch of stuff into Google Base, if any of it shows up in a regular Google search, is there a way for me to control what parameters get passed to the landing page (and thus, what search is performed on the landing page), or is that out of my control? I hope I explained this correctly. Thanks in advance for any help.
first question: Yes, urls containing a query_string part are allowed.
http://base.google.com/support/bin/answer.py?hl=en&answer=78170 says:XML example:
<link>http://www.example.com/asp/sp.asp?cat=12&id=1030</link>
--
Let me rephrase the second question to see if I understand it correctly (might be completely on the wrong track): E.g. products.php?productID=product1 performs a db-search for the product "FooEx" and products.php?productID=product2 for "BarPlus". Now you want google to show the link .../products.php?productID=product1 but not ....?productId=product2 if someone searched for "FooEx" and google decided that your site is relevant? Then it's the same "problem" we all face with search engines: communicate what each url is relevant for. I.e. e.g. have the appropriate (and only the appropriate) keywords appear in the title/h1 element of the page, avoid linking to the same contents with different urls (e.g. product.php?x=1&productId=1 <-> product.php?productId=1&x1, different urls requesting most probably the exact same contents), submit a sitemap, and so on and on....
edit:
and you can avoid the query-string part all together by using something like mod_rewrite (e.g. the front controller for the zend framework makes use of it) or by parsing the contents of $_SERVER["PATH_INFO"] (this requires the webserver to provide that information), e.g. http://localhoast/test.php/foo/bar -> $_SERVER['PATH_INFO']=='/foo/bar'
Also take a look at the link to this thread: How to redirect a Google search result to a dynamic Web page?, it contains the title of the thread, but SO is perfectly happy with How to redirect a Google search result to a dynamic Web page?, too. The title is "only" additional data for search engines and (even more) the user.
You can do the same:
http://www.example.com/products.php/product1/FooEx <-> http://www.example.com/products.php/product2/BarPlus