I'm working on a project I have for my studies. I'm trying to combine economics and programming together by crawling through some public sites, and using that data to generate some sorts of economic reports.
I have managed to do all the sites I need, besides one. The site is written in C# (.aspx extension). And as far as my knowledge of HTTP requests it uses a POST request. All other sites I crawled were using GET requests (so that wasn't so hard).
The next part might be a bit confusing, but I would really appriciate if someone took a bit of their time and helped me :)
The site is on the following address:
http://www.apr.gov.rs/eng/Registers/FinancialLeasing.aspx
Select the option on the right 2. Search by leasing Acceptor
The address of the page option 2 leads to is: http://reliz.apr.gov.rs/Reliz/Search/GeneralLeasingAgreementLeaseeSearch.aspx but if I use it directly it gives me EXPIRED SESSION error (so I guess that is one of the problems also)
This page now is on my native language (Serbian) so please bear with me just a bit longer.
Here:
U need to select Привредни субјект * (second from 2 radio buttons)
And below that Матични број/Рег. бр., земља (first check button)
And then to enter for example 08010536 in the input field that appears after checking the button.
And then press Enter or the button Пронађи that's on the far right.
After that a table appears at the bottom of the page. I need to get (crawl) data from that table, but I don't know how to make it appear with crawling :/
So far for crawling I have been using cUrl for php.
This is my final project at University, and i would appriciate any help or hint that i get.
Related
Our company has an internal MediaWiki installation with a page called "Our Spamfilter". When typing into the search bar, suggestions show up below it in real time (see the $wgEnableOpenSearchSuggest setting.)
Unfortunately, when typing out spamfilt..., the AJAX search box fails to suggest the article. It only suggests it if you type out our spa..... Basically, what you type has to match the title of the article exactly from left to right.
This is already causing confusion as people assume the page doesn't exist if they type "spamfilter" and get no suggestions.
Is there a reasonably simple way (a setting or plugin) to improve the AJAX search box? When I say "improve", I mean that typing "spam" at least has to suggest the "Our Spamfilter" page.
You could probably use the SpecialSearchResults hook or mess with the JS code, but it is much simpler to just make a redirect from Spamfilter to Our Spamfilter.
I am creating a small blog-like website, and i would like to integrate the option to tag other users on comments or post using the # symbol like facebook or twitter does. I have searched throughout the website and haven't found anything similar nor do i know what this is called.
My question is, what programming language would i need to do this? taking in count that you can press the # symbol anywhere in the comment box, a list of your friends will pop up. i am making my website using PHP and MySQL. I would like to know any tips, tutorials or advice on how to do this.
In order to make a "#" functionality for your website will have to use ajax, PHP,and MySQL. Where ajax makes a call to a PHP file that pulls up and searches through the user's friend list and displays what friend the person is looking for. Also, you will have to make sure that as the user types that friend list keeps is constantly searched and shortened so the user can easily find the person they are looking for.
What I am trying to do is to make something similar to what I see all the time on almost any website. The button that says Share to facebook. The goal for me is to let my guests share the item they are viewing in my store (Ran on prestashop) on their blog I run (Running on Oxwall).
The goal is for the button to not only link to a blog post submission webpage but to already have the subject line filled out with the item they are sharing's name and the blog post to display the information about the item. I would like to try and do all this using PHP. I am not sure how to go about doing it but I am sure that I could pass the value. Please note that I can mod BOTH the blog site and the shop as I run both and want to connect them.
As an extra bonus I am also running a forum using phpbb3 if I could do the same thing but onto that as well I would greatly thank you. I am trying to interlink everything into one big network. I know its not an easy task but I am sure there is an easy way to pass data onto the other site so that this can be done.
Facebook a 2 tools to get items informations in the page, it parses the page looking for the most common tags and it uses OpenGraph.
You can also provide product informations in the head of your page (between head tags), then blog side, you retrieve only the contents and parse it as XML.
I advise you to cache this data to avoid useless connections between websites and awful overloads while parsing.
You can use your own specifications, Open Graph or another standard, but i advise to use a standard.
Now I have my website built on PHP & Mysql. Consider this like a forum. Now when a user posts a reply in my website 1 (ex. www.website1.com), I want to be able to show the starting thread and it's related replies in a sister website of mine. I want to do this in a way that it does not show the rest of the page & other page contents (like logo etc.). I don't think iframe would be a solution because an iframe would embed the whole page and the users visiting my sister website (totally different domain i.e. www.website2.com) would be able to see all the page contents, like logo etc. I want to avoid that. I want to make them see only limited information from website 1 and only the info. that I intend.
I hope that makes sense. In a way, you could say that I am trying to replicate my 1 website, and show only a limited part of it. Users browsing 2nd website can post a reply in the 2nd website and it should automatically be posted & visible to the visitors of the website 1. Users of website 1 should not know that a user of website 2 has posted it. They would feel that some user from website 1 has posted it. Do I have to use 2 separate mysql DB or just 1? I think it would be problematic if I am trying to use different DB. I also feel I might have to face DB connectivity issues as I can connect to only 1 DB at a time.
It's basically like users of website1.com should feel that they are replying to users of website1.com & users of website2.com should feel that they are replying to users of website2.com. (I need it this way to bridge the gap between them). At the same time I want to make the front end of the websites different so that they don't feel that they are replying to some other users outside the domain. These websites would be under my control and I will have access to the source code at any time. If I need to change the source code, these changes are welcome.
Is this really possible?
Thank you in advance.
I'd recommend generating RSS (might be runtime) and using it on the sister website. If RSS is not suitable for your needs, you can create your own XML-based format (or any other :) )
Make two forums which use one database. Both websites would put new messages in the same database.
Make an API for website1, so that website2 can retrieve and post messages on the forum. Website2 would do a HTTP request to website1, which returns XML or JSON, so that website2 can request a list of posts that it can display.
Have both sites connect to the same database and display the content they pull in whatever way is appropriate for the particular site. Each site can only pull the fields relevant to that site.
If the idea is to have two websites with the same data but different presentations, then you would want to simply share a single database between them - assuming they are hosted in the same place and can both get at the database.
You can then just create different PHP pages that both access the same database in the same way but display the data differently.
The best way to do this would be to have a shared library of functions or classes that both sites use to manipulate the data. You would then build a different "presentation layer" on top of that for each site.
I want to build a basically interactive website with these tools. I need to make a navagation bar with a submenu. Ex:
Services
then when mouse is over Services it looks like:
Services
boo
foo
I'm familiar with adding css styles, i'm just not sure about sub menus.
On the php side of things, I want people to select from the products and write their quantities, then I'd like a paypal button at the bottom which when clicked redirects to paypal and lists products ordered and their quantities (basically the paypal checkout page)
The other thing I want is a mailing list. I want people to be able to give their name and email and have a button that sends me an email so I can add them manually.
I would appreciate insight on any or all of these topics, maybe tutorials? I think I can manage the rest of the web site on my own.
Thanks
This is too many questions at once.
Dropdown navigations can be done with HTML+CSS alone. You can also realize them with JavaScript. Since PHP is server-side, you would only generate HTML, CSS or JS from that (if at all)
If you are just getting started with PHP, you might best be off first reading the PHP manual.. At least the chapter named Language Reference. There is also this free book on PHP available.
Sitepoint is also a good resource for beginners for any of the above topics. They have many good introductory articles, a good reference and many tutorials.
As for PayPal, consult the PayPal developer site to get information on how to integrate it.
Consider trying to build the page on your own first and then come back to ask individual questions, whenever you run into problems and googling them did not help.