For my school, we have to do these "Advisory Lessons" that tell you about College, etc. After completing the lesson, I am wondering if I would be able to replicate the same process using a set of requests from a PHP script with cURL.
I went through the lesson again, this time with Firebug on and an HTTP Analyzer.
Much to my surprise, the only GET requests were sent out during the entire lesson.
In case your curious, here is what the "Lesson" window looks like. It's sort of powerpoint-type thing where you read the slide and then some slides have questions on them. At the end, there is a quiz and if you don't pass it, the lesson doesn't count.
My question is this: If I were to setup a PHP/cURL script that logged into my account, and then made every single one of those requests, would the lesson be counted as complete?
Now obviously it's impossible for you guys to know how their server works and such...
I guess what I am saying is, is there any hidden content or fields that you can pass through a GET request? It just doesn't seem like the lesson window is passing enough info to the server for it to know if the lesson was complete or not.
Thanks so much for any advice and tips on my project!
EDIT: Here is my official test run (please don't do it too many times):
As many of you hinted, it did not work....but I am still not completely sure why.
Like you say, we can't speak to the details of their server, but it is possible to do these kinds of things with GET requests only because servers can use cookies and store state (associated with these cookies) on the server.
This gives the appearance, probably, of passing extra hidden information to the server.
You can research cookies, and even that jsessionid thing that is appearing in their URLs. That BTW tips you off that they are using at least some Java. :)
The lesson application may very well be storing data in a session or some other persistant data store server-side and using a token from your browser (usually a cookie or a GET parameter) to look up that data when needed.
Its a kinda complicated task. With only cURL you can't emulate execution of javascript code, AJAX requests etc
I am not sure what you are trying to do. For one HTTP is stateless protocol meaning the server gets request and gives a response to that particular request (that might be GET, POST or whatever and might have some request parameters). Statefullness in system usage is usually achieved by server creating a session and setting up a cookie on client side to pass session id in later requests. Session id is used to recognize the client and track his session. Everything you send during request is plain text. What response you get most likely will depend on session state and will also be a plain text. There is nothing hidden on a client side about client side. You just don't get to know what information server keeps in session and how requests are processed based on that and information you give during requests.
Related
I am new to the world of programming and I have learnt enough about basic CRUD-type web applications using HTML-AJAX-PHP-MySQL. I have been learning to code as a hobby and as a result have only been using a WAMP/XAMP setup (localhost). I now want to venture into using a VPS and learning to set it up and eventually open up a new project for public use.
I notice that whenever I send form data to my PHP file using AJAX or even a regular POST, if I open the Chrome debugger, and go to "Network", I can see the data being sent, and also to which backend PHP file it is sending the data to.
If a user can see this, can they intercept this data, modify it, and send it to the same backend PHP file? If they create their own simple HTML page and send the POST data to my PHP backend file, will it work?
If so, how can I avoid this? I have been reading up on using HTTPS but I am still confused. Would using HTTPS mean I would have to alter my code in any way?
The browser is obviously going to know what data it is sending, and it is going to show it in the debugger. HTTPS encrypts that data in transit and the remote server will decrypt it upon receipt; i.e. it protects against any 3rd parties in the middle being able to read or manipulate the data.
This may come as a shock to you (or perhaps not), but communication with your server happens exclusively over HTTP(S). That is a simple text protocol. Anyone can send arbitrary HTTP requests to your server at any time from anywhere. HTTPS encrypted or not. If you're concerned about somebody manipulating the data being sent through the browsers debugger tools… your concerns are entirely misdirected. There are many simpler ways to send any arbitrary crafted HTTP request to your server without even going to your site.
Your server can only rely on the data it receives and must strictly validate the given data on its own merits. Trying to lock down the client side in any way is futile.
This is even simpler than that.
Whether you are using GET or POST to transmit parameters, the HTTP request is sent to your server by the user's client, whether it's a web browser, telnet or anything else. The user can know what these POST parameters are simply because it's the user who sends them - regardless of the user's personal involvement in the process.
You are taking the problem from the wrong end.
One of the most important rules of programming is :
Never trust user entries is a basic rule of programming ! Users can and will make mistakes, and some of them will try to damage you or steal from you.
Welcome into the club.
Therefore, you must not allow your code to perform any operation that could damage you in any way if the POST or GET parameters you receive aren't what you expect, be it by mistake or from malicious intents. If your code, by the way it's designed, renders you vulnerable to harm simply by sending specific POST values to one of your pages, then your design is at fault and you should redo it taking that problematic into account.
That problematic being a major issue while designing programs, you will find plenty of documentation, tutorials and tips regarding how to prevent your code to turn against you.
Don't worry, that's not that hard to handle, and the fact that you came up with that concern by yourself show how good you are at figuring things out and how commited you are to produce good code, there is no reason why you should fail.
Feel free to post another question if you are stuck regarding a particular matter while taking on your security update.
HTTPS encrypts in-transit, so won't address this issue.
You cannot trust anything client-side. Any data sent via a webform can be set to whatever the client wants. They don't even have to intercept it. They can just modify the HTML on the page.
There is no way around this. You can, and should, do client side validation. But, since this is typically just JavaScript, it can be modified/disabled.
Therefore, you must validate all data server side when it is received. Digits should be digits, strip any backslashes or invalid special characters, etc.
Everyone can send whatever they want to your application. HTTPS just means that they can't see and manipulate what others send to your application. But you always have to work under the assumption that what is sent to your application as POST, GET, COOKIE or whatever is evil.
In HTTPS, the TLS channel is established before and HTTP data is transfered so, from that point of view, there is no difference between GET and POST requests.
It is encrypted but that is only supposed to protects against mitm attacks.
your php backend has no idea where the data it receives comes from which is why you have to assume any data it receives comes straight from a hacker.
Since you can't protect against unsavoury data being sent you have to ensure that you handle all data received safely. Some steps to take involve ensuring that any files uploaded can't be executed (i.e. if someone uploads a php file instead of an image), ensuring that data received never directly interacts with the database (i.e. https://xkcd.com/327/), & ensuring you don't trust someone just because they say they are logged in as a user.
To protect further do some research into whatever you are doing with the received post data and look up the best practices for whatever it is.
I have read some similar questions but all of them about answering to a human client behind a browser which is not my case. I need to answer another server using sessions, so no browser and cookie!
What I'm working on: I'm creating a telegram bot (using telegram webHook mode and PHP) which gets a user's message (handed by telegram server) and checks if it is the right answer. In each of the states of Yes/No, bot will do the proper action. So there is a database here to read the right answer from and lots of recieved messages to be checked. The whole code would be executed for each message.
Question: How can I use sessions to respond to all of the requests comming from a server (telegram server)? Since there is a lot of requests, so retrieving information from MySQL database each time whould cause low performance. Is there any other better solution for that?
AS fusion3k told you, sessions are stored in files and are not designed for handle a large amount of data. In the same way, session needs cookie for working.
If yout information does'nt change between request, you can use memcached for caching.
When I print $data = file_get_contents('php://input'),It printed lots of messy code. I want to know from the php server which android device has uploaded a certain file to the server. Thank you so much!
That isn't messy code, it's details about the request which you do not understand.
Go learn how the HTTP protocol works, you can't tackle this problem if you don't understand what you're working with.
You can only find out what a user-agent provides. If they are not sending a request header over HTTP with a device version number, then no you will not get that.
Typically the OS is included in the User-agent string. This is likely to be the only place you'll find anything close to what you're looking for.
If you want to identify the specific user, the best you can do (by design of many of the systems between you and the device), is implement "sessions" using HTTP cookies. If the requests are coming from a browser, once you set a cookie, every subsequent request from that individual user will have the same cookie with it.
The general pattern we use is to look for an existing cookie, and if there isn't one (i.e. new user): create a unique random string, store it somewhere on the server associated with this "user", and then send it to the user as a cookie.
The PHP session features can greatly assist you in this.
Keep in mind you can only track a user in this anonymous way. You cannot get an actual device ID from the device unless you also wrote the app on the device which is sending the request. and this is good.
I have a script that uses JSONP to make cross domain ajax calls. This works great but my question is, is there a way to prevent other sites from accessing and getting data from these URL's? I basically would like to make a list of sites that are allowed and only return data if they are in the list. I am using PHP and figure I might be able to use "HTTP_REFERER" but have read that some browsers will not send this info.... ??? Any ideas?
Thanks!
There really is no effective solution. If your JSON is accessible through the browser, then it is equally accessible to other sites. To the web server a request originating from a browser or another server are virtually indistinguishable aside from the headers. Like ILMV commented, referrers (and other headers) can be falsified. They are after all, self-reported.
Security is never perfect. A sufficiently determined person can overcome any security measures in place, but the goal of security is to create a high enough deterrent that laypeople and or most people would be dissuaded from putting the time and resources necessary to compromise the security.
With that thought in mind, you can create a barrier of entry high enough that other sites would probably not bother making requests with the barriers of entry put into place. You can generate single use tokens that are required to grab the json data. Once a token is used to grab the json data, the token is then subsequently invalidated. In order to retrieve a token, the web page must be requested with a token embedded within the page in javascript that is then put into the ajax call for the json data. Combine this with time-expiring tokens, and sufficient obfuscation in the javascript and you've created a high enough barrier.
Just remember, this isn't impossible to circumvent. Another website could extract the token out of the javascript, and or intercept the ajax call and hijack the data at multiple points.
Do you have access to the servers/sites that you would like to give access to the JSONP?
What you could do, although not ideal is to add a record to a db of the IP on the page load that is allowed to view the JSONP, then on the jsonp load, check if that record exists. Perhaps have an expiry on the record if appropriate.
e.g.
http://mysite.com/some_page/ - user loads page, add their IP to the database of allowed users
http://anothersite.com/anotherpage - as above, add to database
load JSONP, check the IP exists in the database.
After one hour delete the record from the db, so another page load would be required for example
Although this could quite easily be worked around if the scraper (or other sites) managed to work out what method you are using to allow users to view the JSONP, they'd only have to hit the page first.
How about using a cookie that holds a token used with every jsonp request?
Depending on the setup you can also use a variable if you don't want to use cookies.
Working with importScript form the Web Worker is quite the same as jsonp.
Make a double check like theAlexPoon said. Main-script to web worker, web worker to sever and back with security query. If the web worker answer to the main script without to be asked or with the wrong token, its better to forward your website to the nirvana. If the server is asked with the wrong token don't answer. Cookies will not be send with an importScript request, because document is not available at web worker level. Always send security relevant cookies with a post request.
But there are still a lot of risks. The man in the middle knows how.
I'm certain you can do this with htaccess -
Ensure your headers are sending "HTTP_REFERER" - I don't know any browser that wont send it if you tell it to. (if you're still worried, fall back gracefully)
Then use htaccess to allow/deny access from the right referer.
# deny all except those indicated here
order deny,allow
deny from all
allow from .*domain\.com.*
I'm building a web bot to login into some of my accounts on websites but one of the url's are sending cookie from javascript and curl is unable to store them. Any suggestion?
You could parse the javascript file using whatever language your using and look for the document.cookie statement. You could then use this data to set the cookie manually in curl. (CURLOPT_COOKIE)
It wouldn't exactly be the best idea if your hoping for this to work with a number of sites, but since you state that you know the site you'll need to load its a possibility as you'll have an idea of how the Javascript will look.
If you have months of free time on your hands, you could compile webkit and its javascript engine and modify the cookie-setting functionality so that it exports the cookies to stdout (and then grab them with PHP's exec). Good luckkk with that though. Considering you're asking this question with relation to cURL, I don't think this is quite up your alley...
I'd sort of go with Kewley's answer if you're desperate though. You should be able to reverse-engineer the javascript and see the logic behind how the web application sets its cookies. If it authenticates and returns the login-result with XHR, watch what's sent and received by the browser (with Firebug). Add breakpoints on the document.cookie lines and observe what cookies are being set (and what they're being set to). Once you know the precise logic behind authentication, you perform the necessary behind-the-scenes requests necessary to snag a session on the site with cURL.
Curl doesn't parse Javascript, therefore the cookies wont ever be set.