I have a python script which succesfully posts a file to my localhost webserver running apache in <100ms. Now, I want to do exactly the same with Lua. I have come up with a script that posts that same image to my webserver but it takes a whopping ~24s to complete. The php running on the server receives and stores the file properly but for the Python script, the file comes in the $_FILES array whereas for the Lua script, I have to copy content from the php://input stream - also, looking at both POST requests with wireshark, I can see a 7667 POST from the Python script but not from the Lua, instead only a few TCP SYN & ACK frames. Any idea why my Lua script is missing the actual POST (incl. url) but it still seems to work (but really slow):
Some code is below:
Python
#!/usr/bin/python
import urllib2
import time
from binascii import hexlify, unhexlify
import MultipartPostHandler
fname="test.gif"
host = "localhost"
#host = "semioslive.com"
URI="/test.php"
#URI="/api/gateway.php"
nodemac ="AABBCC"
timestamp = int(time.time())
func="post_img"
url = "http://{0}{1}?f={2}&nodemac={3}&time={4}".format(host, URI,func,nodemac,timestamp)
opener = urllib2.build_opener(MultipartPostHandler.MultipartPostHandler)
data = {"data":open(fname,"rb")}
#r.get_method = lambda: 'PUT'
now = time.time()
response = opener.open(url, data, 120)
retval = response.read()
if "SUCCESS" in retval:
print "SUCCESS"
else:
print "RESPONSE sent at "+retval
print " Now "+str(time.time())
print "Request took "+str(time.time()-now)+"s to return"
Lua
#! /usr/bin/lua
http = require("socket.http")
ltn12 = require("ltn12")
local request_body = ltn12.source.file(io.open("test.gif"))
local response_body = {}
http.request{
url = "`http://localohst/test.php`",
method = "POST",
headers = {
["Content-Type"] = "multipart/form-data",
["Content-Length"] = 7333
},
-- source = ltn12.source.file(io.open("test.gif")),
source = request_body,
sink = ltn12.sink.table(response_body)
}
print(response_body[1]) --response to request
PHP
<?
if (isset($_FILES['data']))
move_uploaded_file($_FILES['data']['tmp_name'],"post(python).gif");
else
copy("php://input","post(lua).gif");
echo "SUCCESS!";
?>
Make sure your Lua script is sending the same HTTP headers. The important part for PHP is that the form with attached file upload is sent as "multipart/form-data", and the file must be properly embedded in the POST body of the HTTP request as a multipart mime message.
I cannot see if your Lua script actually does this, but I think no. Otherwise PHP would be happy.
Related
Different variants of this question litter google, but I am not finding a solution specific to my needs. I am trying to send a JSON string from a client running python to a web server that is using PHP. The JSON string contains the results from a select query.
I have found lots of examples and tutorials on how to generate the JSON in python and how to publish data from the server database to a user via the server's page.
What I am not finding is how to actually transfer the JSON from the Python script to the PHP script. I am using the Arduino YUN with Linino, and have tried a request.post() but get a "module not found" error.
A plain english explanation of how the data hand off should take place between the two devices would help greatly! Any suggestions on a good resource that actually shows an example of what I have described would be great too.
Python (Post):
#!/usr/bin/python
import requests
import json
url = 'http://xxx.xxx.x.xxx/reciever.php'
payload = {"device":"gabriel","data_type":"data","zone":1,"sample":4,"count":0,"time_stamp":"00:00"}
headers = {'content-type': 'application/json'}
response = requests.post(url, data=json.dumps(payload), headers=headers)
PHP (Receiver):
<?php
print_r(json_decode($_POST['payload']));
?>
Maybe there is a totally different way that is better for posting a select query from a client to a server? Client will have dynamic IP's and the server will be web based.
Library requests isn't installed in Python by default. You need to get it first via pip or another package manager:
pip install requests
After that, you can use it.
There is also library, that is built in: urllib2. Example:
import urllib
import urllib2
url = 'http://xxx.xxx.x.xxx/reciever.php'
payload = {"device":"gabriel","data_type":"data","zone":1,"sample":4,"count":0,"time_stamp":"00:00"}
headers = {'content-type': 'application/json'}
data = urllib.urlencode(payload)
req = urllib2.Request(url, data, headers)
response = urllib2.urlopen(req)
the_page = response.read()
I have an internal server that is generating JSON every few minutes. I need to pass this to and external server so I can manipulate the data and then present it in web interface.
Here is my python that send the data to PHP script:
x = json.dumps(data)
print '\nHTTP Response'
headers = {"Content-type": "application/json"}
conn = httplib.HTTPConnection("myurl.com")
conn.request("POST", "/recieve/recieve.php", x, headers)
response = conn.getresponse()
text = response.read()
print "Response status: ",response.status,"\n",text
conn.close()
and Here is my PHP to receive the data
$data = file_get_contents('php://input');
$objJson = json_decode($data);
print_r ($objJson);
My python script return with response status 200 which is good and it returns my JSON. But on the PHP side I want to be able to store this information to manipulate and then have a web client grab at it. However it appears that even if I say
print_r ($objJson);
when I visit the .php page it does not print out my object. I suppose the data is gone because file::input will read only and then end?
Don't use file_get_contents()
Just:
if($_POST){ echo "WE got the data"; }
and print_r will help you with where to go from there, if you are unfamiliar with PHP arrays.
Try to use named parameter containing your JSON data while sending data to PHP and try to use the $_POST superglobal array in PHP (because you obviously connecting via cgi or similar interface not via cli).
You can see all the POST data by printing your $_POST array:
print_r($_POST);
Here is what I did in the end as a quick fix
the python
x = json.dumps(data)
headers = {"Content-type": "application/json"}
conn = httplib.HTTPConnection("myurl.com")
conn.request("POST", "/script.php", x, headers)
response = conn.getresponse()
text = response.read()
print "Response status: ",response.status,"\n",text
conn.close()
then I realized that the PHP was receiving the post data but I was having a hard time manipulating it. So I just sent the JSON to a receiving PHP file that wrote the data as file and then has another JavaScript request grab that.
the php
if($_POST){ echo "WE got the data\n"; } //thanks rm-vanda
$data = file_get_contents('php://input');
$file = 'new.json';
$handle = fopen($file, 'a');
fwrite($handle, $data);
fclose($handle);
exit;
Then from my client just a ajax request for the file and parsed it on the client side.
Eventually I have to rethink this but it works for now. I also have to rewrite over the new.json file each time new data comes in or parse the old data out on the JavaScript success callback. Depends what you want to do.
This might sound trivial for some of you, but I need to be sure...
I simply need to use dropbox for 2 things :
upload image files via php from my web server with the possibility of creating folder (as I would do on a normal web server) or sync folders from my web server to dropbox via rsync;
display these image files in a web page
I have downloaded the api sdk, then run into the 64bit exception error, then invalid redirect-uri...
So I would really appreciate if someone can reply to my 2 questions above and point me to a good example to just do that.
I solved it in another way. Rather than displaying the raw-file I use the API to generate Direct Download Link. This will give me a weblink that I then modify by adding a "raw=1" and subsitute the "dl=0" with "dl=1". This new link than then be used as the source for a normal html image.
As per above suggestions
import dropbox
import json
import httplib, urllib, base64
access_token='your token'
client = dropbox.client.DropboxClient(access_token)
url = client.share('test.jpg',short_url=False)
imageurl = url['url'].replace("dl=0","raw=1")
body = {
"url":imageurl
}
print json.dumps(body)
headers = {
# Request headers
'Content-Type': 'application/json',
'Ocp-Apim-Subscription-Key': '00759a20e705487a91e4db51b80bdfa7',
}
params = urllib.urlencode({
})
try:
conn = httplib.HTTPSConnection('api.projectoxford.ai')
conn.request("POST", "/emotion/v1.0/recognize?%s" % params,json.dumps(body), headers)
response = conn.getresponse()
data = response.read()
print(data)
conn.close()
except Exception as e:
print("[Errno {0}] {1}".format(e.errno, e.strerror))
I want to send some parameters from a python script on my server to a php script on my server using HTTP. Suggestions?
This is pretty easy using urllib:
import urllib
myurl = 'http://localhost/script.php?var1=foo&var2=bar'
# GET is the default action
response = urllib.urlopen(myurl)
# Output from the GET assuming response code was 200
data = response.read()
I have a PHP page that has 1 textbox and when I press on the submit button. My SQL is going to store this product name into my database. My question is; is it possible to send/post the product name using Python script that asks for 1 value and then use my PHP page to send it to my database? Thanks!
Check out the urllib and urllib2 modules.
http://docs.python.org/library/urllib2.html
http://docs.python.org/library/urllib.html
Simply create a Request object with the needed data. Then read the response from your PHP service.
http://docs.python.org/library/urllib2.html#urllib2.Request
When testing, or automating websites using python, I enjoy using twill. Twill is a tool that automatically handles cookies and can read HTML forms and submit them.
For instance, if you had a form on a webpage you could conceivably use the following code:
from twill import commands
commands.go('http://example.com/create_product')
commands.formvalue('formname', 'product', 'new value')
commands.submit()
This would load the form, fill in the value, and submit it.
Yes. urllib2 is a nice Python way to form/send HTTP requests.
I find http://www.voidspace.org.uk/python/articles/urllib2.shtml to be a good source of information about urllib2, which is probably the best tool for the job.
import urllib
import urllib2
url = 'http://www.someserver.com/cgi-bin/register.cgi'
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
values = {'name' : 'Michael Foord',
'location' : 'Northampton',
'language' : 'Python' }
headers = { 'User-Agent' : user_agent }
data = urllib.urlencode(values)
req = urllib2.Request(url, data, headers)
response = urllib2.urlopen(req)
the_page = response.read()
The encoding is actually done using urllib, and this supports HTTP POST. There is also a way to use GET, where you have to pass the data into urlencode.
Don't forget to call read() though, otherwise the request won't be completed.
Personally i prefer to use Requests. As its summary, this is a HTTP library for Python, built for human beings.
To quickly solve problems, rather than study deeply in HTTP, Requests will be a better choice. (Yeah, i mean compared to urllib or socket)
For example,
A python file which send a POST request with userdata:
import requests
userdata = {"firstname": "Smith", "lastname": "Ayasaki", "password": "123456"}
resp = requests.post('http://example.com/test.php', data = userdata)
And the following text.php treating this request:
$firstname = htmlspecialchars($_POST["firstname"]);
$lastname = htmlspecialchars($_POST["lastname"]);
$password = htmlspecialchars($_POST["password"]);
echo "FIRSTNAME: $firstname LASTNAME: $lastname PW: $password";
Finally, the response text (resp.content in python) will be like:
FIRSTNAME: Smith LASTNAME: Ayasaki PW: 123456
A simple Quickstart: http://docs.python-requests.org/en/latest/user/quickstart/
Another answer recommending Requests: Sending data using POST in Python to PHP
Just an addendum to #antileet's procedure, it works similarly if you're trying to do a HTTP POST request with a web-service-like payload, except you just omit the urlencode step; i.e.
import urllib, urllib2
payload = """
<?xml version='1.0'?>
<web_service_request>
<short_order>Spam</short_order>
<short_order>Eggs</short_order>
</web_service_request>
""".strip()
query_string_values = {'test': 1}
uri = 'http://example.com'
# You can still encode values in the query string, even though
# it's a POST request. Nice to have this when your payload is
# a chunk of text.
if query_string_values:
uri = ''.join([uri, '/?', urllib.urlencode(query_string_values)])
req = urllib2.Request(uri, data=payload)
assert req.get_method() == 'POST'
response = urllib2.urlopen(req)
print 'Response:', response.read()