Related
Im quite new in web developing, could someone please point me in the right direction of help with the scripts to run a .py script on a web page. Below are the ones that i am using. Do i have to create a html file and a php file??
If so please help me. I have a internal server running on XAMPP with Apache and configured to run CGI, .py scripts.
Work flow:
Upload > button to run the below .py script > download
Upload script(php):
<?php
if(isset($_POST['UploadButton'])){ //check if form was submitted
$target_dir = '/opt/lampp/htdocs/pic-el/Dump/';
$target_file = $target_dir . basename($_FILES["filepath"]["name"]);
$imageFileType = pathinfo($target_file,PATHINFO_EXTENSION);
move_uploaded_file($_FILES["filepath"]["tmp_name"], $target_file);
}
?>
Python script:
#!/usr/bin/env python
import CGIHTTPServer
CGIHTTPServer.test()
import os
import urllib
import cgi
import webcolors
import xlsxwriter
from PIL import Image
filename = "/home/Desktop/tess/test1"
imageNameArray = []
def downloadfile(urlData):
urllib.urlretrieve(urlData[0], urlData[1])
print " image downloaded: " + str(urlData[1])
return
# open file to read
with open("{0}.csv".format(filename), 'r') as csvfile:
# iterate on all lines
i = 0
for line in csvfile:
splitted_line = line.split(',')
# check if we have an image URL
if splitted_line[1] != '' and splitted_line[1] != "\n":
# urllib.urlretrieve(splitted_line[1], '/home/tf_files/images/{0}.jpg'.format (splitted_line[0]))
imageNameArray.append(
(splitted_line[1], '/home/Desktop/tess/images/{0}.jpg'.format(splitted_line[0])))
print "Image added to list for processing for {0}".format(splitted_line[0])
i += 1
else:
print "No result for {0}".format(splitted_line[0])
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
from multiprocessing import Pool
processPool = Pool(5)
processPool.map(downloadfile, imageNameArray)
# Create a workbook and add a worksheet.
workbook = xlsxwriter.Workbook('output1.xlsx')
worksheet = workbook.add_worksheet()
# Start from the first cell. Rows and columns are zero indexed.
row = 0
col = 0
# search for files in 'images' dir
files_dir = os.getcwd() + '/images'
files = os.listdir(files_dir)
def closest_colour(requested_colour):
min_colours = {}
for key, name in webcolors.css3_hex_to_names.items():
r_c, g_c, b_c = webcolors.hex_to_rgb(key)
rd = (r_c - requested_colour[0]) ** 2
gd = (g_c - requested_colour[1]) ** 2
bd = (b_c - requested_colour[2]) ** 2
min_colours[(rd + gd + bd)] = name
return min_colours[min(min_colours.keys())]
def get_colour_name(requested_colour):
try:
closest_name = actual_name = webcolors.rgb_to_name(requested_colour)
except ValueError:
closest_name = closest_colour(requested_colour)
actual_name = None
return actual_name, closest_name
for f in files:
if f.lower().endswith(('.png', '.jpg', '.jpeg')):
image_path = files_dir + '/' + f
im = Image.open(image_path)
n, cl = max(im.getcolors(im.size[0] * im.size[1]))
requested_colour = cl
actual_name, closest_name = get_colour_name(requested_colour)
width = im.size
if width < (500, 500):
worksheet.write(row, 4, "False")
else:
worksheet.write(row, 4, "True")
print image_path
print cl
print width
print "Actual colour name:", actual_name, ", closest colour name:", closest_name
worksheet.write_string(row, 1, image_path)
worksheet.write(row, 3, closest_name)
row += 1
workbook.close()
you can't run .py on a web page, only u can run on the server since Python's a server side programming. But you can run python script from PHP (since u r using XAMPP.)
Example -
<?php
$output = exec('./filename.py');
?>
You don't need to create separate php and html files.
First, when the server goes back to apache2
sudo apt-get install libapache2-mod-wsgi
(It connects apache2 with wsgi.)
Second, you need to create a configuration file and
Move the configuration file to Document_Root.
ex> server.conf
WSGIScriptAlias /test /var/www/wsgi/main.py
MainOption | Address to be connected | Python file location
Third, restart apache2 service.
EXAMPLE_CODES
main.py
server.conf
Info: I am trying to read and write to a file at the same time, but if i have r+ it will only write and if i only have r it will only read, but i need it do both so it can get the info into the document and the info out of it to a cmd window.
Question: Is there i way i can do this and that both will work?
$outputMXA = shell_exec($commandMXA);
$fhMXA = fopen('MXA.txt','r+');
fwrite($fhMXA,$outputMXA);
$MXA = fgets($fhMXA);
Try this :
$outputMXA = shell_exec($commandMXA);
$fhMXA = fopen('MXA.txt','rw');
fwrite($fhMXA,$outputMXA);
$MXA = fgets($fhMXA);
fclose($fhMXA);
Edit:
Or you must separate both actions :
$outputMXA = shell_exec($commandMXA);
$fhMXA = fopen('MXA.txt','w+');
fwrite($fhMXA,$outputMXA);
fclose($fhMXA);
$fhMXA = fopen('MXA.txt','r+');
$MXA = fgets($fhMXA);
fclose($fhMXA);
I currently have an internal site for my company where our customer support users will be uploading files from our clients. Originally, I had planned on using the help of mysql and a protected, shared network folder and used mysql to hold the filename and path. However, we also utilize ImageNow for other processes. Does anyone know if ImageNow works with javascript and php outside of the software itself? I'm new to ImageNow so any advice is appreciated.
logObArray = getDocLogobArray(workingDoc);
for (var i=0; i<logObArray.length; i++)
{
var docObj = logObArray[i];
var filePath = docObj.filePath;
var fileType = docObj.fileType;
var ftToCheck = fileType.toUpperCase();
var phsobID = docObj.phsobId;
//write OSM info to the file, you'll have to add the other code around this but premise is correct and tested
var outRow = filePath;
outRow = outRow + '\n';
if (Clib.fputs(outRow, outCsvFP) >= 0)
{
debug.log('DEBUG', 'Wrote OSM Path [%s] to file successfully.\n', filePath);
stats.inc('Wrote OSM Path to file');
}
}
Unfortunately, ImageNow doesn't let you get at the information it stores outside of Perceptive Software provided tools. Even if you dig directly into the SQL database and look at the filesystem where it is storing the files, you can't get the information out. ImageNow stores the files unencrypted on the filesystem, so that's fine, and it stores the metadata for those images in easy to search tables in the database. However, the path from the metadata to the filesystem it encrypts before it stores it in the database. So if you are trying to go from the metadata to the images, the farthest along you can get is to the encrypted path. Without the decryption key, you can't get to the images.
However, there is a way you can write code to use ImageNow data. You need the add-on Message Agent - which you need to purchase from Perceptive. That opens up interfaces for using web services and SOAP to get at the ImageNow data.
This is the complete solution for this. It gets the root file and subsequent pages. All other solutions I've found do not get anything other than the first page of the scanned document. Change your drawer to your own drawer name (btw). I hope this helps someone. Companies that lock down people's content really make me mad. Just use the intool.exe utility. It's located in the /bin folder of your installation. The call is: intool --cmd run-iscript --file yourfile.js
var curDocId = 0;
var more = true;
// printf("curDocId : %s\n", curDocId );
while (more) {
var rulestext = "[drawer] = 'AR' AND [docID] > '" + curDocId + "'";
var items = INDocManager.getDocumentsByVslQuery(rulestext, 1000, more, "DOCUMENT_ID");
var start = items[0];
var dataDesc = new Array();
var headerDelim = "\03"
var dataDelim = "\02";
for (var line=1; line <= start; line++) {
var temp = items[line].split(headerDelim);
dataDesc[temp[1].toUpperCase()] = new Object();
dataDesc[temp[1].toUpperCase()].idx = line - 1;
dataDesc[temp[1].toUpperCase()].name = temp[1];
dataDesc[temp[1].toUpperCase()].datatype = temp[2];
}
for ( ; line < items.length; line++) {
var doc = new INDocument(items[line].split(dataDelim)[dataDesc["DOCUMENT ID"].idx]);
doc.id = items[line].split(dataDelim)[dataDesc["DOCUMENT ID"].idx];
doc.getInfo();
var masterDocId = doc.id;
var itCounter = 150;
var i = 1;
for( ; i <= itCounter; i++)
{
doc.getInfo();
var logob = INLogicalObject( doc.id, -1, i );
logob.retrieveObject();
if(logob && logob.logobCount > 0)
{
var fp = Clib.fopen("c:\\inowoutput.txt", "a");
var line = masterDocId + ',' + logob.id + ',' + logob.workingName + ',' + logob.filePath + '\n';
Clib.fputs(line, fp);
Clib.fclose(fp);
}
else
{
break;
}
}
curDocId = doc.id;
}
//printf("curDocId : %s\n", curDocId );
}
ImageNow has a scripting language that lets you get past the encrypted file path in the database. The file path is available in an undocumented member of the INLogicalObject. Details below for accessing taken from the following blog article. Accessing the encrypted file path in ImageNow
A search of the ImageNow 6.x specific object documentation will find that the INLogicalObject provides information about the actual files stored in the file system. However, it does not contain any information about the file path. A little closer inspection under the hood of the object reveals that it does have a file path field and the value is not encrypted. It is a member of INLogicalObject. The following very simple example shows finding a single document and displaying its file type and unencrypted file path on the console.
// get a single document
var results = INDocManager.getDocumentsBySqlQuery( "", 1, var more );
if ( results )
{
var doc = results[0];
doc.getInfo();
// get a single page for the document
var logob = INLogicalObject( doc.id, -1, 1 );
logob.retrieveObject();
printf( "file type: %s\n", logob.filetype ); // this member is in the documentation
printf( "unencrypted file path: %s\n", logob.filepath ); // this member is not in the documentation
}
Check out External Messaging Agent (EMA) functionality in ImageNow. It's a free module that is available in every installation.
EMA allows you to receive data from outside the ImageNow system (e.g. from a PHP web form, for example).
To use EMA, you merely would need to have the PHP script insert into the IN_EXTERN_MSG and IN_EXTERN_MSG_PROP tables. One of the properties could be the location of the file that was uploaded via PHP.
You'd then need an iScript to parse the data from the EMA tables and create a document in ImageNow.
I've built a solution like this before, and it works pretty well.
I tried to write a counter for visits of my page, which only counts each IP once per day.
I transcribed a found code to understand how it works and to implement some other features later.
Unfortunately my new code doesn't work.
Code I made from it:
<?php
$timeReset = 86400; //24h
$ipPath = "ipsafe.txt";
$countPath = "counts.txt";
$timePath = "time.txt";
$ip = $REMOTE_ADDR;
$time = time();
$resetTime = $time+$timeReset;
$timeFile = fopen($timePath, "r+");
$timeData = fread($timeFile, filesize($timePath));
//if resetTime is passed -> Reset
if($time >= $timeData) {
//clear ipSafe
$ipFile1 = fopen($ipPath, "w+");
fclose($ipFile);
//set new resetTime
rewind($timeData);
fwrite ($timeData, $resetTime);
}
fclose($timeFile);
//creat IP-List
$ipData = file($ipPath);
while (list ($line_num, $line) = each ($ipData)) {
$digits = explode(",",$line);
}
//if IP was not saved in last timeframe
if(!in_array($ip, $digits))
{
//write new IP
$ipFile2=fopen($ipPath, "a+");
#flock($ipFile1,2);
fwrite($ipFile2, "$ip".",");
#flock($ipFile1,3);
fclose($ipFile2);
//increment counter
$countFile = fopen($countPath,"r+");
$countData = fread($countFile, filesize($countPath);
rewind($countFile);
$countData++;
fwrite($countFile, $countData);
fclose($countFile);
}
?>
with the following questions:
what's wrong with my code?
flock is used to manage the access to the files, but why shall I use different names for the same file?
Thanks for your suggestions.
EDIT:
Sorry for being so unspecific in explaining the problem. After integrating a debugger, I changed "REMOTE_ADDR" to "$_SERVER['REMOTE_ADDR']" so I fixed 1 error. Now i got the following problems:
Warning: fopen(time.txt): failed to open stream: No such file or directory in .../counter.php on line 15
But the file is in the same directory as counter.php - of course the fread and filesize failed too
Warning: fopen(ipsafe.txt): failed to open stream: Permission denied in .../counter.php on line 20
The file doesn't exist yet, but I thought "fopen($ipPath, "w+");" creats it if it doesn't exist.
Fatal error: Call to undefined function fb() in .../counter.php on line 26
Doesn't "fb($timeFile);" send the value to firePHP?
Not doing a full code review (which would result in an almost full rewrite of your code), here are the things I would recommend changing to fix your listed issues:
The following line uses $REMOTE_ADDR, which is undefined.
$ip = $REMOTE_ADDR;
You most likely meant to use $_SERVER['REMOTE_ADDR'] (which you confirmed with your edit), so the line should read:
$ip = $_SERVER['REMOTE_ADDR'];
You state you're receiving an error attempting to open the time.txt file: "No such file or directory". You also say that the file exists in the same directory as counter.php - and yet you're still receiving this error. This can mean one of two things; the first, the file doesn't exist and you're mistaken; the second, your counter.php file is in a different directory than whatever script is executing it - this would mean that the time.txt file would need to be in the directory of the executing script. I would recommend two things for this.
The first, if the file actually exists, use an absolute path for it:
$timePath = '/path/to/your/files/time.txt';
The second, verify if the file exists (in code) and if not, create it. fopen() with r/r+ flags does not create a file if it doesn't exist, so this needs to be manual. Try changing the code to the following:
if (!file_exists($timePath)) touch($timePath);
$timeFile = fopen($timePath, 'r+');
if ($timeFile) {
// the rest of your code
}
You say you're receiving an error when opening the ipsafe.txt file: "Permission denied". This is a fun one; Though the line numbers don't match with your sample code, I believe these are the lines in question:
//clear ipSafe
$ipFile1 = fopen($ipPath, "w+");
fclose($ipFile);
The first issue is that you never use $ipFile1 in your code (except erroneously locking/unlocking it; see below for that info). The second is that you call fclose($ipFile);. There is no handle named $ipFile! You can remove these lines of code!
You attempt to rewind the file pointer in your time.txt file to write the new time, however, you're using the value you read-in from the file $timeData - not the file itself $timeFile. Update the lines to the following:
rewind($timeFile);
fwrite($timeFile, $resetTime);
You read in the file ipsafe.txt, again, to look through the full list of IP addresses - if the current user's IP isn't in it, you add it. This may be a logic issue, but you're only checking the IP addresses of the last line of the file - but you still iterate through every line of the item. Per the comment in the code: //if IP was not saved in last timeframe, I'm going to assume that this is what you're intending to do - but just in case - I would recommend moving the full if-statement block that appears below the loop to inside the loop.
Here's the fun block:
//write new IP
$ipFile2=fopen($ipPath, "a+");
#flock($ipFile1,2);
fwrite($ipFile2, "$ip".",");
#flock($ipFile1,3);
fclose($ipFile2);
You open the file into $ipFile2 (2), then you lock a completely different file $ipFile1 (1), write to 2, unlock 1, and then close 2. I think that you're intentions were to lock/unlock 2, which would make perfect sense here, so I'd recommend changing $ipFile1 to $ipFile2 in those lines.
That should about do it (potentially). After everything, the code should resemble something similar to:
<?php
$timeReset = 86400; //24h
$ipPath = "ipsafe.txt";
$countPath = "counts.txt";
$timePath = "time.txt";
$ip = $_SERVER['REMOTE_ADDR'];
$time = time();
$resetTime = $time+$timeReset;
if (!file_exists($timePath)) touch($timePath);
$timeFile = fopen($timePath, "r+");
if ($timeFile) {
$timeData = fread($timeFile, filesize($timePath));
//if resetTime is passed -> Reset
if($time >= $timeData) {
//set new resetTime
rewind($timeFile);
fwrite ($timeFile, $resetTime);
}
fclose($timeFile);
//creat IP-List
$ipData = file($ipPath);
while (list ($line_num, $line) = each ($ipData)) {
$digits = explode(",",$line);
}
//if IP was not saved in last timeframe
if(!in_array($ip, $digits)) {
//write new IP
$ipFile2=fopen($ipPath, "a+");
#flock($ipFile2, LOCK_EX);
fwrite($ipFile2, "$ip".",");
#flock($ipFile2, LOCK_UN);
fclose($ipFile2);
//increment counter
if (!file_exists($countPath)) touch($countPath);
$countFile = fopen($countPath,"r+");
if ($countFile) {
$countData = fread($countFile, filesize($countPath);
if (empty($countData)) $countData = 0; // initialize the counter to '0' if it hasn't been set
rewind($countFile);
$countData++;
fwrite($countFile, $countData);
fclose($countFile);
}
}
}
?>
Now, I also have one additional logic issue regarding your $resetTime variable. Ideally, incrementing by a single day is to add 24 hours to a given timestamp (24 hours = 86400 seconds). However, daylight savings time will break this. Instead of doing the following:
$resetTime = time() + 86400;
Try going a different route (if you're worried about this, that is):
$resetTime = strtotime('+1 day');
need to pull a ton of info, i.e.
file1:
10948|Book|Type1
file2:
SHA512||0||10948
file3:
0|10948|SHA512|c3884fbd7fc122b5273262b7a0398e63
I'd like to get it into soething like
c3884fbd7fc122b5273262b7a0398e63|SHA512|Type1|Book
I do not have access to an actual database, is there any way to do this? Basically looking for a $id = $file1[0]; if($file3[1] == $id) or something unles sthere's more efficient.
Each CSV file is anywhere from 100k-300k lines. I don't care if it takes a while, I can just let it run on EC2 for a while.
$data = array();
$fh = fopen('file1') or die("Unable to open file1");
while(list($id, $val1, $val2) = fgetcsv($fh, 0, '|')) {
$data[$id]['val1'] = $val1;
$data[$id]['val2'] = $val2;
}
fclose($fh);
$fh = fopen('file2') or die ("Unable to open file2");
while(list($method, null, null, null, $id) = fgetcsv($fh, 0, '|')) {
$data[$id]['method'] = $method;
}
fclose($fh);
$fh = fopen('file3') or die("Unable to open file3");
while(list(null, $id, null, $hash) = fgetcsv($fh, 0, '|')) {
$data[$id]['hash'] = $hash;
}
fclose($fh);
Tedious, but should you get an array with the data you want. Outputting it it as another csv is left as an exercise to the reader (hint: see fputcsv()).
All three files appear to have a common field (i.e. in your example, "10948" was common to all three lines). If you're not worried about using a lot of memory, you could load all three files in different array, setting the common field as the array key and using a foreach loop to reassemble all three.
For example:
$result = array();
// File 1
$fh = fopen('file1');
while ( ($data = fgetcsv($fh, 0, '|')) !== FALSE )
$result[$data[0]] = $data;
fclose($fh);
// File 2
$fh = fopen('file2')
while ( ($data = fgetcsv($fh, 0, '|')) !== FALSE )
$result[$data[5]] = array_merge($result[$data[3]], $data);
fclose($fh);
// File 3
$fh = fopen('file3')
while ( ($data = fgetcsv($fh, 0, '|')) !== FALSE )
$result[$data[1]] = array_merge($result[$data[1]], $data);
fclose($fh);
I would suggest to perform a merged-sort using basic unix tools:
a) sort your .CSV files by the columns common between each file, sort -d" " -K? -K? -K?
b) Using the unix 'join' command to output records common between pairs of .CSV files.
The 'join' command only works with 2 files at a time, so you'll have to 'chain' the results for multiple data sources:
# where 'x' is field number from file A, and 'y' is field number from file B
sort -kx "fileA"
sort -ky "fileB"
join -1x -2y "fileA" "fileB" > file1
sort -kx "fileC"
join -1x -2y "file1" "fileC" > file2
sort -kx "fileD"
join -1x -2y "file2" "fileD" > file3
etc...
This is very fast, and allows you to filter your .CSV files as if an impromptu database join occurred.
If you have to write your own merge-sort in php: (Read Here: Merge Sort )
The easiest implementing to merge-sort of .CSV files is 2-stage: a) unix sort your files, then B) 'merge' all the sources in parallel, reading in a record from each, looking for the case where your value in your common fields match all the other sources (JOIN in database terminology):
rule 1) Skip the record that is less than (<) ALL the other sources.
rule 2) When a record's common value is equal to (==) ALL other sources do you have a match.
rule 3) When a record's common value is equal to (==) is SOME of the other source, you can use 'LEFT-JOIN' logic if desired, otherwise skip that record from all sources.
Pseudo code for a join of multiple files
read 1st record from every data source;
while "record exists from all data sources"; do
for A in each Data-Source ; do
set cntMissMatch=0
for B in each Data-Source; do
if A.field < B.field then
cntMissMatch+=1
end if
end for
if cntMissMatch == count(Data-Sources) then
# found record with lowest values, skip it
read next record in current Data-source;
break; # start over again looking for lowest
else
if cntMissMatch == 0 then
we have a match, process this record;
read in next record from ALL data-sources ;
break; # start over again looking for lowest
else
# we have a partial match, you can choose to have
# 'LEFT-JOIN' logic at this point if you choose,
# where records are spit out even if they do NOT
# match to ALL data-sources.
end if
end if
end for
done
Hope that helps.