I'm looking for an efficient and secure way to store small amount of user data like a line of text, for an indefinite period of time.
The scenario: One client 'A' sends some data to another client 'B' , asynchronously via a server 'S'. For simplicity, consider the data to be a single line of text. Now, this data would be delivered to client 'B' by server 'S' , only when B asks for it. So until that time Server 'S' has to store this data.
For a simple demo, I implemented it very crudely as follows: Client 'A' does a POST request with XMLHttpRequest, sending the text to 'S' as follows
xhr.open("POST",url+"?sentText=text"); //url points to a php file on S
xhr.send();
The PHP code then saves the received text to a file as follows:
<?php
$selected = $_GET["sentText"];
$writefile = "text.txt";
$fh = fopen($writefile,'w');
fwrite($fh,$selected);
fclose($fh);
?>
Later, after any amount of time, B asks for the text in this file,using BufferedInputStream as follows:
URL fileurl = new URL("url/text.txt");
BufferedReader in = new BufferedReader(new InputStreamReader(fileurl.openStream()));
while((tmpReceived = in.readLine())!=null){
received = tmpReceived;
}
This demo works properly, but this is just a demo. However, if this were to be a full scale real world application with millions of users, how would I implement the following:The part where server S stores the text. Obviously, I can't save the text as plain text in a file, as it would breach my users' privacy. And it would also be ineffecient when there are millions of users. How do commercial services like Dropbox achieve this?
To keep things simple, I just need to know how to do this for text, then I can extrapolate to other types possibly.
Thanks a lot!
Related
I want download a report that I'm receiving every day on my inbox.
This email contains a link to a platform (where I have to be logged-in on the platform to open the link correctly, otherwise I will be redirected to login page) and finally there is another link that downloads the report to my computer.
My question is: It's possible to make a PHP script that can do this automatically?. All the process to read the email and identify the link it's easy, I want some guidelines to continue working.
Thanks!
This may be harder than you anticipate and PHP may not be the best language to achieve what you are after as it can not run on top of Outlook or any other desktop based email client. To do this you would have to run the PHP code from the command line, and have it set to regularly intercept all your emails and check to see if the email is from a specific user, and if it is parse the email for the two necessary links. Once you have the links you can have the code establish a cURL connection to the login form and pass the username and password, ensuring to pass any cookies defined from the authenticated session, and then establish a new cURL link to the download link to download the file to the local file system, after which you can send a new email to yourself attaching the file as a native attachment, dependant on file size.
Information on sending cookies with cURL can be found at How can I send cookies using PHP curl in addition to CURLOPT_COOKIEFILE?
The net result though is that this would have to run in the background continuously and would have to be set to regularly connect to your email server and check all emails for the existence of emails from the automated email sender.
An easier solution would be to find out if the automated tool can be setup to simply send the file as an email attachment so that you don't have to run the excess code. In addition any time the initial email structure changes or the download or login links change you would have to update the code to deal with the associated changes.
Main point is that PHP isn't the most ideal solution for what you are trying to do and what you are trying to do, in any language, is going to be a complex task to achieve.
When you already have the email link you could login to the platform using a curl request and afterwards make the same call again. the second time you'd be logged in and curl would download your report.
You could use the cURL library.
How does your platform identifies you ? If you can use cookies, look for
CURLOPT_COOKIE
curl_setopt($ch, CURLOPT_COOKIE, session_name() . '=' . session_id());
i have the a similar issue, i want to open on googlesheets a series of reports from my email, managed to do it when its attached evenif is a zip, but when its only a link o cannot do it, heres the script i use, maybe someone could upgraded to open links and import the csv
function importCSV() {
// Change the report name Report
var threads = GmailApp.search("subject:XXXXXX has:attachment from:XXXXX#google.com newer_than:1d");
var message = threads[0].getMessages()[threads[0].getMessages().length-1];
var allAttachment = message.getAttachments();
var attachment = allAttachment[0];
var contentType = attachment.getContentType();
var name = attachment.getName();
if (attachment.getContentType() === "text/csv") {
// Change the sheet name, where you want the data to appear
var sheetActive = SpreadsheetApp.openById("1slVepI8s2Ekdiha0u4NpqFLZ4NnJrhyc8in9hYNzJA0");
var sheet = sheetActive.getSheetByName("Youtube");
var csvData = Utilities.parseCsv(attachment.getDataAsString(), ",");
// This clears the document from previous data
sheet.clearContents().clearFormats();
// Import the CSV file to the spreadsheet
sheet.getRange(1, 1, csvData.length, csvData[0].length).setValues(csvData);
}
else if (attachment.getContentType() === "application/zip") {
attachment.setContentTypeFromExtension();
// Unzip Attachment Change the sheet name, where you want the data to appear
var sheetActive = SpreadsheetApp.openById("1slVepI8s2Ekdiha0u4NpqFLZ4NnJrhyc8in9hYNzJA0");
var sheet = sheetActive.getSheetByName("Youtube");
try {
var unzip = Utilities.unzip(attachment)[0];
var csv = unzip.getDataAsString();
var csvData2 = Utilities.parseCsv(csv, ",");
// This clears the document from previous data
sheet.clearContents().clearFormats();
// Import the CSV file to the spreadsheet
sheet.getRange(1, 1, csvData2.length, csvData2[0].length).setValues(csvData2);
} catch (e) {
// Logs an ERROR message.
console.error('myFunction() yielded an error: ' + e);
}
}}
I've got a Minecraft Software written in C# that I want to send a heartbeat to my site. I've got the way to send the beat already written.
if (Server.Uri == null) return;
string uri = "http://GemsCraft.comli.com/Heartbeat.php";
// create a request
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
// turn request string into a byte stream
byte[] postBytes = Encoding.ASCII.GetBytes(string.Format("ServerName={0}&Url={1}&Players={2}&MaxPlayers={3}&Uptime={4}",
Uri.EscapeDataString(ConfigKey.ServerName.GetString()),
Server.Uri,
Server.Players.Length,
ConfigKey.MaxPlayers.GetInt(),
DateTime.UtcNow.Subtract(Server.StartTime).TotalMinutes));
request.ContentType = "application/x-www-form-urlencoded";
request.CachePolicy = new System.Net.Cache.RequestCachePolicy(System.Net.Cache.RequestCacheLevel.NoCacheNoStore);
request.ContentLength = postBytes.Length;
request.Timeout = 5000;
Stream requestStream = request.GetRequestStream();
// send it
requestStream.Write(postBytes, 0, postBytes.Length);
requestStream.Flush();
requestStream.Close();
/* try
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Logger.LogToConsole(new StreamReader(response.GetResponseStream()).ReadToEnd());
Logger.LogToConsole(response.StatusCode + "\n");
}
catch (Exception ex)
{
Logger.LogToConsole("" + ex);
}*/
}
Now, I want to be able to retrieve the heartbeat in PHP, upload it to the SQL database, and then display each user's server in a table that will be displayed on the webpage
How do I do this?
portforwardpodcast's answer isn't very well-suited for your purposes, here's a process for you to ponder
Server accesses the following page: heartbeat.php?port=25565&maxplayers=25&players=2&name=Cheese_Pizza_Palace
Your PHP script will then do the following...
Go through each value, making sure they're all the types you want them to be (integers/strings)
Connect to the database
Update the server in the database if it already exists, create it if it doesn't
Return some value so the server knows that it completed successfully.
And to display the servers
Fetch all 'active' servers
Loop through them and display each one.
Things you'll need to figure out:
How to determine uptime
How to determine "active" servers
How to update/create MySQL entries
How to (properly) connect to a database. I would suggest using PDO since you're using PHP. It's a bit difficult to learn, but it's much more secure than writing the queries directly.
How to loop through all the GET variables.
Good hunting!
I would create a simple php page accept a get variable. something like www.site.com/beat.php?lasttime=123456&serverid=1 where the number us the unix timestamp. Then you need to re-work your c# to do a simple get request on a website. Finally your php should insert into a mysql table with a column for id, timestamp, server_id etc.
First you need to pull the data from the request. The $_REQUEST variable in php is nice because it works for both GET and POST:
http://php.net/manual/en/reserved.variables.request.php
Start out by var_dump or echo the fields you want. Once you can get the needed data into variables you are done with the first part. For the next part you need to create a database and table in MySQL. The best tool for this is phpmyadmin. If you have a host like godaddy (or some others) you can get at this from the control panel. If not you may need to install upload the phpmyadmin files yourself. It's a pretty simple tool to use:
http://www.youtube.com/watch?v=xxQSFHADUIY
Once your database has the correct columns, you need to insert the data from your php file. This page should help:
http://www.w3schools.com/php/php_mysql_insert.asp
This code works, but I just hacked it together with my limited knowledge of PHP and I'm sure there's a more elegant and efficient way to go about it. If you'd be so kind as to point out how I can improve, that would be great!
So I have a CSV file, structured like so:
Code Class Value Status Date Created Date Redeemed
========================================================================
a51f3g45 gold 50 valid 2012-08-20
4f6a2984 silver 200 redeemed 2012-08-23 2012-08-27
gf3eb54b gold 150 valid 2012-08-30
etc...
The user fills out a form to change the Class, Value, and Status fields of a given line. I cobbled together the following code to replace the old values with the new ones:
$file = 'codes.csv';
$old_csv_string = file_get_contents($file);
preg_match('/('.$_POST['code'].',.*,.*,.*,.*,.*)\n/',$old_csv_string,$matches);
$old_row = $matches[1];
preg_match('/'.$_POST['code'].',(.*,.*,.*),.*,.*\n/',$old_csv_string,$matches_part);
$old_row_part = $matches_part[1];
$new_row_part = $_POST['class'].",".$_POST['value'].",".$_POST['status'];
$new_row = str_replace($old_row_part,$new_row_part,$old_row);
$new_csv_string = str_replace($old_row,$new_row,$old_csv_string);
file_put_contents($file,$new_csv_string);
So can I do better than 10 lines of code? Any advice would be greatly appreciated :)
Note: I tried using fgetcsv, but I couldn't figure out how to find the unique code within a 2D array, then replace its siblings.
Why are you doing this ?
I think you should store the data in a SQL table.
Each time user update data, do it in the table.
If you want the CSV to be downloadable at any moment. Use a .htaccess to redirect your.csv to csv_generator.php only if your.csv does not exist.
csv_generator.php will regenerate the whole csv if it does not exist, save it on hard drive for later use, and send it with correct mime/type in header (so it's transparent for user). User don't see he is requesting a php page.
Then you need to delete the csv on hard drive each time someone update the data (so it will be regenerated on next request)
I think this is the way to have an always ready to download csv online.
Do you know google doc does this ? Users can change data in a spreadsheet wich is available to download as a csv from a url (you need to publish this spreadsheet as a csv file).
Try using split like that for each line:
list($code, $class, $value, $status, $created, $redeemed) = split(",", $line, 6) ;
Thus you will have each field in separate variable.
Of course you need to take care of the first row in case you don't want to copy header.
I am doing this flash banners for multiple clients and one major request is to have some sort of counter so they know how many times the banner has been clicked.
I know how to do it in ActionScript 3.0, I make a simple var:int and i increase it +1 when a click is made on the banner. What do I do with the value of this var(say its 121) where do I store it online so its safe and can be changed by multiple flash banners(as3).
But how do I save this information so next time when the banner is loaded(on diffrent webpages) the number of clicks is whatever it was last time it was loaded.
Should I look into PHP for that ? I have no clue how to do this... some examples, tutorials, whatever works... would be much appreciated.(I am a designer, not programmer...please dont speak php-ish, or you know... :D)
I've googled a bit, and found some help, but i am still confused, and much of it its not AS3, I'm thinking maybe stuff has evolved a bit since the stuff that I found(2008)...
Thank you very much.
You'd have to store (and fetch) the value somewhere - either in the DB, in a text-file, ...
I'd go search for a tutorial on PHP+MySQL. If you don't like PHP-ish, you're probably better of finding another solution though :p
Example tutorial: http://www.freewebmasterhelp.com/tutorials/phpmysql
You need to store the data you want be retrievable/update-able from multiple clients, to be stored on a server.
You can use any server side language with a database.
Server Languages : PHP, ASP.net, JSP, ColdFusion
Database : MySQL, MSSQL, PostgreSQL, Oracle, DB2 etc..
Use whatever combination you are comfortable with.
In general:
You have a web app that increments the counter in the database
call the page using URLLoader from your AS3 banner.
Database
counter_table
-------------
counter INT
PHP File
$db = mysql_connect('localhost', 'mysql_user', 'mysql_password');
mysql_select_db('database_name');
mysql_query('UPDATE counter_table SET counter = counter + 1');
AS3 Banner
// url request with your php page address
var scriptRequest:URLRequest = new URLRequest("http://www.example.com/script.php");
// loader
var scriptLoader:URLLoader = new URLLoader();
// load page to trigger database update
scriptLoader.load(scriptRequest);
Do you also want to retrieve the value of the number of clicks in Banner ?
Easy solution (really not the best :) You should use one of the other answers.. anyways, make a php file that reads txt file containing the count of visits.. and in your flashbanner just call the php file. It'll add one hit per call..
PHP:
<?php
/**
* Create an empty text file called counterlog.txt and
* upload to the same directory as the page you want to
* count hits for.
*
*
* #Flavius Frantz: YOU DONT NEED THESE:
* Add this line of code on your page:
* <?php include "text_file_hit_counter.php"; ?>
*/
// Open the file for reading
$fp = fopen("counterlog.txt", "r");
// Get the existing count
$count = fread($fp, 1024);
// Close the file
fclose($fp);
// Add 1 to the existing count
$count = $count + 1;
// Display the number of hits
// If you don't want to display it, comment out this line
//echo "<p>Page views:" . $count . "</p>";
// Reopen the file and erase the contents
$fp = fopen("counterlog.txt", "w");
// Write the new count to the file
fwrite($fp, $count);
// Close the file
fclose($fp);
?>
Example code from: (google: php counter file) http://www.totallyphp.co.uk/text-file-hit-counter
Code is not tested, but looks ok. I only commented just a little..
I need to accomplish the following and need help with #2 below
My site has a page with form and the submitted form data needs to be written to a database on my site.
After it is written to the database, the same data submitted on the form needs to be sent to a page that processes it on another site so as if the form submission came from a page on that other site. The page that processes it on the other site is a php page.
It's a bit unclear, but my guess is that you're trying to do a 'form post' to the other .php page after your data is written to the database.
You can more information from this wonderful Scott Hanselman article, but here is the summary:
public static string HttpPost(string URI, string Parameters)
{
System.Net.WebRequest req = System.Net.WebRequest.Create(URI);
req.Proxy = new System.Net.WebProxy(ProxyString, true);
//Add these, as we're doing a POST
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
//We need to count how many bytes we're sending. Post'ed Faked Forms should be name=value&
byte [] bytes = System.Text.Encoding.ASCII.GetBytes(Parameters);
req.ContentLength = bytes.Length;
System.IO.Stream os = req.GetRequestStream ();
os.Write (bytes, 0, bytes.Length); //Push it out there
os.Close ();
System.Net.WebResponse resp = req.GetResponse();
if (resp== null) return null;
System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream());
return sr.ReadToEnd().Trim();
}
The ideal solution to your problem is that you create a web service on the php site and your asp.net code calls the web service. http://en.wikipedia.org/wiki/Web_service
Creating a web service in PHP: http://www.xml.com/pub/a/ws/2004/03/24/phpws.html
Calling a web service in ASP.Net: http://www.codeproject.com/KB/webservices/WebServiceConsumer.aspx
Alternatively you could create a http request from your asp.net to the php site posting all the form elements to the php site.
Here is an example: http://www.netomatix.com/httppostdata.aspx
NB: You are almost guaranteed to run into problems with the second approach in the medium to long term, I don't recommend it unless you don't have control over the php site.