Im looking to pull a status field from a web management interface of a UPS so that the data could be utilised in another web application I am writing. I was wondering if anyone would know a way to go about this as I cant seem to find the information I'm looking for through my web searches. Id also need it the value to refresh or re check. Example of UPS web interface below looking at the online field first off:
This is a very basic example, that I haven't tested (php isnt installed).
You need to look at the source of your control panel, and find out how to identify the elements that contain your desired information.
The code below (hopefully) searches for an element with an id server-status if that element exists it then checks its class attribute to determine the state of the server.
You do not have to use the dom stuff, you could also do it with regex or whatever. So long as you can accurately find the information you need.
You may also need to use cURL or something a little more advanced than file_get_contents() as you will likely require login credentials to view the page in question.
<?php
$html = file_get_contents("http://path.to/your/control.panel");
// you may need to use cURL or something more advanced if you need to provide login credentials
$dom = new DOMDocument;
$dom->loadHTML($html);
$test = $dom->getElementById('server-status');
if ($test == NULL) {
// unable to find element, somethings up!
} else {
if ($test->getAttribute('class') == "online") {
// status element has "online" class, server is online
} else {
// status element does not have "online" class, somethings up!
}
}
?>
Update
Had a quick look at a demo of that management software and it wont be quite as simple as my example as there don't appear to be any helpful element id or class names. Its still do-able though.
Related
The actual questions
How to "map" access restrictions so it can be used from php and javasript?
What kind of method should I use to share access restrictions / rules between php and javascript?
Explanation
I have created a RESTful backend using php which will use context-aware access control to limit data access and modification. For example, person can modify address information that belongs to him and can view (but not modify) address information of all other persons who are in the same groups. And of course, group admin can modify address details of all the persons in that group.
Now, php side is quite "simple" as that is all just a bunch of checks. Javascript side is also quite "simple" as that as well is just a bunch of checks. The real issue here is how to make those checks come from the same place?
Javascript uses checks to show/hide edit/save buttons.
PHP uses checks to make the actual changes.
and yes,
I know this would be much more simpler situation if I ran javascript (NodeJS or the like) on server, but the backend has already been made and changing ways at this point would cause major setbacks.
Maybe someone has already deviced a method to model access checks in "passive" way, then just use some sort of "compiler" to run the actual checks?
Edit:
Im case it helps to mention, the front-end (js) part is built with AngularJS...
Edit2
This is some pseudo-code to clarify what I think I am searching for, but am not at all certain that this is possible in large scale. On the plus side, all access restrictions would be in single place and easy to amend if needed. On the darkside, I would have to write AccessCheck and canAct functions in both languages, or come up with a way to JIT compile some pseudo code to javascript and php :)
AccessRestrictions = {
Address: {
View: [
OWNER, MEMBER_OF_OWNER_PRIMARY_GROUP
],
Edit: [
OWNER, ADMIN_OF_OWNER_PRIMARY_GROUP
]
}
}
AccessCheck = {
OWNER: function(Owner) {
return Session.Person.Id == Owner.Id;
},
MEMBER_OF_OWNER_PRIMARY_GROUP: function(Owner) {
return Session.Person.inGroup(Owner.PrimaryGroup)
}
}
canAct('Owner', 'Address', 'View') {
var result;
AccessRestrictions.Address.View.map(function(role) {
return AccessCheck[role](Owner);
});
}
First things first.
You can't "run JavaScript on the server" because Javascript is always run on the client, at the same way PHP is always run on the server and never on the client.
Next, here's my idea.
Define a small library of functions you need to perform the checks. This can be as simple as a single function that returns a boolean or whatever format for your permissions. Make sure that the returned value is meaningful for both PHP and Javascript (this means, return JSON strings more often than not)
In your main PHP scripts, include the library when you need to check permissions and use the function(s) you defined to determine if the user is allowed.
Your front-end is the one that requires the most updates: when you need to determine user's permission, fire an AJAX request to your server (you may need to write a new script similar to #2 to handle AJAX requests if your current script isn't flexible enough) which will simply reuse your permissions library. Since the return values are in a format that's easily readable to JavaScript, when you get the response you'll be able to check what to show to the user
There are some solutions to this problem. I assume you store session variables, like the name of the authorized user in the PHP's session. Let's assume all you need to share is the $authenticated_user variable. I assume i'ts just a string, but it can also be an array with permissions etc.
If the $authenticated_user is known before loading the AngularJS app you may prepare a small PHP file whish mimics a JS file like this:
config.js.php:
<?php
session_start();
$authenticated_user = $_SESSION['authenticated_user'];
echo "var authenticated_user = '$authenticated_user';";
?>
If you include it in the header of your application it will tell you who is logged in on the server side. The client side will just see this JS code:
var authenticated_user = 'johndoe';
You may also load this file with ajax, or even better JSONP if you wrap it in a function:
<?php
session_start();
$authenticated_user = $_SESSION['authenticated_user'];
echo <<<EOD;
function set_authenticated_user() {
window.authenticated_user = '$authenticated_user';
}
EOD;
?>
UPDATE: The source code is very much different from what Developer Tools shows.
Check out the source: view-source:http://www.machinerytrader.com/list/list.aspx?ETID=1&catid=1002
Is that javascript that needs to be rendered by a browser into html? If so, how can I have php do that process so that I have Html to parse? It's weird that you can use Xpath Checker to return the items I'm looking for (see below), but you cannot access the full html!
(Xpath: //table[contains(#id, 'ctl00_ContentPlaceHolder1') and (contains(#id,"tblContent") or contains(#id,"tblListingHeader"))])
END UPDATE
I need to scrape some information off of this site for work on a regular basis. I am attempting to write some PHP code to scrape this data. I think I have some namespace issues here, having read a number of other posts on SO. I have never encountered namespace problems before and used the approach shown on another SO post (to no avail :().
It appears the xpath query is just not happening for whatever reason. If you have any guesses or solutions as to how to handle this issue, I am open for suggestions.
Also here is the output from my code:
object(DOMXPath)#2 (0) {
}
Debug 1
array(0) {
}
array(0) {
}
I left out the bottom of the code where I var_dump testarray and create and var_dump otherarray. Their output is included above. Obviously the two arrays will be empty if the DOMXPath element has length 0 as well.
$string = 'http://www.machinerytrader.com/list/list.aspx?ETID=1&catid=1002';
$machine_trader = file_get_contents($string);
$xml = new DOMDocument();
$xml->loadHTML($machine_trader);
$xpath = new DOMXPath($xml);
$rootNamespace = $xml->lookupNamespaceUri($xml->namespaceURI);
$xpath->registerNamespace('x', $rootNamespace);
$tableRows = $xpath->query("//x:table[contains(#id, 'ctl00_ContentPlaceHolder1') and (contains(#id,'tblContent') or contains(#id,'tblListingHeader'))]");
var_dump($xpath);
$testarray = array();
$otherarray = array();
foreach ( $tableRows as $row )
{
echo "Debug 1"."\n";
$testarray[] = $row->nodeValue;
}
This is not an XPath issue insofar that the actual content is found from a form post, which you didn't reach yet. JS Source code here does nothing more than authenticate a proper 'user' for the information request, and then send the request via form submission.
At each request, the salt / encryption 'key' is randomized and changes, preventing simple scrapes.
You could rewrite that JavaScript to PHP and then issue two requests, battling the authentication process along the way.
Or, rather than diddle with reverse-engineering this, you could switch your scraping to NodeJS and use something like PhantomJS since it can evaluate javascript but give you programmatic access. Given the complexity of this task, it'd be much simpler to use the right tool.
I am working on my personal site, where I want to store my customers recent search result limited to that particular session.
I am using PHP platform and Javascripts.
Here is an example of what I am exactly looking at :
It stores your previously searched domain name for that particular session so that user can make decision by comparing those results.
Thanks.
EDIT- Well Thanks for all of your answers and suggestions.
But If you have noticed
above example
It looks like some kind of script loading a new content on the same page without refreshing it and keeping previous search content <div> as it is.
How to achieve this using javascripts or some sort of div layer ????
UPDATE START
This example uses page reload. If you want to do it without page reload, you can but you'll have to use AJAX to load new search results. But then, it's not a PHP question. I suggest looking at jquery library, as it makes it easy. Tutorials: http://docs.jquery.com/Tutorials and e.g. this one ( http://docs.jquery.com/Tutorials:Getting_Started_with_jQuery#Rate_me:_Using_Ajax ).
When loading data via AJAX, the page rendering result (in my example search.php) should return only HTML for results part, not whole HTML page. This is generally a first part of my tutorial (without session).
But I really think that AJAX in here is not really needed. Session is more reliable and allows access to your page from older / mobile browsers where not always JS works correctly.
UPDATE END
Ok then. Let's try the simple tutorial then. Sorry if too simple, but I don't know your exact level.
PHP has mechanism called sessions. In reality they are just bytes stored on server. Server knows which session is for each client by reading session cookie from client browser.
Not every page uses sessions (not every page needs it, and session uses server space, even if only temporarily), session is not enabled by default. To turn on session you use command
<?php session_start(); ?>
In most cases this is either run by PHP framework you use, or put near the top of your site. Session is definitely needed if you want to authenticate user somehow. Or in your case :)
To access session you can use superglobal $_SESSION variable (superglobal means that you can access it anywhere). It's an array, so session element will be e.g. $_SESSION['search'] etc.
As example, let's assume that your page looks like that
<html>
...
<form action="search.php" method="post">
Search: <input type="text" name="searchQuery" />
<input type="submit" value="Search" />
</form>
...
</html>
this very form will send user search to file named search.php. It can be the same file where the form resides - in simplest case when you put both your code and HTML in one file. Beginners often use this schema, although it's not advisable as result is a mess and hard to further change.
In search.php then, you'll use similar code:
<?php
if (!empty($_POST['searchQuery'])) //we have a new search
{
$result = do_search($_POST['searchQuery']);
}
?>
Then, somewhere below you'll display your search result ($result variable). do_search() function is your search mechanism, I guess you have it somewhere. You may have it not 'wrapped' in a function, then I advise to create it like that, it's much more useful.
function do_search($searchQuery)
{
...
return $result;
}
mind it, the above code doesn't use sessions yet. Let's add saving previous search results in session. The code may then look like that:
<?php
session_start(); //Starting session
//let's create session variable used to store results
if (!isset($_SESSION['searches']))
$_SESSION['searches'] = array();
if (!empty($_POST['searchQuery'])) //we have a new search
{
if (isset($_SESSION['searches'][$_POST['searchQuery']]) //User already searched on this value, delete previous result from sesion
{
unset($_SESSION['searches'][$_POST['searchQuery']]);
}
$result = do_search($_POST['searchQuery']);
//Let's add new search on the begining of session array to make iterations easier.
$result = array($_POST['searchQuery'] => $result); //convert result to same format as session table
$_SESSION['searches'] = array_merge($result, $_SESSION['searches']);
}
?>
In display you'll now not iterate on $result variable as before, but instead you will do something like
foreach ($_SESSION['searches'] as $query => $result)
{
...//display of single result
}
I haven't tested following code and it's not a full program. Parts to display result and to do actual search are not described but I guess you have them already prepared. Also, this is only one possible approach of countless possibilities. But I hope this helps :)
Possible modification - now I always perform search, even if user already searched on this term. You may want to receive the result from cache without second search. Then the code will look like
if (isset($_SESSION['searches'][$_POST['searchQuery']]) //User already searched on this value
{
$result = $_SESSION['searches'][$_POST['searchQuery']];
unset($_SESSION['searches'][$_POST['searchQuery']]);
}
else
{
$result = do_search($_POST['searchQuery']);
}
For more in-depth information about sessions and some other constructs used in my example I suggest PHP manual
http://pl.php.net/manual/en/book.session.php
and various tutorials over the network. Or you can add a comment here :)
Put this code near the beginning of your script(s):
if (!isset($_SESSION['previous_searches']) || !is_array($_SESSION['previous_searches'])) {
$_SESSION['previous_searches'] = array();
}
[edit]
This code snippet checks if if there is already an array with prevous searches and if not it will be created.
[/edit]
Then when the user hits the search page put this code in the receiving script of the search:
$_SESSION['previous_searches'][] = $_GET['what_ever_your_search_value_might_be'];
[edit]
This code snippet adds the current search value to the and of the array with previous search values
[/edit]
Now you have all previous search values in $_SESSION['previous_searches']
If your website is a web application where you never reload the page nor change the page, you can keep it JavaScript in a global store (declare at top level something like var StoredSearch = []; and use it). If not, then use $_SESSION to store this and AJAX to save/load searches from JavaScript to PHP.
I want to create a way to test different layouts on a page to see which get more conversions.
For example. If I have 2 versions of a page and I send 50% to page A and 50% to page B and see which one converts more sales.
So I am thinking maybe use .htaccess to rewrite half to page A and the other half to page B.
But how can I do that with .htaccess is there a way? do I need to use PHP instead to do this?
Also if there is a better way to do this, or any cautions I should be aware of, please let me know.
Lots of ways to deal with it on your own code. If however you're already using Google Analytics and don't care to use javascript for the test, spare yourself a lot of trouble and look at http://www.google.com/websiteoptimizer/index.html
Update (Reconfine): Google website optimizer no longer exists, this has been replaced with "Google Analytics content experiments" https://developers.google.com/analytics/devguides/platform/experiments-overview
I would do that using php, following way:
After the user got to the default php file, i would store his browser data in a db table, and the active layout identifier (filename, row id, etc...).
Everytime the server gets a request from this user, it shows the page, mapped to him.
But! If you have two independent pages, i would only store how many people visited site one, and site two, and redirect them to page A, and to page B by a 50% division.
If you've got a supported database and are using PHP 5.2 or later, you can use a free split testing library called phpScenario, found at www.phpscenario.org
Then you write more or less something like this:
require_once 'scenario_setup.php'; // you write this
if (Scenario::IsControl('experimentname')) {
// read and output version 1
} else {
// read and output version 2
}
Then when you get to your conversion point (say, sign up):
require_once 'scenario_setup.php'; // same
Scenario::Complete('experimentname');
And to view the stats (probably on your admin page):
require_once 'scenario_setup.php'; // yup
Scenario::RenderXml('experimentname');
I think php will be very useful. For example, you can use rand or push a variable in a file :
$int = file_get_contents('var');
if ($int) {
$int++;
} else{
$int = 1;
}
if (($int % 2) == 0) {
header('Location: url1);
} else {
header('Location: url2);
}
file_put_contents ('var', $int);
With apache, you have to setup a load balancer : http://httpd.apache.org/docs/2.1/mod/mod_proxy_balancer.html
I would use php personally. Then you can save which page layout you chose for them as a session var making it easy to load that layout on each page refresh. You would probably also want to save into the database with their username (if they login) and if they visit later show them the same layout.
I'm using the Zend framework and the openid selector from http://code.google.com/p/openid-selector/ - however I find I can't login using sites like Google and Yahoo as they use direct identity based login system whereby one is just redirected to a url as opposed to entering a unique url of their own for authentication.
I've checked out many options and hacks but none of them seem to work. How can i get this to work here btw - how is it implemented at stack overflow? I could really use all the help here guys..
Edit
Well the issue here is that from what I have noticed is that the Zend OpenID class doesn't support OpenID 2.0 the thing is that a typical open ID providor gives you a unique url such as your-name.openid-providor.com or openid-providor.com/your-name and the Zend OpenId class just parses through that url and then redirects you to the providor website where upon authentication you are redirected back.
In the case of Yahoo and google - you don't enter a unique url instead you are redirected to the providors login site and upon login and authentication you are redirected back - so basically whats happeining is that the zend_openID object when it parses to tell who the providor is it fails to tell from the general url itself. Like when you click on teh Google link it redirects you to https://www.google.com/accounts/o8/id
Its more an issue with the zend openid object here and there isn't any help on zend related forums - so I was wondering if someone had already hacked or had an alteration I could make to the class to accomplish this. Sorry if I'm missing something but I'm kinda new to this and programming with open ID and have just started to get my feet wet.
Thanks for the follow up - I did check into RPX a while back and they do have a php class but I wasnt able to check it out plus I really just want to for now get the code selector used as on stackoverflow to work with Yahoo and Google authentication. There has to be some kind of way to tweak the parsing which the Zend OpenID class uses as it runs a series of regular expression checks to make a discovery.
Little late to the game but I was able to get this working with some hacks I found around the interwebs.
First. Yahoo. To get Yahoo working all I had to do was change the JavaScript to use me.yahoo.com instead of just yahoo.com and it worked perfectly with the version of the Zend Framework I'm using. Unfortunately Google still wasn't, so some hacking was in order.
All of these changes go in Zend/OpenId/Consumer.php
First, in the _discovery method add the following on the series of preg_match checks that starts at around line 740.
} else if (preg_match('/<URI>([^<]+)<\/URI>/i', $response, $r)) {
$version = 2.0;
$server = $r[1];
I added this right before the return false; statement that's in the else {} block.
Second, in the _checkId method you'll need to add 3 new blocks (I haven't dug around enough to know what causes each of these three cases to be called, so I covered all to be on the safe side.
Inside the $version <= 2.0 block, you'll find an if/else if/else block. In the first if statement ($this->_session !== null) add this to the end:
if ($server == 'https://www.google.com/accounts/o8/ud') {
$this->_session->identity = 'http://specs.openid.net/auth/2.0/identifier_select';
$this->_session->claimed_id = 'http://specs.openid.net/auth/2.0/identifier_select';
}
In the else if (defined('SID') block add this to the end:
if ($server == 'https://www.google.com/accounts/o8/ud') {
$_SESSION['zend_openid']['identity'] = 'http://specs.openid.net/auth/2.0/identifier_select';
$_SESSION['zend_openid']['claimed_id'] = 'http://specs.openid.net/auth/2.0/identifier_select';
}
And then after the else block (so outside the if/else if/else block all together, but still inside the $version <= 2.0 block) add this:
if ($server == 'https://www.google.com/accounts/o8/ud') {
$params['openid.identity'] = 'http://specs.openid.net/auth/2.0/identifier_select';
$params['openid.claimed_id'] = 'http://specs.openid.net/auth/2.0/identifier_select';
}
Link to the bug in Zend Framework Issue Tracker
I need to use Google's OpenID stuff, and I tried Steven's code and couldn't get it to work as-is. I've made some modifications.
The _discovery change method is still the same:
Zend/OpenId/Consumer.php, line 765, add:
} else if (preg_match('/<URI>([^<]+)<\/URI>/i', $response, $r)) {
$version = 2.0;
$server = $r[1];
The rest is different, though:
Zend/OpenId/Consumer.php, line 859 (after making the above change), add:
if (stristr($server, 'https://www.google.com/') !== false) {
$id = 'http://specs.openid.net/auth/2.0/identifier_select';
$claimedId = 'http://specs.openid.net/auth/2.0/identifier_select';
}
This is right before:
$params['openid.identity'] = $id;
$params['openid.claimed_id'] = $claimedId;
And to get it to return the ID, once authorized:
Zend/Auth/Adapter/OpenId.php, line 278:
if(isset($_REQUEST['openid_identity']))
{
$this->_id = $_REQUEST['openid_identity'];
$id = $this->_id;
}
This is right before:
return new Zend_Auth_Result(
Zend_Auth_Result::SUCCESS,
$id,
array("Authentication successful"));
Note that I have not thoroughly tested this code. The code below is even more shakey.
I have spent more time and I've gotten it to work with my Google Apps domain with the following changes, in addition to the above:
Zend/OpenId/Consumer.php, line 734
$discovery_url = $id;
if(strpos($discovery_url, '/', strpos($discovery_url, '//')+2) !== false) {
$discovery_url = substr($discovery_url, 0, strpos($discovery_url, '/', strpos($discovery_url, '//')+2));
}
$discovery_url .= '/.well-known/host-meta';
$response = $this->_httpRequest($discovery_url, 'GET', array(), $status);
if ($status === 200 && is_string($response)) {
if (preg_match('/Link: <([^><]+)>/i', $response, $r)) {
$id = $r[1];
}
}
This is right after:
/* TODO: OpenID 2.0 (7.3) XRI and Yadis discovery */
I believe that was the only change I had to make. I'm pretty sure there's supposed to be some checking involved with the above for security reasons, but I haven't looked far enough into it to see what they would be.
Going over all the advice provided - I've decided to ditch using the zend_openid class [ sorry about that zend ] and instead I've switched to using JanRains OpenID library. Its taken a few hours to get it up and running with my project but atleast its working like a breeze. Had to make a lot of hacking and a bit of code spill over to get it working but its worth it.
I couldn't use any of Zend adapters with Zend-Auth to settle this new code library in as the library did the authentication on its own. SO I hacked and made a generic adapter that just returned a filled zend_result set to the Auth object thus I authenticate using my library and merely store the result in the Auth object pulling a bit of a fast one one the Zend-Auth object rather than have to rewrite my code again.
The library is available at http://openidenabled.com/php-openid/
Thanks for all the help guys.
I'm dealing with similar issues. I'm planning on using RPX now with Zend Framework. Maybe I'll write an adapter. Just to let you know.
Info: 'RPS now' provides an all-in-one interface and UI for user registration with
facebook
Google
Yahoo
mySpaceID
Windows LiveID
OpenID
aol
I'm pretty sure that Yahoo only works with OpenID 2.0. If you want to support Yahoo users, you're going to have to upgrade to a library with 2.0 support. That's going to be a matter of more than tweaking some parsing.
Did you check out the manual -- Zend_OpenId_Consumer basics? Check out 38.2.2 on that page and let me know if this helps, because it should.
Specifically, I don't know if Google offers OpenID. I know that Yahoo worked because I've tried it a while back.
Thanks for the information. I started by using JanRain's library, but I have problems with getting Simple Registration to work: I have not succeeded in getting any data that way. And there is no documentation on using Attribute Exchange. :(
So, I found and was trying Zend/OpenId, but had the same problem as you: no Yahoo!, Google and who knows what else support. Reading this, it seems I'll have to get back to JanRain; RPX is not an option in my case as it's a third party service.