I am trying to create a bot to automatically scrape some data from a website. I am trying to use the Snoopy PHP class for this.
The problem I am having is that I cannot submit the login form page. I am passing the username and password in the field, but "nothing happens" - I don't get logged in and an error is not returned (such as invalid password).
I have tried using cURL directly (without the Snoopy class), but that doesn't help.
I feel that I am not passing the form variables correctly. I would appreciate it if someone could point me in the right direction.
The code I am using is:
$snoopy = new Snoopy;
$submit_url = "";
$submit_vars['ctl00$cphMainContent$ctl00$UsernameTextBox'] = "";
$submit_vars['ctl00$cphMainContent$ctl00$PasswordTextBox'] = "";
if($snoopy->submit($submit_url,$submit_vars)) {
while(list($key,$val) = each($snoopy->headers)) {
echo $key.": ".$val."<br>\n";
}
echo "<p>\n";
echo "<PRE>".htmlspecialchars($snoopy->results)."</PRE>\n";
} else {
echo "error fetching document: ".$snoopy->error."\n";
}
Related
I'm currently putting together a small web-based GUI to generate kickstart-scripts. I got a confirmation page that's sending the relevant data via POST to the PHP-page where the actual shell script is called to build the iso. So far it's working, but the page seems to execute the script before it outputs anything else (for example, the 'echo' I put in at the beginning of the page ...), and I'm absolutely clueless why. Would anyone care to enlighten me?
Here's the code to the PHP-page that's executing the shell script ...
echo 'Generating your ISO; this might take a while...';
sleep(20);
if (!isset($_POST['auth'])) {
$ad = 'N';
}
else {
$ad = 'Y';
}
if (!isset($_POST['oracle'])) {
$oracle = 'N';
}
else {
$oracle = 'Y';
}
if ((!isset($_POST['ip'])) or (!isset($_POST['hostname'])) or (!isset($_POST['rhsel'])) or (!isset($_POST['submit'])) or (!isset($_POST['gw'])) or (!isset($_POST['nm']))) {
die('Please use the correct form !');
}
if (isset($_POST['ip'])) {
$ip = trim($_POST['ip']);
}
if (isset($_POST['gw'])) {
$gw = trim($_POST['gw']);
}
if (isset($_POST['nm'])) {
$nm = trim($_POST['nm']);
}
if (isset($_POST['hostname'])) {
$hostname = trim($_POST['hostname']);
}
if (isset($_POST['rhsel'])) {
$rhsel = $_POST['rhsel'];
}
passthru("/usr/bin/sudo /data/skripte/webconfig.sh $rhsel $oracle $ad $ip $gw $nm $hostname 2>&1");
PHP scripts accessed via a browser are request-response, meaning all processing is done on the server prior to headers and content being sent to the client. This means you will not get a continually updating output like you would see on the command line. There is no way around this. Sorry.
I am working on a custom script to automatically send out invites and reminders. I have everything working fine up until a point. My function to send invites looks like this:
function sendInvites($iSurveyID) {
$oSurvey = Survey::model()->findByPk($iSurveyID);
if (!isset($oSurvey)) {
die("could not load survey");
}
if(!tableExists("{{tokens_$iSurveyID}}")) {
die("survey has no tokens or something");
}
$SQLemailstatuscondition = "emailstatus = 'OK'";
$SQLremindercountcondition = '';
$SQLreminderdelaycondition = '';
$iMaxEmails = (int)Yii::app()->getConfig("maxemails");
$iMaxReminders = 1;
if(!is_null($iMaxReminders)) {
$SQLremindercountcondition = "remindercount < " . $iMaxReminders;
}
$oTokens = Tokens_dynamic::model($iSurveyID);
$aResultTokens = $oTokens->findUninvited(false, $iMaxEmails, true, $SQLemailstatuscondition, $SQLremindercountcondition, $SQLreminderdelaycondition);
if (empty($aResultTokens)) {
die("No tokens to send invites to");
}
$aResult = emailTokens($iSurveyID, $aResultTokens, 'invite');
}
I also have a simple little file that starts up Yii:
Yii::createApplication('LSYii_Application', APPPATH . 'config/config' . EXT);
Yii::app()->loadHelper('admin/token');
Yii::app()->loadHelper('common');
Everything works as expected up until I actually try to send emails to the tokens. I've tracked the problem down to the following, on of the functions called by emailTokens has this in it:
$clang = Yii::app()->lang;
$aBasicTokenFields=array('firstname'=>array(
'description'=>$clang->gT('First name'),
'mandatory'=>'N',
'showregister'=>'Y'
),
The Yii::app()->lang part seems to be causing issues because then php is unable to call the gT method. However, when LimeSurvey is running "properly" this never happens. I can't even seem to find where "lang" is in the LimeSurvey source.
What can I do to make it work?
Why do you make it so hard on yourself and not use the RemoteControl2 API ?
See http://manual.limesurvey.org/wiki/RemoteControl_2_API#invite_participants
On that page you will also find a PHP example script.
maybe
Yii::import('application.libraries.Limesurvey_lang');
$clang = new Limesurvey_lang($oTokens->language);
I got two php pages:
client.php and server.php
server.php is on my web server and what it does is open my amazon product page and get price data and serialize it and return it to client.php.
Now the problem I have is that server.php is getting the data, but when I return it and do echo after using unserialize(), it shows nothing. But if I do echo in server.php, it shows me all the data.
Why is this happening? Can anyone help me please?
This the code I have used:
client.php
$url = "http://www.myurl.com/iec/Server.php?asin=$asin&platform=$platform_variant";
$azn_data = file_get_contents($url);
$azn_data = unserialize($azn_data);
echo "\nReturned Data = $azn_data\n";
server.php
if(isset($_GET["asin"]))
{
$asin = $_GET["asin"];
$platform = $_GET["platform"];
echo "\nASIN = $asin\nPlatform = $platform";
//Below line gets all serialize price data for my product
$serialized_data = amazon_data_chooser($asin, $platform);
return($serialized_data);
}
else
{
echo "Warning: No Data Found!";
}
On server.php , you need to replace your following line:
return($serialized_data);
for this one:
echo $serialized_data;
because client.php reads the output of server.php, return is used to pass information from functions to caller code.
UPDATE:
Apart from the fixes above, you're hitting a bug in unserialize() function that presents with some special combination of characters, which your data seems to have, the solution is to workaround the bug by base64() encoding the data prior to passing it to serialize() , like this:
In client.php:
$azn_data = unserialize(base64_decode($azn_data));
In server.php:
echo base64_encode($serialized_data);
Source for this fix here .
You are not serializing your data on server side so there is nothing to deserialize on client side.
return(serialize($serialized_data));
Edit:
if(isset($_GET["asin"]))
{
$asin = $_GET["asin"];
$platform = $_GET["platform"];
echo "\nASIN = $asin\nPlatform = $platform";
//Below line gets all serialize price data for my product
$serialized_data = amazon_data_chooser($asin, $platform);
die(serialize($serialized_data));
}
else
{
echo "Warning: No Data Found!";
}
well i'm writing a php code to edit tags and data inside those tags but i'm having big trouble getting my head around the thing.
basically i have an xml file similar to this but bigger
<users>
<user1>
<password></password>
</user1>
</users>
and the php code i'm using to try and change the user1 tag is this
function mod_user() {
// Get global Variables
global $access_level;
// Pull the data from the form
$entered_new_username = $_POST['mod_user_new_username'];
$entered_pass = $_POST['mod_user_new_password'];
$entered_confirm_pass = $_POST['mod_user_confirm_new_password'];
$entered_new_roll = $_POST['mod_user_new_roll'];
$entered_new_access_level = $_POST['mod_user_new_access_level'];
// Grab the old username from the last page as well so we know who we are looking for
$current_username = $_POST['mod_user_old_username'];
// !!-------- First thing is first. we need to run checks to make sure that this operation can be completed ----------------!!
// Check to see if the user exist. we just use the normal phaser since we are only reading and it's much easier to make loop through
$xml = simplexml_load_file('../users/users.xml');
// read the xml file find the user to be modified
foreach ($xml->children() as $xml_user_get)
{
$xml_user = ($xml_user_get->getName());
if ($xml_user == $entered_new_username){
// Set array to send data back
//$a = array ("error"=>103, "entered_user"=>$new_user, "entered_roll"=>$new_roll, "entered_access"=>$new_access_level);
// Add to session to be sent back to other page
// $_SESSION['add_error'] = $a;
die("Username Already exist - Pass");
// header('location: ../admin.php?page=usermanage&task=adduser');
}
}
// Check the passwords and make sure they match
if ($entered_pass == $entered_confirm_pass) {
// Encrypt the new password and unset the old password variables so they don't stay in memory un-encrytped
$new_password = hash('sha512', $entered_pass);
unset ($entered_pass, $entered_confirm_pass, $_POST['mod_user_new_password'], $_POST['mod_user_confirm_pass']);
}
else {
die("passwords did not match - Pass");
}
if ($entered_new_access_level != "") {
if ($entered_new_access_level < $access_level){
die("Access level is not sufficiant to grant access - Pass");
}
}
// Now to load up the xml file and commit changes.
$doc = new DOMDocument;
$doc->formatOutput = true;
$doc->perserveWhiteSpace = false;
$doc->load('../users/users.xml');
$old_user = $doc->getElementsByTagName('users')->item(0)->getElementsByTagName($current_username)->item(0);
// For initial debugging - to be deleted
if ($old_user == $current_username)
echo "old username found and matches";
// Check the variables to see if there is something to change in the data.
if ($entered_new_username != "") {
$xml_old_user = $doc->getElementsByTagName('users')->item(0)->getElementsByTagName($current_username)->item(0)->replaceChild($entered_new_username, $old_user);
echo "Username is now: " . $current_username;
}
if ($new_pass != "") {
$current_password = $doc->getElementsByTagName($current_user)->item(0)->getElementsByTagName('password')->item(0)->nodeValue;
//$replace_password = $doc
}
}
when run with just the username entered for change i get this error
Catchable fatal error: Argument 1 passed to DOMNode::replaceChild() must be an instance of DOMNode, string given, called in E:\xampp\htdocs\CGS-Intranet\admin\html\useraction.php on line 252 and defined in E:\xampp\htdocs\CGS-Intranet\admin\html\useraction.php on line 201
could someone explain to me how to do this or show me how they'd do it.. it might make a little sense to me to see how it's done :s
thanks
$entered_new_username is a string so you'll need to wrap it with a DOM object, via something like$doc->createElement()
$xml_old_user = $doc->getElementsByTagName('users')->item(0)->getElementsByTagName($current_username)->item(0)->replaceChild($doc->createElement($entered_new_username), $old_user);
This may not be quite right, but hopefully it points you in the correct direction.
alright got it writing and replacing the node that i want but i have ran into other issues i have to work out (IE: it's replacing the whole tree rather then just changing the node name)
anyway the code i used is
// For initial debugging - to be deleted
if ($old_user == $current_username)
echo "old username found and matches";
// Check the variables to see if there is something to change in the data.
if ($entered_new_username != "") {
try {
$new_node_name = $doc->createElement($entered_new_username);
$old_user->parentNode->replaceChild($new_node_name, $old_user);
}
catch (DOMException $e) {
echo $e;
}
echo "Username is now: " . $current_username;
}
if ($new_pass != "") {
$current_password = $doc->getElementsByTagName($current_user)->item(0)->getElementsByTagName('password')->item(0)->nodeValue;
//$replace_password = $doc
}
$doc->save('../users/users.xml');
I am using gigya php sdk, It works well I am able to post to user's wall or profile using socialise.setStatus method but I am have problems when I try to use the publishuseraction method. I get an error Invalid request signature.
$method = "socialize.publishUserAction";
$request = new GSRequest($apiKey,$secretKey,$method);
$userAction = new GSDictionary("{\"title\":\"This is my title\", \"userMessage\":\"This is a user message\", \"description\":\"This is a description\", \"linkBack\":\"http://google.com\", \"mediaItems\":[{\"src\":\"http://www.f2h.co.il/logo.jpg\", \"href\":\"http://www.f2h.co.il\",\"type\":\"image\"}]}");
$request->setParam("userAction", $userAction);
$request->setParam("uid", $userUID);
$response = $request->send();
if($response->getErrorCode()==0){
echo "Success";
} else {
echo ("Error: " . $response->getErrorMessage());
}
UPDATED AFTER USING $response->getLog()
apiMethod=socialize.publishUserAction
apiKey=correct_api_key
params={"userAction":{},uid":"MY_UID","format":"json","httpStatusCodes":"false"}
URL=http://socialize.gigya.com/socialize.publishUserAction postData=uid=urlencoded(MY_UID)&format=json&httpStatusCodes=false&apiKey=correct_api_key×tamp=1296229876&nonce=1.29622987636E%2B12&sig=HEdzy%2BzxetV8QvTDjdsdMWh0%2Fz8%3D
server= web504
I used both put method
$userAction = new GSDictionary();
$userAction->put("title", "This is my title");
$userAction->put("userMessage", "This is my user message");
$userAction->put("description", "This is my description");
$userAction->put("linkBack", "http://google.com");
$mediaItems = array();
$mediaItems[0] = new GSDictionary("{\"src\":\"http://www.f2h.co.il/logo.jpg\", \"href\":\"http://www.f2h.co.il\",\"type\":\"image\"}");
& JSON method
$userAction = new GSDictionary("{\"title\":\"This is my title\", \"userMessage\":\"This is a user message\", \"description\":\"This is a description\",
\"linkBack\":\"http://google.com\", \"mediaItems\":[ {\"src\":\"http://www.f2h.co.il/logo.jpg\", \"href\":\"http://www.f2h.co.il\",\"type\":\"image\"}]}");
I get the same error. And User action is empty using both methods. I appreciate any help.
Thanks.
Try calling getLog() for a bit more information about the error:
if($response->getErrorCode()==0){
echo "Success";
} else {
echo ("Error: " . $response->getErrorMessage());
print "<pre>";
print_r($response->getLog());
print "</pre>";
}
Also, are you able to run getUserInfo w/ the SDK? This is a simple test that also requires a signature. Use the example on the PHP SDK page (about 1/2 way down).