Using elasticsearch-php on Laravel 5 through Shift31/laravel-elasticsearch (provides facade for elasticsearch-php). Hosting an ES instance on Facetflow.
config/elasticsearch.php:
return array(
'hosts' => array(
'https://accessUsername#ilovelasagna.east-us.azr.facetflow.io:443'
)
);
WelcomeController.php:
public function elasticsearch()
{
$searchParams['index'] = 'your_index';
$searchParams['size'] = 50;
$searchParams['body']['query']['query_string']['query'] = 'foofield:barstring';
return Es::search($searchParams);
// throws Authentication401Exception exception
}
This returns Authentication401Exception. My guess is the username is not passed to the Facetflow server, but I have not found a way to check if this is true.
The problem does not seem to be in the server's settings, because when I use plain curl, I get a 200 response:
public function justCurl()
{
$ch = curl_init();
// Now set some options (most are optional)
// Set URL to download
curl_setopt($ch, CURLOPT_URL, "https://accessUsername#ilovelasagna.east-us.azr.facetflow.io:443");
// Download the given URL, and return output
$output = curl_exec($ch);
// Close the cURL resource, and free system resources
curl_close($ch);
return $output;
// returns 200 response
}
How do I fix this issue?
Facetflow's blazing-fast customer support pointed out the issue here.
Host was supposed to be marked as
'https://accessUsername:#ilovelasagna.east-us.azr.facetflow.io:443'
Note the colon after accessUsername.
Related
I am posting data(including a media file (.wav)) from my app to an API with curl. When submitting my data, i check for the data including the mediafile submitted in my API. From the response i get from my API, see below
Response
{"status":"success","media":false,"data":{"message":"Media Campaign","recipient":["34505140704"],
"file":{"name":"\/Users\/path\/to\/folder\/public\/Voice\/aaaah.wav","mime":null,"postname":null}}}true
In the response, the file is being retrieved as well but when i check for the file using $request->hasFile('file') or $request->file('file'), I get false and null respectively.
Can someone let me know why this is happening in my code please ?
Controller
public function test()
{
$file_name_with_full_path = '/Users/path/to/folder/public/Voice/aaaah.wav';
if(function_exists('curl_file_create'))
{
$cFile = curl_file_create($file_name_with_full_path);
}
else
{
$cFile = '#' . realpath($file_name_with_full_path);
}
$post = array('message' => 'Media Campaign', 'recipient' => ['34505140704'],'file' => $cFile);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_POST,1);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post));
$result=curl_exec ($ch);
curl_close ($ch);
}
APIController
public function campaign(Request $request)
{
if (($request->get('message')) {
return response()->json([
'status' => 'success',
'data' => $request->all()
]);
}
}
To be honest, I'd use Guzzle to hide the details of cURL requests in PHP. The way PHP's cURL extension handles file transfers changed a couple of years ago, which broke a lot of legacy code at the company I was working for. By using a third-party wrapper like Guzzle, you can let the Guzzle developers worry about changes in the underlying extension - all you need to do is keep your Guzzle package up to date.
PHP - Why Use Guzzle Instead of cURL?
UPDATE
I was able to at least get the client side code to work with authentication using firebase-simple-login.js and auth.login('anonymous'). Server side (ie "write") still does not work.
ORIGINAL QUESTION
I am creating an app with Firebase integration and simply need to secure my Firebase data. Being able to delete everything if you know my Firebase URL and where to look without any authentication is obviously less than ideal.
I am not trying to log my users in (or at least not in the traditional sense), I just want to make sure that there is SOME SORT of authentication going on when I read and write from my Firebase data. I have spent hours on this and cannot seem to make it work (so much for "easy").
First, my Firebase security rules - simple
{
"rules": {
".read" : "auth != null",
".write": "auth != null"
}
}
I am pushing to Firebase from server side code and reading the results client side. This is a simple polling app - poll responses are pushed to Firebase in the following format: clients\ClientA\polls\POLLID(random)\RandomDataID(from Firebase)\Response Data. I am using firebase/php-jwt to generate my server side JWT:
<?php
class generateFirebaseToken {
public static function generate(array $data = array())
{
$key = 'FIREBASE SECRET KEY';
$token = array(
'iss' => 'https://example.com',
'iat' => time()
);
// Add additional data to token
$token = array_merge($token, $data);
$jwt = JWT::encode($token, $key);
return $jwt;
}
}
I am then pushing data to Firebase with the following code. There are several variables from the user's session included. This uses a class I wrote that prepares a CURL request for Firebase. Everything works fine if I remove the auth != null from the Firebase rules. Otherwise, nothin':
$fbData = array(
'name' => "{$this->user->first_name} {$this->user->last_name}",
'answer' => $fbAnswer,
'gravatar' => Gravatar::src($this->user->email)
);
$token = generateFirebaseToken::generate();
$fb = new fireBase("clients/{$this->client->nickname}/polls/{$poll->uniquid}.json?auth=$token", $fbData);
$fb->execute('POST');
Source of $fb->execute()
public function execute($method)
{
$data_string = json_encode($this->data);
$ch = curl_init($this->root . $this->path); // http://myapp.firebaseio.com/
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json',
'Content-Length: ' . strlen($data_string))
);
$result = curl_exec($ch);
return $result;
}
Client side also does not work. I retrieve a JWT by performing a $.getJSON request to my server side code and pass it on to Firebase. It seems to authenticate correctly but then I receive the error FIREBASE WARNING: on() or once() for /clients/exampleclient/polls/FdV4RM9LHcoB7U7W failed: Error: permission_denied: Client doesn't have permission to access the desired data.. Here is the client side code:
$.getJSON('/secure/jwt-token', function(json) {
jwtToken = json.token;
launchFirebase(jwtToken);
});
function launchFirebase(token)
{
var fb = new Firebase('https://myapp.firebaseio.com/clients/exampleclient/polls/' + pollID);
fb.auth(token, function(e) {
if(e) {
alert('Authentication error : ' + e);
} else {
alert('Authenticated'); // Alert shows, so I assume authenticated
fb.on('child_added', function(snapshot) {
// do stuff
// Error occurs here.
});
}
});
}
I am assuming I am missing something simple here, perhaps I do not understand how to use JWT.. whatever the case, any help would be appreciated. Thanks!
HOURS of wasted effort but I finally found the problem, hopefully this helps someone else new to using Firebase. Also goes to show that if you're working on the same issue for hours, take a break and the fog will begin to clear.
Okay, the problem was -- of course -- simple. I was using the firebase/php-jwt library. I had everything right EXCEPT for the fact that I did not add 'd' to the token data array - THIS IS WHERE auth COMES FROM. So, my security rules was checking for auth, missing because I did not add d to my token.
Here is the fixed code:
$key = 'YOUR-SECURITY-KEY';
$token = array(
'iss' => 'https://EXAMPLE.com',
'iat' => time(),
'd' => array(
'foo' => 'bar' // THIS GIVES YOU THE AUTH VARIABLE!!!!!
)
);
// Add additional data to token
$token = array_merge($token, $data);
$jwt = JWT::encode($token, $key);
return $jwt;
If you write code for people to plug into a service, as in this case, PLEASE provide better documentation. It seems too often that such things are just entirely omitted, perhaps because the developer should just know intrinsically? I finally discovered the answer reading the docs for making a token WITHOUT the use of a helper library (like this one).
Using Symfony2, I need to access an external API based on HTTPS.
How can I call an external URI and manage the response to "play" with it. For example, to render a success or a failure message?
I am thinking in something like (note that performRequest is a completely invented method):
$response = $this -> performRequest("www.someapi.com?param1=A¶m2=B");
if ($response -> getError() == 0){
// Do something good
}else{
// Do something too bad
}
I have been reading about Buzz and other clients. But I guess that Symfony2 should be able to do it by its own.
I'd suggest using CURL:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'www.someapi.com?param1=A¶m2=B');
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-type: application/json')); // Assuming you're requesting JSON
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
// If using JSON...
$data = json_decode($response);
Note: The php on your web server must have the php5-curl library installed.
Assuming the API request is returning JSON data, this page may be useful.
This doesn't use any code that is specific to Symfony2. There may well be a bundle that can simplify this process for you, but if there is I don't know about it.
Symfony doesn't have a built-in service for this, but this is a perfect opportunity to create your own, using the dependency injection framework. What you can do here is write a service to manage the external call. Let's call the service "http".
First, write a class with a performRequest() method:
namespace MyBundle\Service;
class Http
{
public function performRequest($siteUrl)
{
// Code to make the external request goes here
// ...probably using cUrl
}
}
Register it as a service in app/config/config.yml:
services:
http:
class: MyBundle\Service\Http
Now your controller has access to a service called "http". Symfony manages a single instance of this class in the "container", and you can access it via $this->get("http"):
class MyController
{
$response = $this->get("http")->performRequest("www.something.com");
...
}
Best client that I know is: http://docs.guzzlephp.org/en/latest/
There is already bundle that integrates it into Symfony2 project:
https://github.com/8p/GuzzleBundle
$client = $this->get('guzzle.client');
// send an asynchronous request.
$request = $client->createRequest('GET', 'http://httpbin.org', ['future' => true]);
// callback
$client->send($request)->then(function ($response) {
echo 'I completed! ' . $response;
});
// optional parameters
$response = $client->get('http://httpbin.org/get', [
'headers' => ['X-Foo-Header' => 'value'],
'query' => ['foo' => 'bar']
]);
$code = $response->getStatusCode();
$body = $response->getBody();
// json response
$response = $client->get('http://httpbin.org/get');
$json = $response->json();
// extra methods
$response = $client->delete('http://httpbin.org/delete');
$response = $client->head('http://httpbin.org/get');
$response = $client->options('http://httpbin.org/get');
$response = $client->patch('http://httpbin.org/patch');
$response = $client->post('http://httpbin.org/post');
$response = $client->put('http://httpbin.org/put');
More info can be found on: http://docs.guzzlephp.org/en/latest/index.html
https://github.com/sensio/SensioBuzzBundle seems to be what you are looking for.
It implements the Kris Wallsmith buzz library to perform HTTP requests.
I'll let you read the doc on the github page, usage is pretty basic:
$buzz = $this->container->get('buzz');
$response = $buzz->get('http://google.com');
echo $response->getContent();
Symfony does not have its own rest client, but as you already mentioned there are a couple of bundles. This one is my prefered one:
https://github.com/CircleOfNice/CiRestClientBundle
$restClient = $this->container->get('ci.restclient');
$restClient->get('http://www.someUrl.com');
$restClient->post('http://www.someUrl.com', 'somePayload');
$restClient->put('http://www.someUrl.com', 'somePayload');
$restClient->delete('http://www.someUrl.com');
$restClient->patch('http://www.someUrl.com', 'somePayload');
$restClient->head('http://www.someUrl.com');
$restClient->options('http://www.someUrl.com', 'somePayload');
$restClient->trace('http://www.someUrl.com');
$restClient->connect('http://www.someUrl.com');
You send the request via
$response = $restclient->get($url);
and get a Symfony response object.
Then you can get the status code via
$httpCode = $response-> getStatusCode();
Your code would look like:
$restClient = $this->container->get('ci.restclient');
if ($restClient->get('http://www.yourUrl.com')->getStatusCode !== 200) {
// no error
} else {
// error
}
Use the HttpClient class to create the low-level HTTP client that makes requests, like the following GET request:
use Symfony\Component\HttpClient\HttpClient;
$client = HttpClient::create();
$response = $client->request('GET', 'https://api.github.com/repos/symfony/symfony-docs');
$statusCode = $response->getStatusCode();
// $statusCode = 200
$contentType = $response->getHeaders()['content-type'][0];
// $contentType = 'application/json'
$content = $response->getContent();
// $content = '{"id":521583, "name":"symfony-docs", ...}'
$content = $response->toArray();
// $content = ['id' => 521583, 'name' => 'symfony-docs', ...]
This is compatible with Symfony 5. Symfony Manual on this topic: The HttpClient Component
The manual is unclear as to how to implement this (it assumes you already know what exactly you're doing and in some cases feels like an afterthought), and I've been scratching my head for a fair while trying to figure it out.
The problem: authentication via HTTP auth headers for all API requests
As far as I've been able to test, I can use Basic auth and the normal form based login in CakePHP, but only by first hitting up the login action I define in the Auth component. This is fine when I'm accessing the site directly, and works as expected (with the exception of Digest, which appears to be utterly buggered). Via cURL, though, I've had no luck unless I'm already logged in.
Obviously, for an API, this is far from ideal. I don't want to post a request to /login before doing what I want to do, and I can't expect a user to log in manually so Cake has a cookie to read. It needs to be stateless.
Any attempt to supply authentication credentials along with each request I make (via cURL) is ignored and I get a 403 error in return. Neither the login method or any of the Auth classes are touched.
What do I need to do to make Cake behave like an actual API and allow me to authorise statelessly on a per request basis? Am I going to have to roll my own solution?
I have a centralized API that allows for user authentication via HTTP Digest and requires users to login for many user related functions. The way CakePHP forces a login is by checking if the action requires login, redirecting to your login action (defaults to /users/login), then you can redirect back.
I created my API by doing the following:
//Config/routes.php
///////////////////////////
/**
* Users Controller routes for REST API
*/
Router::mapResources('users');
/**
* Parses extensions for data serialization
*/
Router::parseExtensions();
//Controller/UserController.php
////////////////////////////////
<?php
App::uses('DigestAuthenticate', 'Controller/Component/Auth/');
class UsersController extends AppController {
var $name = 'Users';
//Login callback
function login() {
//dont render for login, just a call back for auth
$this->autoRender = false;
if ($this->Auth->login()) {
$this->redirect($this->Auth->redirect());
}
}
//GET /users.[xml|json]
//this is the basic call that tests user authentication
//basically a login then echo
function index() {
if ($this->Auth->login()) {
$user = $this->Auth->user();
$this->User->id = $user['id'];
$this->User->saveField('last_login', date('Y-m-d H:i:s'));
$this->set('response', array(
'response' => array(
'code' => 'users_auth_success',
'message' => 'User has passed authentication',
'data' => $user
)
));
//will serialize to xml or json based on extension
$this->set('_serialize', 'response');
}
}
}
?>
You can then use this API in something like:
$c = curl_init($uri . '.' . $this->_format);
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_USERPWD, $login['user'] . ':' . $login['pass']);
curl_setopt($c, CURLOPT_HTTPAUTH, CURLAUTH_DIGEST);
curl_setopt($c, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($c, CURLOPT_FOLLOWLOCATION, true);
$response = curl_exec($c);
$info = curl_getinfo($c);
curl_close($c);
if($info['http_code'] == $this->_http_codes['OK']) {
//success
if($this->_format == 'xml')
$response = Xml::toArray(Xml::build($response));
else//JSON
$response = json_decode($response);
return $response['response']['data'];
} else if($info['http_code'] == $this->_http_codes['Unauthorized']) {
return false;
} else {
return null;
}
Edit: Why the minus one?
What I am trying to do is the following:
I am trying to login to my school site using cURL and grab the schedule to use it for my AI.
So I need to login using my pass and number, but the form on the school site also needs a hidden 'token'.
<form action="index.php" method="post">
<input type="hidden" name="token" value="becb14a25acf2a0e697b50eae3f0f205" />
<input type="text" name="user" />
<input type="password" name="password" />
<input type="submit" value="submit">
</form>
I'm able to successfully retrieve the token. Then I try to login, but it fails.
// Getting the whole website
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.school.com');
$data = curl_exec($ch);
// Retrieving the token and putting it in a POST
$regex = '/<regexThatWorks>/';
preg_match($regex,$data,$match);
$postfields = "user=<number>&password=<secret>&token=$match[1]";
// Should I use a fresh cURL here?
// Setting the POST options, etc.
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields);
// I won't use CURLOPT_RETURNTRANSFER yet, first I want to see results.
$data = curl_exec($ch);
curl_close($ch);
Well... It doesn't work...
Is it possible the token changes every curl_exec? Because the site doesn't recognize the script the second time...
Should I create a new cURL instance(?) for the second part?
Is there another way to grab the token within 1 connection?
Cookies?
What's the error message you get? Independently of that; your school's website might check the referrer header and make sure that the request is coming from (an application pretending to be...) its login page.
This is how I solved it. The problem was probably the 'not-using-cookies' part.
Still this is probably 'ugly' code, so any improvements are welcome!
// This part is for retrieving the token from the hidden field.
// To be honest, I have no idea what the cookie lines actually do, but it works.
$getToken= curl_init();
curl_setopt($getToken, CURLOPT_URL, '<schoolsite>'); // Set the link
curl_setopt($getToken, CURLOPT_COOKIEJAR, 'cookies.txt'); // Magic
curl_setopt($getToken, CURLOPT_COOKIEFILE, 'cookies.txt'); // Magic
curl_setopt($getToken, CURLOPT_RETURNTRANSFER, 1); // Return only as a string
$data = curl_exec($token); // Perform action
// Close the connection if there are no errors
if(curl_errno($token)){print curl_error($token);}
else{curl_close($token);}
// Use a regular expression to fetch the token
$regex = '/name="token" value="(.*?)"/';
preg_match($regex,$data,$match);
// Put the login info and the token in a post header string
$postfield = "token=$match[1]&user=<number>&paswoord=<mine>";
echo($postfields);
// This part is for logging in and getting the data.
$site = curl_init();
curl_setopt($site, CURLOPT_URL, '<school site');
curl_setopt($site, CURLOPT_COOKIEJAR, 'cookies.txt'); // Magic
curl_setopt($site, CURLOPT_COOKIEFILE, 'cookies.txt'); // Magic
curl_setopt($site, CURLOPT_POST, 1); // Use POST (not GET)
curl_setopt($site, CURLOPT_POSTFIELDS, $postfield); // Insert headers
$forevil_uuh_no_GOOD_purposes = curl_exec($site); // Output the results
// Close connection if no errors
if(curl_errno($site)){print curl_error($site);}
else{curl_close($site);}
As you're building a scraper, you can create your own classes to work for what you need to do in your domain. You can start by creating your own set of request and response classes that deal with what you need to deal with.
Creating your own request class will allow you to implement the curl request the way you need it. Creating your own response class can you help you access/parse the returned HTML.
This is a simple usage example of some classes I've created for a demo:
# simple get request
$request = new MyRequest('http://hakre.wordpress.com/');
$response = new MyResponse($request);
foreach($response->xpath('//div[#id="container"]//div[contains(normalize-space(#class), " post ")]') as $node)
{
if (!$node->h2->a) continue;
echo $node->h2->a, "\n<", $node->h2->a['href'] ,">\n\n";
}
It will return my blogs posts:
Will Automattic join Dec 29 move away from GoDaddy day?
<http://hakre.wordpress.com/2011/12/23/will-automattic-join-dec-29-move-away-from-godaddy-day/>
PHP UTF-8 string Length
<http://hakre.wordpress.com/2011/12/13/php-utf-8-string-length/>
Title belongs into Head
<http://hakre.wordpress.com/2011/11/02/title-belongs-into-head/>
...
Sending a get request then is easy as pie, the response can be easily accessed with an xpath expression (here SimpleXML). XPath can be useful to select the token from the form field as it allows you to query data of the document more easily than with a regular expression.
Sending a post request was the next thing to build, I tried to write a login script for my blog and it turned out to work quite well. I needed to parse response headers as well, so I added some more routines to my request and response class.
# simple post request
$request = new MyRequest('https://example.wordpress.com/wp-login.php');
$postFields = array(
'log' => 'username',
'pwd' => 'password',
);
$request->setPostFields($postFields);
$response = new MyResponse($request->returnHeaders(1)->execute());
echo (string) $response; # output to view headers
Considering your scenario you might want to edit your own request class to better deal with what you need, mine already uses cookies as you're using them, too. So some code based on these classes for your scenario could look like:
# input values
$url = '<schoolsite>';
$user = '<number>';
$password = '<secret>';
# execute the first get request to obtain token
$response = new MyResonse(new MyRequest($url));
$token = (string) $response->xpath('//input[#name="token"]/#value');
# execute the second login post request
$request = new MyRequest($url);
$postFields = array(;
'user' => $user,
'password' => $password,
'token' => $token
);
$request->setPostFields($postFields)->execute();
Demo and code as gist.
If you want to further improve this, the next step is that you create yourself a class for the "school service" that you make use of to fetch the schedule from:
class MySchoolService
{
private $url, $user, $pass;
private $isLoggedIn;
public function __construct($url, $user, $pass)
{
$this->url = $url;
...
}
public function getSchedule()
{
$this->ensureLogin();
# your code to obtain the schedule, e.g. in form of an array.
$schedule = ...
return $schedule;
}
private function ensureLogin($reuse = TRUE)
{
if ($reuse && $this->isLoggedIn) return;
# execute the first get request to obtain token
$response = new MyResonse(new MyRequest($this->url));
$token = (string) $response->xpath('//input[#name="token"]/#value');
# execute the second login post request
$request = new MyRequest($this->url);
$postFields = array(;
'user' => $this->user,
'password' => $this->password,
'token' => $token
);
$request->setPostFields($postFields)->execute();
$this->isLoggedIn = TRUE;
}
}
After you've nicely wrapped the request/response logic into your MySchoolService class you only need to instantiate it with the proper configuration and you can easily use it inside your website:
$school = new MySchoolService('<schoolsite>', '<number>', '<secret>');
$schedule = $school->getSchedule();
Your main script only uses the MySchoolService.
The MySchoolService takes care of making use of MyRequest and MyResponse objects.
MyRequest takes care of doing HTTP requests (here with cUrl) with cookies and such.
MyResponse helps a bit with parsing HTTP responses.
Compare this with a standard internet browser:
Browser: Handles cookies and sessions, does HTTP requests and parses responses.
MySchoolService: Handles cookies and sessions for your school, does HTTP requests and parses responses.
So you now have a school browser in your script that does what you want. If you need more options, you can easily extend it.
I hope this is helpful, the starting point was to prevent written the same lines of cUrl code over and over again and as well to give you a better interface to parse return values. The MySchoolService is some sugar on top that make things easy to deal with in your own website / application code.