Authentication with curl php - php

Question

I'm trying to access this url with curl in PHP :
https://www.maxityre.fr/search?matchcode=2055516&t=V&l=205&mid=&bquery=&h=55&d=16&seasonAll=on&from=
However, since the website requires authentication, I am redirected to :
https://www.maxityre.fr/
Here is what I tried so far :
$username='myusername';
$password='mypassword';
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, 'https://www.maxityre.fr/search?matchcode=2055516&t=V&l=205&mid=&bquery=&h=55&d=16&seasonAll=on&from=');
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); //status code is 200 thanks to this, otherwise 302
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
But it doesn't work and I'm redirected to https://www.maxityre.fr/ (I tried to echo the content and I'm getting the content of https://www.maxityre.fr/).
Am I doing this wrong ?

Answer 1

yes, you are doing plenty of things wrong, most important of which, is that you think they are using 1 of the login methods natively supported by libcurl, they're not. they're using a custom login scheme (like most websites these days), and here's how it goes:
first make a GET request to the login page. now you get a session cookie, which you must provide with all further requests (use CURLOPT_COOKIESESSION to make libcurl handle cookies for you automatically - that's definitely the easiest way), additionally, you get a html <input tag called callback, this callback probably includes a csrf token, and some additional info, - but there's a weird twist, SOMETIMES its empty, and sometimes not, not sure why - in any case, if you don't provide the correct callback variable in your next POST login request, the server will probably refuse to log you in, so you must parse out that from the HTML. now you're ready to login, make a POST request, with the cookies you received in the first GET request, and the callback you extracted from the HTML in the POST data, along with the variables action=>login,login=>your#email.com,password=>your password here, and the POST request body must be encoded in the application/x-www-form-urlencoded-format (easiest way to encode in this format is to use the http_build_query function). now you should be logged in. and to continue to get the logged in html, you must send the same cookie with all further requests, this cookie is tied with your logged in session. sending a request without this cookie, would be the equivalent of opening the page in a new web browser and notice that you're not signed in on the new browser.
here's an example implementation with hhb_curl from https://github.com/divinity76/hhb_.inc.php/blob/master/hhb_.inc.php (a convenience wrapper around curl_ functions)
<?php
declare(strict_types = 1);
require_once ('hhb_.inc.php');
const USERNAME = '???';
const PASSWORD = '???';
$hc = new hhb_curl ( 'https://www.maxityre.fr/', true );
$html = $hc->exec ()->getResponseBody ();
$csrf_token = (new DOMXPath ( #DOMDocument::loadHTML ( $html ) ))->query ( '//input[#name=\'callback\']' )->item ( 0 )->getAttribute ( "value" );
$html = $hc->setopt_array ( array (
CURLOPT_POST => true,
CURLOPT_URL => 'https://www.maxityre.fr/',
CURLOPT_POSTFIELDS => http_build_query ( array (
'action' => 'login',
'callback' => $csrf_token,
'login' => USERNAME,
'password' => PASSWORD
) )
) )->exec ()->getResponseBody ();
if (false !== stripos ( $html, 'Erreur d\'identification' )) {
throw new Exception ( 'failed to login! (maybe wrong username/password?)' );
}
echo "logged in!";
now you might be wondering, how did i figure that out? well, i recorded logging in with the Chrome Developer Tools,- (the thing you get if you open chrome and press Ctrl+Alt+I), the "Network Tab" - and then i simply replicated that login process with php :)