php curl post not processing, acting like get with no data - php

Question

I am trying to use php's curl to post to a sites form for me then extract the result, but it is not working. Instead it shows a blank form, like I just did a basic GET reequest to the page.
<?php
$domains = [
'expireddomains.net',
'stackoverflow.com',
'toastup.com'
];
$ccd = '';
foreach ($domains as $domain) {
$ccd .= $domain . '\r\n';
}
// set post fields
$post = [
'removedoubles' => '1',
'removeemptylines' => '1',
'showallwordmatches' => '1',
'wordlist' => 'en-v1',
'camelcasedomains' => $ccd,
'button_submit' => 'Camel+Case+Domains'
];
$ch = curl_init('https://www.expireddomains.net/tools/camel-case-domain-names/');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
$headers = [
'Referer: https://www.expireddomains.net/tools/camel-case-domain-names/',
'Content-Type: application/x-www-form-urlencoded',
'Origin: https://www.expireddomains.net'
];
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
// execute!
$response = curl_exec($ch);
// close the connection, release resources used
curl_close($ch);
I confirmed the postdata formatting and names using the network tab in the browser's dev tools to check the request.
Originally I wasn't sending any headers, then I thought maybe the site validated the origin or referer, but even adding that didn't work.
I checked, the form doesn't include any hidden fields for something like a CSRF token or anything.
Any ideas?

Answer 1

For application/x-www-form-urlencoded, use http_build_query and let it encode the values like +'s etc, plus the seperator between domains is | not new lines.
<?php
$domains = [
'expireddomains.net',
'stackoverflow.com',
'toastup.com'
];
// set post fields
$post = [
'removedoubles' => 1,
'removeemptylines' => 1,
'showallwordmatches' => 1,
'wordlist' => 'en-v1',
'camelcasedomains' => implode(' | ', $domains),
'button_submit' => 'Camel Case Domains'
];
$ch = curl_init('https://www.expireddomains.net/tools/camel-case-domain-names/');
$headers = array();
$headers[] = 'authority: www.expireddomains.net';
$headers[] = 'pragma: no-cache';
$headers[] = 'cache-control: no-cache';
$headers[] = 'origin: https://www.expireddomains.net';
$headers[] = 'upgrade-insecure-requests: 1';
$headers[] = 'dnt: 1';
$headers[] = 'content-Type: application/x-www-form-urlencoded';
$headers[] = 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36';
$headers[] = 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9';
$headers[] = 'sec-fetch-site: same-origin';
$headers[] = 'sec-fetch-mode: navigate';
$headers[] = 'sec-fetch-user: ?1';
$headers[] = 'sec-fetch-dest: document';
$headers[] = 'referer: https://www.expireddomains.net/tools/camel-case-domain-names/';
$headers[] = 'referrer-policy: same-origin';
$headers[] = 'accept-language: en-GB,en-US;q=0.9,en;q=0.8';
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_ENCODING, 'gzip, deflate');
// execute!
$response = curl_exec($ch);
// close the connection, release resources used
curl_close($ch);
// parse whats in textarea
$dom = new DOMDocument();
libxml_use_internal_errors(true);
$dom->loadHTML($response);
libxml_clear_errors();
$result = [];
foreach ($dom->getElementsByTagName('textarea') as $textarea) {
if ($textarea->getAttribute('name') === "camelcasedomains") {
$result = explode(' | ', $textarea->nodeValue);
}
}
print_r($result);
Result:
Array
(
[0] => ExpiredDomains.net
[1] => ExpiredDoMains.net
)
You could probably remove most of the headers, if not needed. I just added them all to exactly match the request, but ended up being the aforementioned encoding.