I use LinkedIn and other Social Media to promote certain blog content on my Website. My website is on Wordpress and I therefore wrote a Plugin that created a widget for displaying the visit source. Once I have this source, I can store this on a table and can view the number of visits from each Social Media channel to test the effectiveness of the post.
The problem is that while I can identify click throughs from Facebook, Twitter, G+ etc. with no problem, I do not get any referer information from LinkedIn.
I use the read the information as follows:
//Get the referer from Wordpress
$ref_raw = wp_get_raw_referer();
echo '<li>RAW: '.( strlen($ref_raw)? $ref_raw: 'None' ).'</li>';
//Get the IP from the server to check
$ref_srv = $_SERVER['HTTP_REFERER'];
echo '<li>PHP: '.( strlen($ref_srv)? $ref_srv: 'None' ).'</li>';
//Get other information
$ip = strlen($_SERVER["HTTP_X_REAL_IP"])? $_SERVER["HTTP_X_REAL_IP"]: $_SERVER['REMOTE_ADDR'];
echo '<li>REFERRER IP: '.( strlen($ip)? $ip: 'None' ).'</li>';
$agent = isset($_SERVER['HTTP_USER_AGENT'])&& strlen($_SERVER['HTTP_USER_AGENT'])? $_SERVER['HTTP_USER_AGENT']: '';
echo '<li>USER AGENT: '.( strlen($agent)? $agent: 'None' ).'</li>';
echo '<li>URI: '.$_SERVER['REQUEST_URI'].'</li>';
The output from this code when a visit originated from Facebook is as follows:
RAW: https://www.facebook.com/ PHP: https://www.facebook.com/
REFERRER IP: 89.127.12.32
USER AGENT: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36 REFERRAL
URI: /how-speed-affects-your-website/?fbclid=I123C-as
However, a visit from LinkedIn looks like this:
RAW: None
PHP: None
REFERRER IP: 89.127.12.32
USER AGENT: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
REFERRAL TYPE: external
URI: /how-speed-affects-your-website/
Any ideas why there is no referer data being transferred from LinkedIn? or any thoughts on how I can observe visits from LinkedIn by any other means?
Is your own site using HTTPS? And do those links the users follow use HTTPS URLs?
If not, you will simply not get a referrer from LinkedIn in modern browsers, due to Referrer-Policy.
no-referrer-when-downgrade (default)
This is the user agent's default behavior if no policy is specified. The URL is sent as a referrer when the protocol security level stays the same (HTTP→HTTP, HTTPS→HTTPS), but isn't sent to a less secure destination (HTTPS→HTTP).
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy#Directives:
LinkedIn doesn’t actively specify a Referrer-Policy as far as I can see, so this fallback value will be used automatically.
So if your site doesn’t support HTTPS, or the links go to the HTTP version first, the browser is forbidden from sending you this info.
or any thoughts on how I can observe visits from LinkedIn by any other means?
Check if they append any specific, uniquely named tracking parameters to outgoing links, like Facebook does with the ?fbclid=I123C-as. Then you could take the existence of that parameter as an indicator that the user most likely followed a link from LinkedIn to your site.
Related
I have researched many python post and sites to build a few other scripts that log into specific accounts of mine to scrape/parse data from a site. I have come across one I cannot figure out, relayrides.com. The site uses PHP and I have successfully logged into other php based pages using python, but it seems a JSESSIONID and rr_u_cid are set when the pages loads and passed with the POST from, see sample code used to login to other pages:
`payload = {
'action': '/login',
'email': 'user#gmail.com',
'password': 'pass1234'
}
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36'
}
with session() as c:
c.post('https://relayrides.com/login', data=payload)
response = c.get('https://relayrides.com/earnings', headers=headers)
print(response.text)`
I've tried pass the settings a cookie in the header, using CookiJar, etc. I am just new to python and web-scraping with python.
Thanks,
dickie
I'm trying to login into a website and send an sms with python. I'm using the request module, passing the user name and password. After I have passed that I am unable to judge whether the login has succeeded or not.
url='http://www.connectexpress.in'
headers={"User-Agent":"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36"}
s=requests.session() ; s.headers.update(headers)
r=s.get(url) ; soup=BeautifulSoup.BeautifulSoup(r.content)
VIEWSTATE=soup.find(id="__VIEWSTATE")['value']
#VIEWSTATEGENERATOR=soup.find(id="__VIEWSTATEGENERATOR")['value']
EVENTVALIDATION=soup.find(id="__EVENTVALIDATION")['value']
login_data={"__VIEWSTATE":VIEWSTATE,
"__EVENTVALIDATION":EVENTVALIDATION,
"txtUserName":usrNme,
"txtPassword":paswrd,
"submit":'Submit'}
r1=s.post(url, data=login_data)
This is the piece of code that I used and could any one give some ideas ?
I need to get requesting browsers name in my web app.( for analytics )
In core php when I use $visitor_user_agent=$_SERVER['HTTP_USER_AGENT']it returns Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36 string when using with chrome.And then preg_match('/Chrome/i', $visitor_user_agent) can be used to know if its chrome or not.I am not sure if that was efficient way to find browser name or not.
I also found get_browser link but it is not giving browser name.
Is there a way in cakephp3 or core php to get browser name ?
This would return the user agent used for the request:
$this->request->header('User-Agent');
http://book.cakephp.org/3.0/en/controllers/request-response.html
Look into documentation of Request object.
You can get HTTP_USER_AGENT using env() method:
$this->request->env('HTTP_USER_AGENT');
You can also prepare custom detector:
$this->request->addDetector(
'chrome',
['env' => 'HTTP_USER_AGENT', 'pattern' => '/Chrome/i']
);
And then in controller just use is() method as follows:
if($this->request->is('chrome')) {
// do stuff for chrome
}
I'm trying to show the browser that I used to open my site using Codeigniter User Agent Libraries. When I open with IE,Chrome,Firefox,and Safari, User agent return the right value. But when I open with Opera, it return "Chrome" value.
here is part of my code:
Controller
*** another code ***
$this->load->library('user_agent');
$data['browser'] = $this->agent->browser();
$this->load->view('agent',$data);
*** another code ***
View
Your browser is <span><?php echo $browser; ?></span>.
the result if I open with opera is this:
Your browser is Chrome.
My question is, why it return Chrome? How can I fix this problem?
Thanks.
It looks like the user agent string in opera (since version 15) is this:
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.52 Safari/537.36 OPR/15.0.1147.100
You might need to extend the user_agent library to do a check to see if the user agent string contains OPR.
https://dev.opera.com/blog/opera-user-agent-strings-opera-15-and-beyond/
How to detect dolphin browser in php? Unfortunatly I don't have android, so I can't check the dolphin browser's user agent.
thanks in advance,
The user agent contains "Dolfin/2.0".
On my version of Dolphin, 8.8.2 - running on a Samsung Galaxy Note, there is no particular user agent and 'desktop' is set. The user-agent is set by tapping on More -> Settings -> User Agent. I am then able to change the user agent to any of the following:
Android
Desktop, which identifies as: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16
iPhone
iPad
Custom (I guess you could set a custom user-agent such as 'dolfin' if you want).
So if you're trying to deliver content specific to the Dolphin browser in PHP you could test if 'mobi' or 'Android' is present in the user agent. If you're trying to detect that the exact browser being used is Dolphin then that will be tricky unless your client device's custom user-agent is set.
You can check the User Agent
if (strpos(strtolower($_SERVER['HTTP_USER_AGENT']), 'dolphin') !== false) {
echo 'Your UA is Dolphin';
}
You can do a more accurate check if you know the exact User Agent, but I assumed that if it contains the word dolphin is accurate enough.