my script always detected why ?!!!!
most of time i use python selenium for scraping website but this time i'am always detected by "https://www.flashseats.com"
Can you help me please!!
here's part of my selenium code , i even try to use different proxies but :( same result,
please propose me a solution either in python or PHP.
chrome_options = webdriver.ChromeOptions()
options = webdriver.ChromeOptions()
options.add_argument('--disable-infobars')
options.add_argument('--disable-extensions')
options.add_argument('--profile-directory=Default')
options.add_argument('--incognito')
options.add_argument('--disable-plugins-discovery')
options.add_argument('--start-maximized')
#_chrome_options = Options()
#_chrome_options.add_argument('disable-infobars')
driver = webdriver.Chrome(r'C:\browser\chromedriver.exe', chrome_options=chrome_options)
time.sleep(10)
driver.get("https://www.flashseats.com/")
I know that might be a late answer, but you can try using Selenium-Profiles or undetected-chromedriver for that.
To answer your question: You're most likely getting detected because of values in javascript like navigator.webdriver, which might be different than in a normal browser.
Related
I plan to use Spree for a shopping site but at some point need to sign some data with a PHP program provided by a bank. The only alternative I can think of is to link to somePage.php that runs PHP program and come back to Spree. Is there any easier way like a sending to some PHP shell inside Ruby? or changing for the view to have php extension?
Any help would be appreciated.
Well, first I would check for a native Ruby way of signing your data in Ruby. Have a look at Spree documentation first, or at your bank specs (they usually are very bad, take how bankers write contracts, they can't be any good at writing software specs).
As a second alternative, if you have the PHP program you should try and translate it in Ruby.
If that is not an option for you then you can play with open4 like this:
status = Open4::popen4("/path/to/php bank_code.php #{data_to_sign}") do |pid, stdin, stdout, stderr|
out_msg = stdout.read
err_msg = stderr.read
logger.error "out_msg #{out_msg}"
logger.error "err_msg #{err_msg}"
end
handle_error_case if status.existatus != 0
Cheers,
I'm working with a very simple mock-up application to query a DB and display the results on a map. The application sends a GET httprequest to a server which returns a serialized array of value. The basic structure of the httprequest is:
httpRequest.open("GET","handle-query.php?query=" + queryJs)
and, on the other side:
$queryPhp = $_GET["query"];
When the query looks like this...
["SELECT%SUM(commit)%FROM%financialdata%WHERE%region='Centre'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%region='Kara'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%region='Maritime'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%region='Plateaux'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%region='Savanes'"]
...then it is passed to the server properly, and generates a response. However, when the query looks like this...
["SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Sotouboua'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Tchamba'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Tchaoudjo'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Assoli'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Bassar'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Bimah'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Doufelgou'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Keran'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Kozah'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Golfe'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Lacs'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Vo'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Yoto'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Zio'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Amou'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Haho'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Kloto'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Ogou'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Wawa'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Oti'", "SELECT%SUM(commit)%FROM%financialdata%WHERE%prefect='Tone'"]
...then the server receives an empty string. Both of the strings are generated by the same function, and both work perfectly on my virtual server (WAMP). If anyone has any ideas it would be greatly appreciated.
(P.S. After reading I realize that I should be using a framework with better sanitization, etc., but this is just a demo that will need to live online for maybe 2 hours, and it would be better to fix this small thing than start over. It works perfectly on my localhost.)
To actually answer your question, you're sending a get parameter as query (handle-query.php?query=) and then pulling it off as queryJS (queryJs).
$_GET['query'] // instead of $_GET['queryJs']
should do it.
(And as everyone has pointed out, don't send SQL in the clear or otherwise over the wire, unless you plan on not actually executing the SQL, and you just like to advertise your db structure, maybe it is that pretty.)
I'm trying very simple in PHP and not very sure what to search here or on google.
Problem is -
In PHP function I want to call/get a URL
http://www.example.com/message?Name=MyNameIsX
and like to read the return value (body) at this URL (which may contain "Your Name is MyNameIsX")
I tried
$data = file_get_contents($url)
This is timing out; although I'm able to open the $url in the browser.
Yes, file_get_contents normal use for files on this server and base on support and setting this perhaps is not allow.
See PHP CUrl http://php.net/manual/en/curl.examples.php or example
http://php.net/manual/en/curl.examples.php, http://php.net/manual/en/curl.examples-basic.php
You could use cUrl as suggested by FIG-GHD742 but I find the HTTP extension a lot easier to use. It's newer and has a neat OOP api.
Another method is that you can actually do an include/require with these, but it's generally a bad idea to do so if you don't control the source from which the data is coming
It sounds like you need to enable loopback calls on the server (self-calls). It would be better to get the data on the backend if you need it on the same server. Via a PHP API or calls to a database.
**
This will help you lot : http://php.net/manual/en/curl.examples.php
http://php.net/manual/en/curl.examples.php,
http://php.net/manual/en/curl.examples-basic.php
**
Yes the above answers is right. some hosting providers disable it for security purpose. You may also try fopen(php) if you are not looking for Curl way. Read documentation here http://php.net/manual/en/function.fopen.php
I have a GTFS protocol buffer message (VehiclePosition.pb), and the corresponding protocol format (gtfs-realtime.proto), I would like to read the message in PHP alone (is that even possible?).
I looked at Google's python tutorial https://developers.google.com/protocol-buffers/docs/pythontutorial and encoding documentation https://developers.google.com/protocol-buffers/docs/encoding and https://github.com/maxious/ACTBus-ui/tree/master/lib/Protobuf-PHP, but I am having a really hard time conceptualizing what is going on. I think I understand that gtfs-realtime.php is a compiled instruction set of the encoding defined in gtfs-realtime.proto (please correct me if I am wrong), but I have no clue how to get it to decode VehiclePosition.pb. Also, what are the dependencies of gtfs-realtime.php (or the python equivalent for that matter)? Is there anything else I have to compile myself or anything that is not a simple php script if all I want to do is read VehiclePosition.pb?
Thanks.
edmonscommerce and Julian are on the right track.
However, I've gone down the same path and I've found that the PHP implementation of Protocol Buffers is cumbersome (especially in the case of NYCT's MTA feed).
Alternative Method (Command Line + JSON):
If you're comfortable with command line tools and JSON, I wrote a standalone tool that converts GTFS-realtime into simple JSON: https://github.com/harrytruong/gtfs_realtime_json
Just download (no install), and run: gtfs_realtime_json <feed_url>
Here's a sample JSON output.
To use this in PHP, just put gtfs_realtime_json in the same directory as your scripts, and run the following:
<?php
$json = exec('./gtfs_realtime_json "http://developer.mbta.com/lib/GTRTFS/Alerts/VehiclePositions.pb"');
$feed = json_decode($json, TRUE);
var_dump($feed);
You can use the official tool: https://developers.google.com/transit/gtfs-realtime/code-samples#php
It was released very recently. I've been using it for a few days and works like a charm.
I would assume something along the lines of this snippet:
<?php
require_once 'DrSlump\Protobuf.php';
use DrSlump\Protobuf;
$data = file_get_contents('data.pb');
$person = new Tutorial\Person($data);
echo $person->getName();
as taken from the man page: http://drslump.github.io/Protobuf-PHP/protobuf-php.3.html
Before that step, I think you need to generate your PHP classes using the CLI tool as described here: http://drslump.github.io/Protobuf-PHP/protoc-gen-php.1.html
so something along the lines of:
protoc-gen-php gtfs-realtime.proto
Sorry Harry Truong, I tried your executable but it returns always NULL.
What I am doing wrong?
Edit: The problem is that I have no permission to execute in my server. Thanks for your executable.
I need to find a way to get all Apache running request at a given moment. I need to list the vhost, cpu, request ip address and some other information.
This information will be consumed by a PHP script.
I have mod_status installed and it has all the information I need. So I tried to use file_get_contents to get the report, generating a request from the server (http://localhost/server-status). It worked perfectly. Then I tried to parse the report, converting it to XML using simplexml_load_string. The problem is that the HTML outputted by mod_status is not well formed.
Here is the HTL from the table I need to parse:
<table border="0"><tr><th>Srv</th><th>PID</th><th>Acc</th><th>M</th><th>CPU
</th><th>SS</th><th>Req</th><th>Conn</th><th>Child</th><th>Slot</th><th>Client</th><th>VHost</th><th>Request</th></tr>
<tr><td><b>0-1</b></td><td>-</td><td>0/0/70</td><td>.
</td><td>0.00</td><td>107</td><td>0</td><td>0.0</td><td>0.00</td><td>0.34
</td><td>127.0.0.1</td><td nowrap>zsce</td><td nowrap>OPTIONS * HTTP/1.0</td></tr>
<tr><td><b>1-1</b></td><td>-</td><td>0/0/55</td><td>.
</td><td>0.04</td><td>108</td><td>0</td><td>0.0</td><td>0.00</td><td>0.70
</td><td>127.0.0.1</td><td nowrap>zsce</td><td nowrap>OPTIONS * HTTP/1.0</td></tr>
</table>
I'm sure someone has tried to do something like this before.
1) Is there another way to access the information I need?
2) Has anybody tried other tools / modules?
Thanks in advance.
I can't see the problem with the HTML. What's wrong with it?
Does PHP not have a liberal HTML parser; something like Python's BeautifulSoup or Ruby's Nokogiri?
Also, remember that mod_status has 'auto' mode for producing machine-readable output.
http://www.apache.org/server-status?auto
http://httpd.apache.org/docs/2.2/mod/mod_status.html#machinereadable
I just found that if I remove "nowrap" from the HTML before parsing it, it works.