Thrift + Cassandra + PHP + Windows? - php

Connecting to Cassandra with PHP is really a pain. The documentations at Apache and DataStax are extremely poorly written - for Windows users.
I have Thrift installed (I believe!) via Chocolatey. But I am still not able to compile php code for Cassandra using thrift.
If you look at this link ,
now we can compile php code for Cassandra using thrift I used command: d:\cassandra\trift\thrift.exe --gen php
d:\cassandra\interface\cassandra.thrift
So what is cassandra.thrift and where does it come from?? WHAT should I put inside it??
If I follow the instruction exactly, I get this error,
Could not open input file: d:\cassandra\interface\cassandra.thrift
So what is going on?
How do I make this work?
I have tried to install DataStax PHP Driver for Apache Cassandra and that documentation even worst.
Why PHP modules do not come with Cassandra like it does for MongoDB? Most of the independent drivers I found are outdated, not supported anymore or abandoned.
EDIT:
From the README,
Install the PHP extension
Installing with pecl
The PHP driver is not published to the official PECL repository yes.
You can still install it using pecl by specifying the provided
package.xml file path as the argument to pecl install command.
Install the 2.0 version of the C/C++ driver
not published to the official PECL repository yes - is it yes or yet?
Obtaining Build Dependencies
CMake
Git
ActiveState Perl
Python v2.7.x
I have downloaded and installed. Then, what? In Building the Driver,
A batch script has been created to detect installed versions of Visual...
What? Where does A batch script suddenly come from??
Then,
First you will need to open a “Command Prompt” (or Windows SDK Command
Prompt) to execute the batch script.
Usage: VC_BUILD.BAT [OPTION...]
--DEBUG Enable debug build
--RELEASE Enable release build (default)
--DISABLE-CLEAN Disable clean build
....
What are these bunch of '--' for?
To build 32-bit shared library:
VC_BUILD.BAT --X86 To build 64-bit shared library:
VC_BUILD.BAT --X64
Where does .BAT come from? What should I put inside it? Where should I run it from??
After all, what are those Build Dependencies for? How do I use them??
Just hope that someone can write a proper guide then the guide above - it is frightening! (if you compare the guides in MongoDB, it is far better and professional)
EDIT 2:
First error when I run the .bat from my desktop,
I have git installed already but I still have this error,
After fixing git issue above, I have a new one - it just frozen there, nothing happens,

The IDL file cassandra.thrift is usually part of the cassandra package, but you can find it following the link above. The link points to trunk, you may want another version.
After you downloaded the right version of that file or better found it in your downloaded Cassandra package in the interface folder, generate the code as outlined in the documentation you have. The rest should be easy.
Why PHP modules do not come with Cassandra like it does for MongoDB? Most of the independent drivers I found are outdated, not supported anymore or abandoned.
I'm not really sure about that, but my guess is that the fact CQL is promoted heavily for a while now instead of using the raw Thrift API - the latter is a complex task, while CQL is more easy to use - is one of the key factors why it is this way. It more or less eliminates the need for another wrapper.
PS: Just to be sure:
thrift.exe --gen php d:\cassandra\interface\cassandra.thrift
could not open input file cassandra.thrift
Of course you point to the right drives, folders and files, do you?

Forget about Thrift and the 'beta', I found a better solution. It very straight forward and extremely easy!
Example codes,
require_once 'lib/Cassandra/Cassandra.php';
$cassandra = new Cassandra();
$s_server_host = '127.0.0.1'; // Localhost
$i_server_port = 9042;
$s_server_username = 'admin'; // We don't use username
$s_server_password = 'password'; // We don't use password
$s_server_keyspace = 'demo'; // We don't have created it yet
$cassandra->connect($s_server_host, $s_server_username, $s_server_password, $s_server_keyspace, $i_server_port);
// Tests if the connection was successful:
if ($cassandra) {
// Select:
// Queries a table.
$cql = "SELECT * FROM users;";
// Launch the query.
$results = $cassandra->query($cql);
// Update:
// Prepares a statement.
$stmt = $cassandra->prepare('UPDATE users SET first_name = ?, last_name = ? where id = ?');
// Executes a prepared statement.
$values = array('first_name' => 'Fred', 'last_name' => 'Smith', 'id' => '1');
$result = $cassandra->execute($stmt, $values);
// Insert:
// Prepares a statement.
$stmt = $cassandra->prepare('INSERT INTO users (id, first_name, last_name)
VALUES (:id, :first_name, :last_name)');
// Executes a prepared statement.
$values = array('first_name' => 'John', 'last_name' => 'Robinson', 'id' => '4');
$result = $cassandra->execute($stmt, $values);
// Delete:
// Prepares a statement.
$stmt = $cassandra->prepare('DELETE FROM users WHERE id = :id');
// Executes a prepared statement.
$values = array('id' => '4');
$result = $cassandra->execute($stmt, $values);
// Closes the connection.
$cassandra->close();
}

Update:
The datastax PHP driver is now GA and binaries are available for download (no need to build it yourself):
https://github.com/datastax/php-driver
Regarding the DataStax PHP driver, the instructions are being improved per your feedback as I type.
Because this Driver is in Beta, we do not yet have pre compiled binaries that you can simply download. They will be available once the driver is GA. For now you will have to build them yourself.
The process for building the binaries is very straight forward. 1) Install the dependencies 2) run vc_build.bat.
You can find the vc_build.bat here (just right click save as from your browser):
https://raw.githubusercontent.com/datastax/php-driver/master/ext/vc_build.bat

Related

Same mariaDB inserts much more slower in PHP 7.4 than PHP 7.1 with FAT FREE

I'm trying to migrate a legacy PHP/Fat Free project from php 7.1 to 7.4 and I found that some queries take too long (like 10x more time) to finish. Particularly some inserts. I'm running the same project in my localhost with xampp (7.1.32 and 7.4.6) and using the exact same MariaDB server (v10.4.8) with the exact same database always.
The code is something like that:
foreach($ridiculouslyLongArray as $row) //I'm talking about some millons of rows
$this->db->exec("INSERT INTO a_table (field1, field2, fieldn) VALUES ('".$row['field1']."', '".$row['field2']."', '".$row['fieldn']."')");
//Yes, it's open to sql injection, I will fix that too
The definition of $this->db is the next:
$this->db = new DB\SQL('mysql:host=localhost;port=3306;dbname=something', 'dbuser', 'dbpassword', array(\PDO::ATTR_ERRMODE=>\PDO::ERRMODE_EXCEPTION));
and is a wrapper of PDO as far as I know.
I've tried to change the code to insert multiple rows per query but the query still taking much more time than in php 7.1.
This is my setup
->Original Project (in which the queries run fine)
PHP 7.1.32 (memory limit 2048mb)
Fat Free 3.6.4
MariaDB 10.4.8
->New Project (in which the queries run slow)
PHP 7.4.6 (memory limit 2048mb)
Fat Free 3.7.2
MariaDB 10.4.8 (same server and db that in the previous one)
Thanks for your time.
EDIT: I Just noticed that the PDO Drivers for MySQL are different between versions
for PHP 7.1:
mysqlnd 5.0.12-dev - 20150407 - $Id: 38fea24f2847fa7519001be390c98ae0acafe387 $
for PHP 7.4:
mysqlnd 7.4.6
Edit 2: The query is in a transaction and it is using the same indexes and same dB engine because is the same insert over the same table in the same database on the same server. Nothing change in the code only the PHP versión.
This wasn't explicitly mentioned in the comments, but something else that may be causing some slowness is query logging.
By default, Fat-Free will log all DB queries. If you are running a gazillion inserts, all those inserts are being logged. If it's not already, I would recommend in production to disable query logging. Wherever your bootstrap/services file is that creates the db connection, I would add this after it:
$f3->set('db', new DB\SQL(/* config stuff */));
if(ENVIRONMENT === 'PRODUCTION') { // or whatever you use to signal it's production
$f3->db->log(false);
}

Share variables between python & php

I have a script in python receiving continuously data from sensors.
I need to publish the last data at request on a webpage using php.
Apache, php and python are all on the same linux install (actually on an raspberry).
I'm not interested in storing previous data and I'm a bit concerned about data corruption on writing on SD. I would prefer to reduce complexity and increase speed refresh (avoid sql).
Could a text file written in ramfs / tmpfs solve the problem? or there is a method to share memory like memcache to share also global variables?
Any practical example or howto will be really well-accepted.
I think you may try System V shared memory.
As example:
In Python side:
python -m pip install sysv_ipc
then somewhere in the python script:
import sysv_ipc
...
KEY=20171220
sysv_memory=sysv_ipc.SharedMemory(KEY, sysv_ipc.IPC_CREAT, 0666, 256)
...
sysv_memory.write('1234'+'\0')
Then in the PHP side:
$SHARED_MEMORY_KEY = 20171220;
...
$shmId = shmop_open($SHARED_MEMORY_KEY, 'a', 0666, 256);
...
$test_string = shmop_read($shmId, 0, 0);
I can get the $test_string as '1234' successfully in PHP.
Here is a solution using memcached that works on Raspbian 10 (buster), PHP 7.3.19, and Python 3.7.3:
1. Install memcached
apt-get install memcached php-memcached
pip install pymemcache
These commands install memchached server and client modules for PHP and Python.
2. PHP code
$m = new Memcached();
// connect
$m->addServer('127.0.0.1', 11211);
// get a value
$value = $m->get('key');
// set a value
$m->set('key', 'value');
// clean up
$m->quit();
3. Python code
from pymemcache.client import base
# connect
m = base.Client(('127.0.0.1', 11211))
# get a value
value = m.get('key')
# set a value
m.set('key', 'value')
# clean up
m.close()
Note:
I used default memcached settings here. If you need to change them edit sudo nano /etc/memcached.conf and restart the daemon: sudo /etc/init.d/memcached restart.
You can use any interoperable format like json or msgpack.
Whenever you generate data in python, add it to a caching layer like memcache/redis using json format ( and preferably a gzip compressed version), then your PHP script can unserialize the json data and display it in the app.
Clearly memcache as a means to share data different application will work.
You will not have any corrupted data for sure as all the memcache operation are atomic. memcache atomic discussion could be useful.
On memcached's FAQ:
Is memcached atomic? Aside from any bugs you may come across, yes all commands are internally atomic. Issuing multiple sets at the same time has no ill effect, aside from the last one in being the one that sticks.
Note: Running memcache service might consume considerable amount of memory.
Hope it helps!

GnuPG isn't working in PHP with nginx

I think that probably I'm missing something, but I don't see it right now. I want create a simple form where users can encrypt automatically messages between them (form message to user2 -> encrypt(message) -> user2 receive it and decrypt). I'm using nginx, I installed gnupg following their instructions and add it to my php.ini (now it shows that GnuPG is enabled with GPGME Version 1.4.3 and Extension Version 1.3.6) I want use a specific keyring located at /usr/share/nginx/.gnupg I tried the following code:
$iterator = new gnupg_keylistiterator("developer");
foreach($iterator as $fingerprint => $userid) {
echo $fingerprint." -> " . $userid . "\n";
}
var_dump($iterator);
And I just obtain the following response from var_dump:
object(gnupg_keylistiterator)#1 (0) { }
Maybe my question is an idiot question, but I never used gnupg in php and I want learning, but I'm stunk since yesterday and I don't understand why it doesn't work...
Thanks for your time
The most common issue is that you imported the keys to another keyring than later is searched for keys. GnuPG uses a per-(system)-user "GnuPG home directory", each containing individual keyrings. If you import a key as the administrator or a developer you import the key to your own keyring, while usually the web server running the PHP application is executed in another user context and will not find this key, resulting in an empty result when listing the keys from within PHP.
You can set this by setting up an environment variable before initializing the GnuPG binding.
putenv("GNUPGHOME=/tmp"); // Set GnuPG home directory to the temp folder
$res = gnupg_init(); // Initialize GnuPG
Obviously /tmp does not actually qualify as a reasonable directory, choose something where your application stores application data anyway. It should not be a directory accessible through HTTP.
As an alternative, gnupg_import($res, $pubkey) the key before using it (but this will result in some performance penalty for importing the key).

locating dependencies on windows php extension builds

I am writing a php extension for a library. I have a generic swig file to build wrappers for the library. This has been very successful so far on Python.
A user is trying to build the library for php and I am trying to help out. I generated the code using swig, and I can build the .dll extension using Visual Studio. The problem is getting it into php. When I build php_mylib.dll, it needs to find mylib.dll and it can't.
So I am trying to build via the command line a la:
http://blog.benoitblanchon.fr/build-php-extension-on-windows/
I have put all the files to be compiled and the libraries needed (i.e. mylib64.lib and mylib64.dll) in a folder called mylib in the C:\php-src\ext folder with all the other extension folders.
The problem is I that I can't get the config.w32 file to communicate the location of mylib. Here is my config.w32 file (pretty standard -- you can see that I copied it from the curl config.w32 file):
// $Id$
// vim:ft=javascript
ARG_ENABLE("mylib", "mylib support", "no");
if (PHP_MYLIB != "no") {
if (CHECK_LIB("mylib64.lib", "mylib", PHP_MYLIB) &&
CHECK_HEADER_ADD_INCLUDE("mylib_cpp.h", "CFLAGS_MYLIB")
) {
EXTENSION("mylib", "mylib_c_wrap.cpp", true);
AC_DEFINE('HAVE_MYLIB', 1, 'Have mylib library');
// TODO: check for curl_version_info
} else {
WARNING("mylib not enabled; libraries and headers not found");
}
}
When I run buildconf and then configure --disable-all --enable-cli --enable-mylib it always shoots me the 'libraries and headers not found' warning from the script.
On Unix systems (config.m4) there appears to be a PHP_ADD_LIBRARY_WITH_PATH macro but I don't see any equivalent for windows. It seems like this is what I need.
I have also tried adding the full path to mylib into the system's path but to no avail. It seems like there might be an environment variable somewhere in the PHP build cosmos that needs to be set to find external dependencies, but I can't find any information about this.
It would also be good to do all this as a Visual Studio project -- easier for Windows users the world over. I have not seen anything on the web that looks like this.
By the way, I have successfully phpized this library using the same swig+phpize procedure on Linux (I followed this guide for the php part:http://www.sitepoint.com/install-php-extensions-source/) and it works beautifully.
In most cases you can use this info to build a PECL extension https://wiki.php.net/internals/windows/stepbystepbuild#building_pecl_extensions . Your config.w32 looks ok, though please note that that when depending on some additional library, usually it should be ARG_WITH(...) for semantics. Your lib stuff can be then put into the deps dir as in the wiki.
Another way could be passing --with-extra-libs and --with-extra-includes to configure. Those have to contain paths to *.lib and *.h dirs respectively. When using a static lib, that should be it, otherwise you'll need to place the corresponding *.dll onto the %path% for the ext to work.
Hope this helps, otherwise you can also gain some attention on the PHP mailing lists or specifically on #winphp-dev at freenode.
Thanks.

Using Cassandra PDO Driver on Windows

Is there any way to have Cassandra PDO at Windows with Wamp?
This is for development purposes I don't want to install Linux and change all the environment.
https://code.google.com/a/apache-extras.org/p/cassandra-pdo/
I'm using Windows 7 (64 Bit), Wamp 2.5, PHP 5.5.
OK, here's what I found out:
1) It's totally possible
2) The docs that appear in the first google search results are a bit obsolete
Start by downloading the latest Datastax Community Cassandra here:
http://planetcassandra.org/cassandra/
Install & setup properly. In fact, most of the configuration is done by the installer, you just have to edit the apache-cassandra/conf/cassandra.yaml file to find all paths to /var/lib... and change those into something like d:/cassandra/... That includes "commitlog", "data", "saved_caches". Restart the Cassandra service, examine the logs. Mine shown no problem. The OpsCenter at ...:8888/opscenter/index.html was working fine, showing one node online.
Now, the PHP part.
There's a sneaky thing called Thrift. From what I've learned today (I first heard about Cassandra and Thrift yesterday), it's a way describe a binary protocol of connecting to some online service, in this case, to Cassandra. It will basically generate PHP files that will provide all the connectivity you need from PHP itself (no extensions needed).
You will need:
1) The Thrift PHP libs
2) The .exe Thrift compiler
Both can be downloaded here:
https://thrift.apache.org/download
Then use the following command to compile PHP files that will act as a "driver" to connect your PHP applications to Cassandra:
thrift --gen php D:\DataStaxCommunity\apache-cassandra\interface\cassandra.thrift
Put the result in some PHP include_path folder.
Also, find the PHP Thrift libs (in the libs archive from the same download page) and put those in a folder accessible to your script (e.g. include_path or the project folder).
Refer this page:
thrift.apache.org/lib/php
I guess that should help!
I have same problem as you, but when i tried this method, it works correctly for me.
Reference link
Here is a code example, very easy to understand :
<?php
require_once 'Cassandra/Cassandra.php';
$o_cassandra = new Cassandra();
$s_server_host = '127.0.0.1'; // Localhost
$i_server_port = 9042;
$s_server_username = ''; // We don't use username
$s_server_password = ''; // We don't use password
$s_server_keyspace = 'cassandra_tests';
$o_cassandra->connect($s_server_host, $s_server_username, $s_server_password, $s_server_keyspace, $i_server_port);
$s_cql = "CREATE TABLE carles_test_table (s_thekey text, s_column1 text, s_column2 text,PRIMARY KEY (s_thekey));";
$st_results = $o_cassandra->query($s_cql);

Categories