I am running a few PHP websocket server scripts. Basically the server scripts should run 24h.
They should be able to access a not so large array like object, maybe 2k to 5k items. The PHP server like scripts should be able to write/read the shared array.
Other PHP scripts (triggered by async random ajax calls) should be able to read/write the shared array. I did some tests with shmop and I am not happy with the fact that I need to serialize data to be stored as string and I am worried about concurrency.
I have no idea how to proceed. Performance should by the main goal, not data security. Any suggestion?
There is no better solution for storage, in situation you've described, than shared memory.
If you care only for access time to bulk data SHM is your "go for it".
You can simply create wrapper to maintain serialization and storage in your code.
If there is need for storage and access data in certain manner (indexed / sorted) you should try something like sqlite with files stored on tmpfs/ramfs (in memory).
Related
I want to use session values in my script which are stored at the server using php can any one kindly explain the process to achieve this.
I want to build a chat app for this am planning to use those session values.
Assume usera and userb are logged in and their userid is sessioned based on this scenario i want to do a chat app.
Now i had done the app but i had used setinterval function of Javascript and am calling the chats i want to avoid the database hits on every 3 mill sec.
Kindly Help me out
Thanks In Advance
You're basically attempting to use PHP session files as a file cache.
Instead, you should use an object caching system such as Memcached or Redis. If memory caching isn't an option (shared hosting, etc), then you could implement your own file cache (or you could use something like PHPFastCache, which supports file caching).
Note: File caching for a chat app may or may not speed up your application. It depends on how you implement it and a number of other factors.
Hi put the session value in input box,
<input type='hidden' id='session_value' value='<?php $_SESSION['value']?>'>
Using the id fetch the session value in script,
<script>
var session_value = document.getElementById ( "session_value" );
</script>
3ms is insanely short delay to run a polling chat system. I suggest increasing it to at least 200ms but preferably around 1000ms.
$_SESSION values are per user and not recommended for viewing a chat stream for a number of reasons. Instead it sounds like you are looking more to just update the chat feed.
The database unless it is hosted on another server and $_SESSION will be the equivalent, since the database is effectively files as well. The database will actually generally be faster than reading raw file storage since Queries are normally cached and Indexing helps lookup records quicker. In addition you won't have to worry about concurrent connections to the files either.
If anything enable OPCache and install APCu for your PHP installation, to help aid the serving of requests. OPCache will cache your compiled OP code into memory so that subsequent requests to the file won't need to be recompiled.
APCu will act as your file cache, again storing your rendered data in memory.
Additionally many Database Frameworks such as Doctrine can also utilize APC caching for query and result caching.
Instead of using a InnoDB or MyISAM storage engines for your chat messages I suggest trying the MEMORY storage engine.
So instead of accessing the File System I/O your database would instead be utilizing the Memory I/O. The general concept is few writes, many reads. Since one person writes to the database, requires everyone to read the data. Just keep in mind that the Memory storage engine is temporary and is lost if the server restarts or power is lost.
For more information see: https://dev.mysql.com/doc/refman/5.6/en/memory-storage-engine.html
Overall if you are able, I would suggest look at using Socket IO (Websockets) instead of either database or file based caching. This puts the load on the clients instead of the server, and everything occurs in real-time instead of polling for changes.
For some examples see:
Ratchet http://socketo.me/
React http://reactphp.org/
Node.js http://tutorialzine.com/2014/03/nodejs-private-webchat/
I am working on a PHP project and asked to implement a system (runs on server) which uses same memory location for every request.
To be simpler, think that there is an array in the memory (RAM) and every client ask for one element of it. Server does not create that array repeatedly. To achieve it, server must use a shared memory and returns the related elements to the clients. The question is, how can I do it? Or is there any source explaining it.
Constraints:
I don't want to use applet technology. And as much as possible, I want to implement it via PHP.
I don't want to use a database since it is too slow for our system and our data does not require to be persistent for any system down.
Data is really small (does not exceed 10MB) and fits to the memory.
Run MySQL and use the MEMORY storage engine. The table(s) will exist only in memory, will not be persisted to disk, and will not be "too slow" as operations are essentially at the speed of memory access.
Whatever you do, don't reinvent the wheel. Lots of in-memory data stores exist with PHP drivers/interfaces.
I have a php script and i want to store some information in RAM. The information should be accessible ONLY from my script and stored all time. What is best way to do that?
What about globals arrays?
I want to do it for security reasons. No one can to get access to the information. For example If somebody hacks one of the my scripts they cannt to get access to the stored data.
I suppose you mean "store in RAM across various requests" since all variables in PHP (inside 1 script) are already stored in RAM. You should look into this (depending on the access you have to the server to install stuff / let someone install stuff)
http://memcached.org/
You can't store for all time in RAM, once the power goes out, it's gone. Usually you use something like memcache or APC. But, if you wanted to role you own solution, you can use shared memory and sempahores.
http://www.php.net/manual/en/book.shmop.php
http://php.net/manual/en/book.sem.php
there's no way around not using the RAM while executing a script. all your variables and objects will be allocated on the heap (which is in the RAM)
You could use APC, a pecl extension http://www.php.net/manual/en/intro.apc.php
Very stable, allows you to store pieces of info into a data store (closest thing to RAM).
Faster than file storage or even databases.
If you're running on a shared server you'll want to secure your data though (by a using a naming scheme or encryption)
1)Memcached
2)Mysql table with Memory engine.
I don't think that PHP can access the computer RAM. But you could store the informatio in a session variable or a cookie.
I'm assuming that for every page request, the webserver (eg. Apache) creates a new instance of a script in memory. Can these instances communicate with each other while running? and pass data too?
If you want to pass data between scripts in PHP I suggest using either memcached or a database. Or possibly APC.
If the scripts belong to the same session, they could theoretically communicate via the session but this would be effectively a one-way communication in most cases because only one script can access the session at any one time (session_start() locks the session until that script ends the session implicitly or explicitly).
I believe Martin and Cletus' suggestions are valid. My choice would be function of the end goal of the script.
How much data will you be throwing around? Can you handle the overhead of an external process?
What kind of data are you exchanging? Is it normalized? Or is it now worth normalizing?
Will you need to refer to that data later on? Or can it be discarded after being processed?
Will those scripts ever run on different servers?
Flat files, with a locking mechanism
Relational DB
Document DB (key/value store, whether persistent or not)
Shared memory (APC, or core functions)
Message queues (Active MQ and company)
I think you'll get the most value by externalizing the process as you can have more than one machine managing the messages/data and more than one producing/consuming them.
The model that PHP scripts operate off of doesn't really contain the notion of any kind of persistence in memory for those scripts, since generally they're designed to only run for the minimum of time required to serve the requested page. This would make it hard to have any meaningful use for stateful communication between those scripts, since typically once the page is served there's nothing more for the script to do. Thus usually any communication between PHP scripts is done more through manipulation of database entries and the like.
If you have some sort of continual processing that should be happening for which you'd want to be passing data around, you might want to look into other web application models such as servlets.
You should be able to do this with some shared memory, as described here: http://blog.taragana.com/index.php/archive/how-to-use-shared-memory-in-php/ (assuming you're not running on Windows)
I need a simple way for multiple running PHP scripts to share data.
Should I create a MySQL DB with a RAM storage engine, and share data via that (can multiple scripts connect to the same DB simultaneously?)
Or would flat files with one piece of data per line be better?
Flat files? Nooooooo...
Use a good DB engine (MySQL, SQLite, etc). Then, for maximum performance, use memcached to cache content.
In this way, you have the ease and reliability of sharing data between processes using proven server software that handles concurrency, etc... But you get the speed of having your data cached.
Keep in mind a couple things:
MySQL has a query cache. If you are issuing the same queries repeteadly, you can gain a lot of performance without adding a caching layer.
MySQL is really fast anyway. Have you load-tested to demonstrate it is not fast enough?
Please don't use flat files, for the sanity of the maintainers.
If you're just looking to have shared data, as fast as possible, and you can hold it all in RAM, then memcached is the perfect solution.
If you'd like persistence of data, then use a DBMS, like MySQL.
Generally, a DB is better, however, if you are sharing a small, mostly static amount of data, there might be performance benefits (and simplicity) of doing it with flat files.
Anything other than trivial data sharing and I would pick a DB however.
1- Where the flat file can be usefull:
Flat file can be faster than a database, but in very specific applications.
They are faster if the data is read from start to finish without any search or write.
If the data dont fit in memory and need to be read fully to get the job done, It 'can' be faster than a database. Also if there is lot more write than read, flat file also shine, most default databases setups will need to make the read queries wait for the write to finish in order maintain indexes and foreign keys. Making the write queries usually slower than simple reads.
TD/LR vesion:
Use flat files for jobs based system(Aka, simple logs parsing), not for web searches queries.
2- Flat files pit falls:
If your going with a flat file, you will need to synchronize your scripts when the file change using custom lock mechanism. Which can lead to slowdown, corruption up to dead lock if you have a bug.
3- Ram based Database ?
Most databases have in memory cache for query results, search indexes, making them very hard to beat with a flat file. Because they cache in memory, making it run entirely from memory is most of the time ineffective and dangerous. Better to properly tune the database configuration.
If your looking to optimize performance using ram, I would first look at running your php scrips, html pages, and small images from a ram drive. Where the cache mechanism is more likely to be crude and hit the hard drive systematically for non changing static data.
Better result can be reach with a load balancer, clustering with a back plane connections up to ram based SAN array. But that's a whole other topic.
5- can multiple scripts connect to the same DB simultaneously?
Yes, its called connection pooling. In php (client side) its the function to open a connection its mysql-pconnect(http://php.net/manual/en/function.mysql-pconnect.php).
You can configure the maximum open connection in php.ini I think. Similar setting on mysql server side define the maximum of concurrent client connections in /etc/mysql/my.cnf.
You must do this in order to take advantage of parrallel processessing of the cpu and avoid php script to wait the query of each other finish. It greatly increase performance under heavy load.
There is also one connection pool/thread pool in Apache configuration for regular web clients. See httpd.conf.
Sorry for the wall of text, was bored.
Louis.
If you're running them on multiple servers, a filesystem-based approach will not cut it (unless you've got a consistent shared filesystem, which is unlikely and may not be scalable).
Therefore you'll need a server-based database anyway to allow the sharing of data between web servers. If you're serious about either performance or availability, your application will support multiple web servers.
I would say that the MySql DB would be better choice unless you have some mechanism in place to deal with locks on the flat files (and some way to control access). In this case the DB layer (regardless of specific DBMS) is acting as an indirection layer, letting you not worry about it.
Since the OP doesn't specify a web server (and PHP actually can run from a commandline) then I'm not certain that the caching technologies are what they're after here. The OP could be looking to do some sort of flying data transform that isn't website driven. Who knows.
If your system has a PHP cache (that caches compiled PHP code in memory, like APC), try putting your data into a PHP file, as PHP code. If you have to write data, there are some security issues.
I need a simple way for multiple
running PHP scripts to share data.
APC, and memcached are both good options depending on context. shared memory may also be an option.
Should I create a MySQL DB with a RAM
storage engine, and share data via
that (can multiple scripts connect to
the same DB simultaneously?)
That's also a decent option, but will probably not be as fast as APC or memcached.
Or would flat files with one piece of
data per line be better?
If this is read-only data, that's a possibility -- but may be slower than any of the options above. Especially if the data is large. Rather than writing custom parsing code, however, consider simply building a PHP array, and include() the file.
If this is a datastore that may be accessed by several writers simultaneously, by all means do NOT use a flat file! Writing to a flat file from multiple processes is likely to lead to file corruption. You can lock the file, but you risk lock contention issues, and long lock wait times.
Handling concurrent writes is the reason applications like mysql and memcached exist.