accessing third-party websites repeatedly efficiently - php

I have to collect the twitter statuses of 100,000+ users daily. Like access twitter for each user. I was wondering what the most efficient way of doing using PHP and mysql (I'm using the Yii PHP framework)? Thanks!

Have a look at the Yii cache functions: http://www.yiiframework.com/doc/guide/1.1/en/caching.data there are many approaches, disk, mysql, redis, etc., the best one for you depends on your set-up, but certainly some kind of cache strategy is advised when fetching 3rd-party data.

Related

Making a dynamic site cache able? PHP

i'm wondering to generate cache pages of my website to avoid same queries again and again. there's almost 80% of queries fetching data that is almost not change able in near future (if cache system doesn't effected of its changing).
is there any good cache library or script which i should use? or is there any conditions to use cache system? Site pages are simple developed in html, js, css and no smarty template etc. please advise me in this regard. Thank you in advance.
Redis is popular caching framework. It is an open-source, networked, in-memory, key-value data store that can be used as a drop-in caching backend for your Drupal or WordPress website.
You can store data as key value in redis, so it is very easy to integrate in your application. For more information look Redis as a Caching Backend. It may be helpful to you.

How to architect a multisite manager application?

I'm trying to create a web application to manage multiple websites. I'm still not sure what platform to built upon this kind of app, Node.js or Laravel (PHP). I want the application to be scalable to millions of users with an average of 2 sites each.
Would you recommend SQLLite3, JSON or MYSQL for storing all individual site's data in separate files or a single database with all individual site's data?
How can I prevent users from hacking into each others site?
mainSite >
Common
Site1/config.json, content/posts/, layout/
Site2/config.json, content/products/, layout/
Site3/config.json, content/pages/, layout/
I'm still not final on the above structure. Please help me with getting started with this app.
For scalable system you should use some system that will allow you easy redistribute resourses, so in this case i think node.js is better one, you could find many services that shares resourses for node.js
Big plans. Lots of work. My advise start small.
Setup one website with laravel and see how far you get.
You will get one website with multiple pages.
Laravel is easy, scales, has tons of extensions and a living, vivid community.
You have the choice to clone the website and set the same thing up under a different domain or just a subdomain.

Secure user authentication frameworks, libraries

I know this question has been asked more then once and more then twice but most answeres i've found here haven't helped me at all.
Since i started to code i've used three selfmade different authentication implementations based on cookies. I've never took much stand against injection or hijacking but this time i am trying to incorporate that in both my user authentiaction and submitting forms.
Since there are thousands of user authentication scripts to choose from i want to ask what frameworks or scripts you would recommend i use? I've tried a few but no one has really done the job? I'm not useing any php framework like Zend, Codeigniter or similar.
I plan on using SSL as well but on some sites my cms will be used on will not have SSL.
Try this one: http://ulogin.sourceforge.net/
If you need an authentication library that ...
isn't tied to a single framework and
works with multiple database systems (e.g. MySQL, PostgreSQL, SQLite, etc.)
... then https://github.com/delight-im/PHP-Auth might be for you.
It's modern (PHP 5.6+), secure and convenient.
Symfony2 provides a security component that is available as a standalone PHP library.
Documentation: Symfony2 - Security
Download it from GitHub

PHP Framework for large scale data intensive web development

I would like to know what would be the best MVC framework that can be used give these parameters
Ability to handle large DB
easy to use and configure
secure(session encryption, etc)
high performance
Available support(community)
ability to handle AJAX would be an added benefit, but not critical
I have been looking all over the web with conflicting results.
Please give your views, Efforts appreciated, thanks!!!
Try Yii Framework. I like the way it designed. And they claim the performance of the framework is outstanding (here is the Yii performance comparison to other PHP frameworks).
If you want to use framework for highloaded website, you can go with any framework you prefer. The bottleneck in highloaded applications is usually a database, so you will have to use different kind of caching mechanisms. Yii supports different kind of caches
To optimize complex SQL queries in Yii, don't use it's ActiveRecord, but use DAO instead.
There is no obvious answer though.
What about Simple PHP Framework?
http://clickontyler.com/simple-php-framework/

What are methods for migrating from one technology to another?

I'm working on a project using Zend Framework, which I initially chose since I had a bit more experience in it. At this point in time, I'm thinking of moving to Django, though, as our current team will be more productive in it.
I'm trying to figure out the best way to (potentially) go about this, as we evaluate it. A lot of our code is still fairly fluid, and being re-written fairly often. The way I had considered is that as something ends up being re-done, redo it in Django, and change my PHP model to simply be a call to the Django underbelly and translation to PHP. Over time, we should have all of the models, at least, in Django.
Are there better ways? Particularly, is there any way to easily manage having two codebases at once?
There is no generic answer to cross-language migration projects.
If you can wrap your php code and hide it behind web services, you will be on a good path because you will be able to migrate those web services later without changing the API.
Make a list of the main components of your architecture and try to see which ones could be hidden behind such a web service API.
I would recommend that if your web app is db driven, move your business logic to your db and have your data mappers call db stored procedures. I'm developing using ZF myself, which is a very strong tool, but the immense 'abstracticity' it offers can deviate you from your goals... Then you can mess as much as you like with your controllers and views...

Categories