Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I am creating a simple web application using PHP, Codeigniter and Google Books (on an WIndows 7 XAMPP localhost environment).
I have a MySQL list of books (a few hundred) and corresponding ISBN numbers. When a user views a book / visits a URL for the first time, and API call is made to Google Books and the title, author and description of the book is saved to my database.
Ideally i'd like to populate the database myself and not rely on the user. So, I was thinking of visiting each URL manually. However there a lot of items!
Is there a script I can use for such a task? I was hoping to run the script once every 5 minutes over a 24 hour period.
My URL is in the following format:
/items/itemView/1 // <-- book 1
/items/itemView/2 // <-- book 2
/items/itemView/3 // <-- book 3
// etc
// etc
Thanks
Short Answer:
A storage API exists so you don't have to catalogue everything.
Long Answer:
It sounds like what you are trying to do is take the API and scour through every single entry and record them for your own purposes.
While this can usually done fairly simply, instead of telling you how to do this, I'm going to tell you why you shouldn't.
An API to a huge database exists so that you don't have to store it all, as the resources required can be absolutely huge, usually more than most enthusiasts would even have.
It's better to have it as you do now, cache what is visited on the chance it is visited again and make sure periodically that any records you DO keep, you compare to it's source so that you don't have an out-of-date record (another pitfall of local caching).
I hope this helps at least show you why people tend not to duplicate large data sources.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I need to do a MASSIVE data mine. I want to find out;
A users location
Look at their tweets for specific words in the last two days
Repeat (ideally) for every twitter user
I've seen R recommended somewhere, but wouldn't really know where to begin.
Happy with CSV, json or SQL endpoint.
As you tagged "python" in your question, I'm going to assume you're ok with it ! Twitter lets you access its data by two APIs :
REST API allows you to make specific user requests (profile, friends, etc.), but it only allows a few queries per hour, so it probably does not meet your "massive data" criterion
The streaming API delivers tweets based on a search on real-time. You can definitely harvest massive data using this API, and if I remember correctly, tweets come up with useful infos (user who tweeted of course, but probably location too if enabled).
Tweepy (http://www.tweepy.org/) is a user-friendly Python library implementing both Twitter APIs, providing particularly helpful functions for capturing data from the streaming API (see examples here : https://github.com/tweepy/examples).
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I have a client that has a list of several thousand email addresses (and their personal info). If they are holding an event in a particular city, they want to send an email to "everyone in these cities with these parameters". They would like to keep track of who attends each event, let people unsubscribe, but not loose their data, track other attributes about each person over time, etc.
The main company site is in Drupal, but this project can use any platform. Any suggestions about php software that can be customized to do this?
There is heaps of software to do this. You will have to get the data into a form that can be imported into a database though like mysql.
Wordpress has lots of plugins that can import mailing lists and news letters.
Once the data is in a database table then you could also write your own quite easily. If you aren't a coder though wordpress plugins or something similar might be the way to go.
Also, checkout mailchimp and services like that.
p.s. if you send several thousands emails you will probably get black listed :P
Obviously there is lots of software to do this, but I was looking for any advice someone might have to help me narrow down the search because there are so many scripts to test.
In this case, I settled on CiviCRM, which has both a drupal and a wordpress integration and does everything that I needed, specifically the ability to record lots of meta data about the individuals in the database and then to segment that data to use for sending email.
https://civicrm.org/
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I need to implement a recommendations system for site that lists songs, the idea being that the system can recommend songs that may be interesting to people viewing the current song, based on what other people voted on.
I realise this is a huge area that could get very complicated, but I was wondering if there are any standard algorithms or 3rd party libraries (PHP) that could be used to generate the recommendations? I know we may need to collect additional data depending on the system we implement, however we have already been logging the following for two years:
VOTES (users can vote on songs based on a 5 star system), data collected is:
song id, user id, session id, vote date & time, star rating 1-5
If you wanted to go all out, e.g. tracking each user's movement on your site and looking for overlaps (as I imagine Amazon, etc. do), then yes, it might get a little hairy.
But if you want to drive this purely with your voting system, I don't think you need anything as complicated as a library, or even a line of PHP. You could probably build a decent heuristic right into your database query. Just off the top of my head:
SELECT FIRST 10 V2.SongID, AVG(V2.Stars)
FROM Votes V1
JOIN Votes V2 USING (UserID)
WHERE V1.SongID = :CurrentSong AND V1.Stars > 3
GROUP BY V2.SongID
ORDER BY 2
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
Here's what I need: I have a table in a SQLite file, that contains items - descriptions, dimensions, image and thumbnail.
I need to allow someone from outside the company to edit this table through an "admin interface". I need a simple login mechanism to authenticate said user, and I need a form that shows all available rows in the table applicable to him/her, and allow editing the values. That may include uploading images.
Now, I've developed tons of these interfaces before, in several languages. What I'd like to ask is: is there a shortcut? Since this needs to be quick and dirty (i.e., this wasn't in the original plan, I'm not being paid for this, but I may lose a client if I don't have it in place) and be up as soon as possible, is there some open source solution, or any previous PHP code, that I can customize and use in this and future cases?
Any solution that will save me time is welcome.
Thanks for YOUR time :)
Guy
Well if you can use a framework you could use an auto generated admin interface or scaffolding from a framework. Symfony and Cake both have this. Of course thats a lot of dead weight to have if the whole app/site isnt using the framework. But it would make it relatively painless to create. IF you can run this interface on a subdomain that would make it even esier since you dont have to worry about integrating it with anything existing except the DB and shared folder for the uploaded files.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I'm developing an administrative panel to manage product listings. The database structure is already done and it takes me no effort to make a little panel where administrative users will insert and remove products. But every product has a variable number of images that should to be uploaded by the administrative user. Doing a good work will take me some hours, and I have absolutely no time. So I need something pre-made that will let the administrative user:
upload the photos of the product, in a certain server directory (passed as a parameter, for example)
delete one or more photos (preferably with a checkbox to select some of them and then a "delete" button)
show the thumbnails of the currently uploaded photos
sort the photos (I understand that's quite an high request, because sorting means renaming the files!), preferably in an intuitive way (drag & drop, etc...).
It would be awesome if the script could be seamlessly included in an another script without using iframes.
Why don't I just Google for it? That's because I would have to try all the stuff around (when 90% of it is ugly) and I hope you already know a good script you have tested yourself that can work in my scenario!
I've already considered Gallery3, it might work, but it is overkill and hard to integrate. I don't need album creation and management, per-user permissions, or photo showcase and slideshow, what I need it's basically a tiny file manager tailored for images.
There's little time for hiring a developer online, it would take too much to get "in tune" with him/her...
Take a look at http://www.codecanyon.net, they might have scripts that fit your needs over 80% and just need little modifications.
I don't know how many of your criteria jAlbum meets, but it might be worth a look. Someone I know has used it extensively with success.
Another one I'm less familiar with, but might suit, is Plogger.
Of course, with the deadline you were facing, you have probably already sorted this by now! :)