I have managed to get some HTML from a website using HTMLDomParser (http://simplehtmldom.sourceforge.net/):
$html = HtmlDomParser::file_get_html('https://example.com/id=1');
This results in the following HTML:
<p> ANDE 2, the Atmospheric Neutral Density Experiment 2, is a pair of microsatellites (Castor and Pollux) launched from Cape Canaveral on STS 127 on 15 July 2009 at 22:03 UT and deployed from the payload bay of the shuttle on 30 July 2009 at 17:22 UT.</p>
<p><strong>Launch Date:</strong> 2009-07-15<br/><strong>Launch Vehicle:</strong> Shuttle<br/><strong>Launch Site:</strong> Cape Canaveral, United States<br/></p>
What I want to attempt to do is to take this HTML and save it to a mySQL database. So, in the database I will have four columns:
Description
Launch Date
Launch Vehicle
Launch Site
The HTML layout will always stay the same. My other database contains the id's for the URL's. I am able to replace the id=1 with a variable, so I would be able to manage to save multiple webpage data.
The only thing I am worrying about if the id does not exist. Would it be possible just to skip that page if it cannot find the HTML it is looking for?
I have never done this before, so I am a bit overwhelmed by this. Would it even be possible to do this?
Basically, I just want to save data from a DomParser into my database.
EDIT: Since there has not been an answer, I think we need to tackle this systematically - first we should try creating the function that saves the data to the database.
EDIT 2: I was just thinking - maybe the function that needs to save the HTML can split the result from HTMLDomParser by tag ('strong', 'p', etc.) and create a variable for each column and then save it. I guess the question here first is how do you split the HTML?
EDIT 3: Since I did not receive any answers / suggestions, I am guessing it is very complex to implement this. I still have not managed to come up with anything that saves the HTML successfully to my database.
To check if the ID is valid to the site, I suggest a curl function, so if $url exists, try and get the data (try and catch), if there is an error with the data, catch it and return appropriate response or log it and move to next valid url.
You will have to format the data with a separate function inside the try.
Hope that helps as methodology goes, good luck. Peace
Related
I would like to make full use out of MySQL for the purpose of a (web) application I have developed for a chiropractor.
So far I have been storing in a single row for [every year] for what are called progress notes. The table structure looks something like this (progress_note_id, patient_id, date (Y-0-0), progress_note). When the client wishes to append for the year of the current progress notes, he simply clicks at the top of a textarea (html), which I use TinyMCE JavaScript library, to make a new entry date along with the shorthand notes to go at the beginning of the column (progress_note). So far its been working ok, if there are 900+ clients (est.) there could potentially be 1300+ progress notes, for each year since the beginning of the application (2018).
Now the client wishes to be able to see previous progress notes (history), but is unable to modify any previous notes, while still be able to write new ones. The solution I have come up with is to use XML inside the textarea, and use PHP to decipher the new notes from the old ones.
My problem however is if I should have to convert my entire table from a yearly to a daily, that it could take up a lot of time and energy to convert multiple notes into each single rows, (est. 10x) Which could end up being 13,000+ rows. I realize that no matter what method I choose to do is going to be a lot of work. Another way around this perhaps I found was to use XML column type in MySQL to potentially store multiple records, and if I wish to append it, all I would need is PHP to interpret the entire XML and add a new child node, to the beginning. Each progress note is 255 - 500 chars. And in worst case scenario, if the patient was to be 52 times a year (1 for every week), there shouldn't be a large enough overhead.
Is this the correct way to solving this problem? I do wish to keep with MySQL DB and I realize that MySQL is not an intended for XML. And for some clarification, what I hope to accomplish is the same thing I intended to do with current progress notes, but with XML. I believe in ascending order (newer -> oldest).
<xml_result>
<progress_note>
<date>2020-08-16</date>
<content></content>
</progress_note>
<xml_result>
Thank-you for any of your time and for any suggestions.
Firstly, 13000+ is not a problem for mysql. In most case for web application, mysql can handle more than 10m+ records for a single instance with a good performance.
Secondly, you can use either XML or JSON format in a text field and handle the decoding in your application.
I need help parsing the following a CSV file in PHP, so I can insert the contents into a database.
I know I use file_get_contents() but after that I feel a bit lost.
What I'd like to store.
Collection1 - events.text & date
Collection2 - position & name.text & total
I'm not sure how best structure the data to insert into a database table.
"**collection1**"
"events.href","**events.text**","**date**","index","url"
"tur.com/events/classic.html","John Deere Classic","Thursday Jul 9
- Sunday Jul 12, 2015","1","tur.com/r.html"
"collection2"
"**position**","name.href","**name.text**","**total**","index","url"
"--","javascript:void(0);","Scott","--","2","tur.com/r.html"
"--","javascript:void(0);","Billy","--","3","tur.com/r.html"
"--","javascript:void(0);","Jon","--","4","tur.com/r.html"
"--","javascript:void(0);","Bill","--","5","tur.com/r.html"
"--","javascript:void(0);","Tim","--","6","tur.com/r.html"
"--","javascript:void(0);","Carlos","--","7","tur.com/r.html"
"--","javascript:void(0);","Robert","--","8","tur.com/r.html"
"--","javascript:void(0);","Rod","--","9","tur.com/r.html"
As per your previous question, I think this needs to be broken down into sections. As it stands it is rather too broad to answer.
Read the information using file_get_contents(). Make sure this works first, by echoing it to the console. (It sounded from your other question that you felt this would not work if the URL does not have a .csv suffix. It should work regardless of the file extension - try it. If it fails it may be dependent on cookies or JavaScript or some other problem).
Design and create your table structure in MySQL. It seems like you have two tables. They should both have a primary key. Are they related in some fashion? If so, perhaps one has a foreign key to the other one?
Explode your text file on the new line character and loop across the resulting array of lines.
If your CSV data has a title row in the first row position, delete that from your array.
For each line, read the elements of interest using PHP's build-in CSV parsing functions, and store them in variables.
Pass these variables to a custom function that saves the data.
For each save, you'll need to do an INSERT. I recommend using PDO here. Make sure you bind your parameters.
Where you get stuck on a specific problem, you can ask a new and focussed question. At present, the task is to break things down into discrete and researchable pieces.
One trick worth remembering is this shortcut to the PHP manual. If you do not know how fgetcsv works, for example, type php.net/fgetcsv into your browser address bar, and the PHP site will find the function for you. The documentation is excellent.
I want to display a large amount of text using a php echo command. I have that data stored in mysql database table in a text field. What i want to achieve is that the data should be displayed in the same manner in which i store it in the text field.
for example:
As entered in Mysql table by its Interface One reason people lie is to achieve personal power.
Achieving personal power is helpful for someone who pretends to be more confident than he really is. For example, one of my friends threw a party at his house last month. He asked me to come to his party and bring a date.
Although this lie helped me at the time, since then it has made me look down on myself.
Should be displayed exactly as the above rather than:
One reason people lie is to achieve personal power.
Achieving personal power is helpful for someone who pretends to be more confident than he really is. For example, one of my friends threw a party at his house last month. He asked me to come to his party and bring a date.
Although this lie helped me at the time, since then it has made me look down on myself.
Any ideas/tips on how this can be achieved?
I know that i can manually insert html tags between the text for formatting but i dont want to manually do so. Any way around?
nl2br($foo); will automatically add a <br> tag wherever there is a linebreak in $foo. You can echo nl2br($foo);.
As an alternative, try the <pre> tag. <pre><?php echo $foo; ?></pre>. You many need more styling, but it will preserve whitespace like your linebreaks.
My solution is:
I'm using GWT TextArea textAreaWidget widget.
Before insert the TextArea string to MySQL table I replace all line change and tab characters:
-new line
String toInsert=textAreaWidget.getText().replaceAll(Character.toString((char) 10), "\n\r"));
-tab
String toInsert=textAreaWidget.getText().replaceAll(Character.toString((char) 9), "\t"));
Example:
http://www.tutorialspoint.com/gwt/gwt_textarea_widget.htm
I'm trying to make a chat history system. So every time a person says hi to one another, they can also say something else. And each of those hi's, i wan't to add what they wrote into a history html file. Being something like this:
James Said: Hi Richard, i saw that hardware you told me about, it is compatible with our software!.
At: 23 November 2011 - 23:09 UTC-08.
________________________________________________________________________________________
Richard Said: Nice!! let's start working with it this week, the project has to be finished before the end of the world.
At: 24 November 2011 - 09:23 UTC-08.
________________________________________________________________________________________
The html file i can build with php, but how do i save it to a MySQL BLOB? Without storing it in a directory (directly to the BLOB).
You're approach to this problem isn't really a good one.
If you try to store the data in a particular output format then you're in real trouble if you suddenly find you need the data in a different format.
You're much better off just storing the particulars of the conversation, and then generating the output to display from the stored conversation. That way you can easily present it in all kinds of formats you might need it in.
EDIT TO ADD:
Something else I should have mentioned (but forgot thanks to all the Christmas brandy ;) ), trying to store the conversation data in a single big block of data will negate most of the advantages using a relational database can confer in the first place. You couldn't, for example, easily store the timestamp of each line of the conversation, or search the database for particular items in the conversation. You could find workarounds of course, but given databases are already designed to solve those kinds of problems anyway, you'd just be wasting effort and your solution wouldn't measure up to what the database already provides.
Since it is not really a binary (the B in Blob), but HTML, I suggest you use the MEDIUMTEXT type and deal with it as just a normal text field.
I have a MySQL Database of more or less 100 teachers, their names, and their phone numbers, stored in tables based upon their department at the school. I'm creating an iPhone app, and I've found that UITableViews and all the work that comes with it is just too time consuming and too confusing. Instead, I've been trying to create a web page on my server that loads all the data from MySQL and displays it using HTML, PHP, jQuery, and jQTouch for formatting.
My concept is that the separators will be by department, and the staff will be sorted alphabetically under each department. On the main page, each person's name will be clickable so they can go to ANOTHER page listing their name, email address, and telephone number, all linked so that the user can tap on the email or number and immediately email or call that person, respectively.
HOWEVER, I am completely at a loss for how I should start. Can anyone point me in the right direction for displaying the data? Am I going about it wrong in using PHP? Should I opt for something COMPLETELY different?
PHP to manage the database interaction and generate HTML is fine. There are heaps of tutorials on how to do that (e.g. http://www.w3schools.com/PHP/php_mysql_intro.asp) How to make it look nice is beyond the scope of this answer, and I'd recommend you search for table/CSS examples to get some ideas of what looks good and how they're implemented. If you need interactivity such as expanding rows or changing colors, then jQuery would be an appropriate next step, though you certainly don't need more than HTML + CSS for a nice looking table representation.
What I don't know about is the auto email/call functionality you're after, and whether you can get that "for free" from whatever is rendering the HTML. That's iPhone specific, not PHP/jQuery/etc... And I'd second Alex's advice that if UITableView is the right tool for the job then you will definitely be better off in the long run just buckling down and learning it. (And going through that will probably make pickup up other parts of the API much easier to boot.)
Instead of loading my PHP in my <body>, I created a function that retrieved the data via mysql_fetch_assoc(), which added all the information and created each individual div of data AS WELL AS injecting a <script> to $.append() the list item content for each item retrieved via the mysql_fetch_assoc(). Thanks for the responses anyway!