I have the following text-file with 48.891 names. I would like to import them to a mySQL database. I need the first 3 characters with the gender and the name. The rest should be ignored.
M Aad 4 $
M Aadam 1 $
F Aadje 1 $
M Ådne + 1 $
M Aadu 12 $
?F Aaf 1 $
F Aafke 4 $
? Aafke 1 $
F Aafkea 1 $
M Aafko 1 $
M Aage 761 $
M Åge + 56 $
F Aagje 1 2 $
F Aagot 1 $
F Ågot + 2 $
F Aagoth 1 $
F Ågoth + 1 $
M Åke + 118 $
M Aalbert 1 $
M Aalderich 1 $
M Aalderk 1 $
The database handling isn't my question. The problem is how to filter only the gender and name in the textfile.
Thnaks for any help in advance.
To do this only in SQL , you can see LOAD DATA INFILE or you can do it with a little program.
I've found that the ETL software Kettle by Pentaho makes it pretty easy to import text files into mysql (especially if you don't have the most technical background. The documentation is kind of sparse, and this software is far from perfect, but I've found that I can often import data in a fraction of the time I would have to spend writing a script for one specific file. You can select a text file input and specify the delimiters, fixed width, etc.. and then simply export directly into your SQL server (they support MySql, SQLite, Oracle, and much more). The drag and drop interface is great for anyone who doesn't want to write code every time they run an import.
Related
I am trying to communicate with the Stockfish chess engine from a PHP script. For that I am using the UCI protocol, so I will need to send commands line by line. For example, this is what I would normally enter on the Windows Command Prompt:
Stockfish.exe
position startpos
go depth 16
Stockfish.exe is the 64-bit version of the Stockfish chess engine.
I failed to do it using exec(). Here is how I attempted it:
exec("stockfish.exe\nposition startpos\ngo depth 16");
The engine runs, but the commands are not executed, so I get:
Stockfish 10 64 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott
where I should get something like:
info depth 1 seldepth 1 multipv 1 score cp 116 nodes 20 nps 10000 tbhits 0 time 2 pv e2e4
info depth 2 seldepth 2 multipv 1 score cp 112 nodes 54 nps 27000 tbhits 0 time 2 pv e2e4 b7b6
info depth 3 seldepth 3 multipv 1 score cp 148 nodes 136 nps 45333 tbhits 0 time 3 pv d2d4 d7d6 e2e4
info depth 4 seldepth 4 multipv 1 score cp 137 nodes 247 nps 61750 tbhits 0 time 4 pv d2d4 e7e6 e2e4 c7c6
bestmove d2d4 ponder e7e6
I've read many threads on related issues, but couldn't find a workaround. Is there any way to accomplish this?
Could anyone help me? I am trying find formula and write piece of code in PHP language which makes next
Imagine, we have 3 types of something, k = 1,2,3 and length of this numbers could be various (n-length), but neighboring type should not(!) be the same - 1,1 or 2,2
For example
k = 1,2,3
n = 5
Output
1,2,3,1,2 |
1,2,3,1,3 |
1,2,3,2,1 |
1,2,3,2,3 |
1,3,2,1,3 |
1,3,2,1,2 |
1,3,2,1,3 |
1,3,2,3,1 |
1,3,2,3,2
.........
Mb this is has some common named problem, share with me pls and I'will try to find some resources about
Thanks
The simplest way of generation such lists is recursive (if n, k are not large -note that variant count is k*(k-1)n-1).
Pseudocode:
Generate(list, n, k, lastvalue)
if (list.length = n)
output(list)
else
for i = 1 .. k
if (i != lastvalue)
Generate(list + i, n, k, i)
Delphi code
procedure Generate(list: string; n, k, lastvalue: Integer);
var
i: Integer;
begin
if (Length(list) = n) then
Memo1.Lines.Add(list)
else
for i := 1 to k do
if (i <> lastvalue) then
Generate(list + IntToStr(i), n, k, i)
end;
begin
Generate('', 4, 3, 0);
Output for n=4, k=3
1212 1213 1231 1232 1312 1313 1321 1323
2121 2123 2131 2132 2312 2313 2321 2323
3121 3123 3131 3132 3212 3213 3231 3232
well u do a loop in a loop. since k has a length, and the numbers in the k variable are the ones you move along, they present the outer loop. (for loop)
now u can run along the output because u have the number n known as well.
u place k[1] as the first variable of n[1] and it doesn't change until the inner loop is over. (in this case k[1] is 1). now u do a while loop with a changeable (a for example) variable that runs over the n array created. n will be (1,null, null, null, null). while a != n.lenth(). u check the n(a-1) for it's value to make sure is not the same. whenever a reaches n.length, u change the value of the last number by the next on the k array, and then you go back 2 spots (n[a-1]) and change it and go back, go recoursive all the way to the start until all spots have been changed and the n[2] of the array has the highest value of the k array. to make life easier, u can make a new array, let us assign as j for the matter, which will get a value as soon as the closest n[a] spot gets the last value possible.
BTW whenever u go back reset the value of spots u ran by to null so that all the numbers in the k array are optional again. when the j array is full, you reset all of it and move on in the for loop.
hope i was of help, if you have any questions feel free to ask
So I exported a part of data from a table using the command:
<?php
$con = mysqli_connect('localhost','root','','database');
$sql = "SELECT * INTO OUTFILE 'data.sql' FROM table WHERE PID<50";
mysqli_query($con, $sql);
mysqli_close($con);
?>
It exported successfully, I got a 'data.sql' file in my directory and the content is exactly what I chose(data with PID from 1 to 49). However whenever I tried to import it back using phpmyadmin into a new table with same name, same columns, in a different database, it keeps showing error
1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near
Full Error message:
SQL query:
2 Austin Benedict Trisanto 002 240000 2015-07-01 1970-01-01 \N 20000 4
3 Faylisha Syema 001 120000 2015-07-01 \N \N 0 2
4 Tiara Aurellia Prasetyo 003 230000 2015-07-01 1970-01-01 \N 0 2
5 Darrell Maximilian Dirgantara 004 210000 2015-07-01 1970-01-01 \N 0 2
6 mikaela reva calista 005 210000 2015-07-01 1970-01-01 \N 0 2
7 Gisella Joycelin Tian Sibarani 006 220000 2015-07-01 1970-01-01 \N 15000 3
8 Putu Pooja Sita Laticia 007 230000 2015-07-01 1970-01-01 \N 0 2
9 Mikhael Lucius Arkananta Prama 008 230000 2015-07-01 1970-01-01 \N 0 2
MySQL said: Documentation
1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '2 Austin Benedict Trisanto 002 240000 2015-07-01 1970-01-01 \N 20000 4
3 Faylish' at line 1
What exactly could be going wrong ? or what I am doing wrong ? should I import it back using php too ?
All answers are greatly welcomed. Thank you in advance.
What you have for import isn't in SQL format.
Additionally, I've never had much luck running it through PHP and phpMyAdmin.
I am only answering this in this fashion, because it is my preferred method of backing up, and restoring databases.
NOTE You have to have shell access for this method.
To back up the database. Use the following:
$ mysqldump -u UserName -p PassWord your_database > /directory/backup_file.sql
Or You can use mysql cli to output a query:
$ mysql -e "select * from yourTable" -u userName -p passWord YourDataBase > /directory/backup_file.sql
To import your file to your new db:
$ mysql -u userName -p passWord newDatabase < /directory/backup_file.sql
With the import, you won't have to specify the table, as that will be stored in the sql file.
Might not be what you are hoping for, but this is a valid alternative to using PHP to do this. And in fact it's the preferred method as it is quick, easy, reliable and effective. There is no code except for your SQL query that is needed. Less code = Less chance of breaking.
Background: Non developer with bash shell scripting knowledge needs to query a READ ONLY DB instance and generate data for given variables.
Data in existing file
A1 B1
A2 B2
......
...
....
An Bn
I would like to connect to the mysql database within the bash script and loop through a select statement "n" number of times
select x, y,z from table tablet_vendors where a=A1 and b=B1
except that A1 and B1 need to be passed as parameters first time, A2 and B2 next time and to n in a for loop. I know the shell side of things, but don't know how to integrate both.
Tried something that can define all possible values for "a" at the beginning of the shell script, but it's way too tedious. Any help is appreciated.
I am open to suggestions on using other scripting languages if they are easy to work with mysql and if you can provide some guidance. My searches are pointing to php and python??
Here's a simpler version:
import MySQLdb as mysql
import csv
con = mysql.connect('localhost','user','password','dbname')
cur = con.cursor()
q = "select x, y,z from tablet_vendors where a=%s and b=%s"
with open('somefile.txt') as f:
rows = csv.reader(f,delimiter='\t')
for row in rows:
cur.execute(q,row)
result = cur.fetchone()
while result:
print result
result = cur.fetchone()
Any errors thrown by MySQL will stop execution. You can wrap the cur. lines in a try/except block to be more elegant.
#!/usr/bin/python
# -*- coding: utf-8 -*-
import _mysql, sys
argumentfile = sys.argv[1]
with open(argumentfile) as f:
sqlarguments = f.readlines()
con = None
username, password = 'testuser', 'testpass'
database = 'testdb'
try:
con = _mysql.connect('localhost', username, password, database)
for line in sqlarguments:
con.query("SELECT x, y,z FROM TABLE tablet_vendors WHERE a='%s' and b='%s;'" % line.split('\t'))
result = con.use_result()
print "MySQL data returned: %s" % \
result.fetch_row()[0]
except _mysql.Error, e:
print "Error %d: %s" % (e.args[0], e.args[1])
sys.exit(1)
finally:
if con:
con.close()
More info on Python and MySQL: http://zetcode.com/db/mysqlpython/
(Skip to "First example" if you've got the MySQL and everything setup)
Note the retrieval of sqlarguments which is taken from a textfile which is passed as a parameter to the script as the first and only parameter like: python2 script.py sqlfile.txt.
Also note that i assumed that the SQL parameters are delimited by \t (tab) and not spaces, which i then pass in as a='...' and b='...' encapsulated in ' in case of spaces.
Having XML Subversion commits log:
<?xml version="1.0"?>
<log>
<logentry
revision="2">
<author>20070299</author>
<date>2012-06-06T05:23:21.999470Z</date>
<msg>add part</msg>
</logentry>
<logentry
revision="1">
<author>20070299</author>
<date>2012-05-22T08:35:55.663875Z</date>
<msg></msg>
</logentry>
</log>
And I want a result as array grouped by date and numbers of commit each day on my site php-javascript. similar to this one:
date[0]=2012-05-22
value[0]=1
date[1]=2012-05-23
value[1]=0
...
date[15]=2012-06-06
value[15]=1
is there a solution to do it?
I consulted this link
But it don't work, non result non error log (apache, php), and i don't know how to send $number[] from PHP code to javascript code
You can play with XPath :
$ xmllint --shell foo.xml
/ > cat //log/*[#revision]/#revision
-------
revision="2"
-------
revision="1"
/ > cat //log/*[#revision]/date/text()
-------
2012-06-06T05:23:21.999470Z
-------
2012-05-22T08:35:55.663875Z
/ >
This is in a shell, but you can iterate on the XPath expressions with any languages you like instead.
! If anyone here knows how to tell XPath to get only the numbers in revision="2", please tell us =)
I realize that you are asking for a Linux solution but since you have not had any proffered here is a way to achieve your goal with PowerShell (and hence Windows-only). Perhaps by presenting this algorithm, another contributor can map it to Linux shell commands for you.
([xml](svn log -v --xml)).log.logentry |
Select-Object -Property `
Author, `
#{n='Revision'; e={([int]$_.Revision)}}, `
#{n='Date'; e={Get-Date $_.Date }}, `
Paths |
Group-Object {$_.Date.ToString("yyyy-MM-dd")} |
Sort-Object name -Descending |
Select-Object #{n='Date'; e={$_.Name}}, Count |
Format-Table -AutoSize
You can likely surmise what the code is doing even without fluency in PowerShell, but here is a summary of key points:
The first line sets up an implicit loop through all log/logentry nodes.
The first Select-Object statement (5 lines) pulls out XML elements and attributes converting them to fields of a PowerShell object; Revision and Date are massaged into appropriate types making them more useful than just as string data.
The Group-Object statement converts what are now true date objects into a format suitable for sorting.
The second Select-Object statement simply modifies the default group names (name and count) to more relevant names (date and count).
The final Format-Table makes the resultant column sizes compact.
Here is a portion of output from my system:
Date Count
---- -----
2012-06-29 4
2012-06-28 6
2012-06-27 8
2012-06-26 16
2012-06-25 1
2012-06-24 1
2012-06-22 5
2012-06-21 8
2012-06-20 8