在维基百科上下载的数据,大概是50G的xml数据,现在需要把这数据导入到mysql中,苛求给位大神指导!!!
xml没法直接导入数据库,你需要用程序解析xml,然后组织各种表来插入数据库
可以用python
xml 的value和name得解析出来,完后导入数据库就行了,当然好像sqlite好像更简单
import os
import MySQLdb
rootdir = "e:/20121110"
i = 1
if __name__ == '__main__':
for parent, dirnames, filenames in os.walk(rootdir):
for dirname in dirnames:
print "parent is:" + parent
print "dirname is:" + dirname
for filename in filenames:
print "filename is:" + filename
try:
conn = MySQLdb.connect(host='localhost', user='root', passwd='******', db='lydtest', port=3306)
cur = conn.cursor()
sql = "load data infile 'e:/20121110/" + filename + "' into table taxi fields terminated by ',' enclosed by '\\'' lines terminated by '\\r\\n'";
cur.execute(sql)
conn.commit();
#count = cur.execute('select * from taxi')
print 'the' + str(i) + 'success!'
#print 'there has %s rows record' % count
i = i + 1
cur.close()
conn.close()
except MySQLdb.Error, e:
print "Mysql Error %d: %s" % (e.args[0], e.args[1])
print 'finished!'
你50G的XML.. 一下子也读不进来啊, 只能分批读