Welcome to MLink Developer Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
292 views
in Technique[技术] by (71.8m points)

关于代码的优化问题

我新手写的代码,用来处理爬虫下来的htm文件内容,虽然解决问题,但是会有遗漏文件不处理。爬虫是爬一些文章的网站下来的,和网页另存为没什么区别。

想大神们帮我看看我的代码,怎么优化不会有遗漏。比较小白的代码,麻烦了!!!

# -*- coding: utf-8 -*
import re
import glob

filename_list = glob.glob('*.html')

for i in filename_list:
    txt = ""
    with open(i, "r") as htmfile:
        txt = htmfile.read()
    scdy = r"<hr[sS]*?<hr"
    onedotxt = re.findall(scdy, txt)
    if onedotxt:
        r = onedotxt[0]
    twotxt=re.sub('<[^>]*>', '', r) 
    threetxt=re.sub('<hr', '', twotxt)
    fourtxt=re.sub('&#8217;', '', threetxt)
    fivetxt=re.sub('&#8221;', '"', fourtxt)
    sixtxt=re.sub('&#8220;', '"', fivetxt)
    
    endstr=re.sub('&#8211;', '-', sixtxt)
    name = endstr.split('
')[1]
    with open(name+".txt", "w") as wf:
            wf.write(endstr)

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

filename_list = glob.glob('.html') + glob.glob('.htm')


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to MLink Developer Q&A Community for programmer and developer-Open, Learning and Share

1.2m questions

2.1m answers

5 comments

56.5k users

...