道客巴巴爬虫（道客巴巴爬虫代码）

道客巴巴爬虫（道客巴巴爬虫代码）

使用xpathhelp控件

import requests, re, json, pandas as pd, timefrom selenium import webdriver # selenium2.48.0 支持phantomjsfrom lxml import etreeimport timeimport os, time# 页 文件 = webdriver.PhantomJS(executable_path=r'C:\Users\wang\Desktop\phantomjs-2.1.1-windows (1)\bin\phantomjs.exe')file_urls_list=[]for i in range(1,30,1): time.sleep(3) url = " driver.get(url=url) tree = etree.HTML(driver.page_source) file_urls = tree.xpath(".//h3[@class='sd-type-title']/a/@href") file_urls=[ "for i in file_urls ] file_urls_list.extend(file_urls) print(file_urls)with open("url.txt","w",encoding="utf-8") as f: for i in file_urls: if len(i)==len(" f.write(i) f.write("\n")f.close()

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。