[問題] 公開資訊觀測站股東會爬蟲

看板Python作者ccherry225 (貓咪QQ)時間2年前 (2021/08/14 22:16)推噓3(3推 0噓 12→)

留言15則, 4人參與討論串1/1

各位大神們好！我在爬公開資訊觀測站的資料遇到一個問題，已經卡了我2天QQ 就是我想要抓取股東會的開會日期跟辦理過戶日期的時間。爬蟲的程式碼，輸入股票代碼跟跳出新的視窗顯示文件都沒問題，但是要抓下開會日期跟停止過戶日期都抓不到。使用 tbody =driver.find_element_by_xpath(' //*[@id="table01"]/center/table[3]/ tbody') 然而 tbody.text都只顯示 '召集受益人大會公告'，以致我自訂函數抓日期都抓不到。不曉得是有哪邊出錯了，或是有什麼相關網址可以參考，再麻煩各位善心的大大們回答了！謝謝！再次感謝大家我的電腦是 Mac 使用的是 Anaconda 下的 Spyder 有參考 https://medium.com/smartalpha/使用-selenium-firefox-動態爬取公開觀測站資料-eb84e7b034c 程式碼如下： from openpyxl import * # for excel from selenium import webdriver # open browser from selenium.webdriver.support.ui import Select from bs4 import BeautifulSoup # 解析html import time #休息時間 import requests driver = webdriver.Firefox(executable_path = '/usr/local/bin/geckodriver') url = 'https://mops.twse.com.tw/mops/web/t108sb16_q1' driver.get(url) # 前往這個網站 data_list = [] data_dict = {} ticker = '2330' try: print(ticker,"Start searchinh!") keyword = driver.find_element_by_id("co_id") keyword.clear() keyword.send_keys(ticker) search_button = driver.find_element_by_css_selector("#search_bar1 > div > input[type=button] ") search_button.click() time.sleep(2) except: driver.quit() try: go_to_data_page_button = driver.find_element_by_xpath( "/html/body/center/table/tbody/tr/td/div[4]/table/tbody/tr/td/div/tabl e/tbody/tr/td[3]/div/div[5]/div/center/form/table[2]/tbody/tr[2]/td[5]/input" ) go_to_data_page_button.click() except Exception as e: print("查無：",ticker) def find_between(s,first,last): try: start = s.index(first)+len(first) end = s.index(last,start) return s[start:end] except ValueError: return "" def analyze_text(text): shareholders_meeting_date = find_between(tbody.text,"開會日期：","").strip () shareholders_meeting_date = shareholders_meeting_date.replace("年",'/') shareholders_meeting_date = shareholders_meeting_date.replace("月",'/') shareholders_meeting_date = shareholders_meeting_date.replace("日",'') shareholders_last_transfer_date = find_between(tbody.text,"辦理過戶日期時間：","").strip() shareholders_last_transfer_date = shareholders_last_transfer_date.replace( "年",'/') shareholders_last_transfer_date= shareholders_last_transfer_date.replace(" 月",'/') shareholders_last_transfer_date = shareholders_last_transfer_date.replace( "日",'') return [shareholders_meeting_date,shareholders_last_transfer_date ] tbody = driver.find_element_by_xpath(' //*[@id="table01"]/center/table[3]/tbod y') analyze_text(tbody.text) data_list.append([ticker, analyze_text(tbody.text)]) data_analyzed = analyze_text(tbody.text) data_dict[ticker] = data_analyzed print(ticker,":",data_analyzed) driver.close() -- ※ 發信站: 批踢踢實業坊(ptt.cc), 來自: 180.217.46.235 (臺灣) ※ 文章網址: https://www.ptt.cc/bbs/Python/M.1628950605.A.401.html

推

a367560

08/15 00:39, 2年前 , 1^F

08/15 00:39, 1^F

→

a367560

08/15 00:40, 2年前 , 2^F

08/15 00:40, 2^F

→

a367560

08/15 00:41, 2年前 , 3^F

08/15 00:41, 3^F