# 9. 動態網頁爬蟲

有些時候前面章節的方法可能都不管用, 或是因為你很懶, 看到要先去分析API或是網頁結構就懶了, 並希望爬蟲可以直接模擬你操作網頁的方式來爬資料, 這時候就可以採用這邊的方法了. 這種不去模擬瀏覽器取得資料, 而是直接使用瀏覽器的方式, 通常都會用到知名的瀏覽器自動化操作工具 --- Selenium.

要在Python中使用Selenium, 除了要安裝selenium library之外(本書一開始的requirements.txt已經安裝好了), 還要去官網下載對應瀏覽器的webdriver([ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver/downloads)). Webdriver除了官網上提供的, 也可以使用PhantomJS, 如此一來, 就可以在背景模仿瀏覽器的行為, 可以在一定程度上加快程式的速度.

Selenium可以做到很多事情, 舉凡網頁元件的定位, 點擊按鈕, 選擇選單, 輸入文字等等...都可以透過程式化的方式做到.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://clu.gitbook.io/python-web-crawler-note/e04.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.