This web tool (apify.com) is very useful.
For example，I have used this tool crawl the website：https://eips.ethereum.org。
Finally，the process of the crawl as the below image：
Result can be viewed use the json link：
https://api.apify.com/v2/datasets/7kuYat4C68IFy0EPe/items?clean=true&format=json