抓取工具 - list crawlers

搜索引擎营销屏幕快照监控信息流链接检查抓取工具工具爬虫速度测试漏洞、病毒检测未分类 AI 爬虫

抓取工具爬虫列表

名称	所属公司	最后一次出现
Fake bot		2026-01-22 22:25:34
Googlebot	谷歌有限公司 Google（中文译名：谷歌）是总部位于美国加州门洛帕克的跨国科技公司，为Alphabet Inc.的子公司，业务范围涵盖互联网广告、互联网搜索、云计算等领域，开发并提供大量基于互联网的产品与服务。	2026-01-22 22:22:42
AliyunSecBot	Alibaba Cloud Computing Ltd.	2026-01-19 20:17:46
GoParserBot		2026-01-06 02:09:27
HuaweiCrawler		2026-01-06 01:55:57
CMS-Checker		2025-12-27 01:40:28
Diffbot	Diffbot Technologies Corp.	2025-12-27 00:33:48
got	Sindre Sorhus（个人）	2025-12-25 07:34:42
SEBot-WA		2025-12-19 00:55:57
TestBot		2025-10-21 01:43:07
seoscanners.net bot		2025-10-10 08:31:40
BitSightBot		2025-09-11 20:24:05
SkyworkSpider		2025-09-05 03:27:49
DnBCrawler-Analytics		2025-08-28 02:22:42
sempi.tech bot		2025-08-03 09:41:18
Headless Chrome		2025-08-03 08:21:15
spider		2025-08-02 03:20:09
newspaper		2025-07-19 03:37:09
Scrapy	Zyte	2025-05-22 02:50:06
AccompanyBot		2025-04-12 06:23:38
Orbbot		2025-03-11 09:59:02
chimebot		2025-02-22 02:31:40
FlfBaldrBot		2024-07-15 10:07:00
PHPCrawl	Uwe Hunfeld（个人）德国	2024-06-15 03:12:57
TMM Crawler		2024-06-15 01:21:35
dcrawl	Kuba Gretzky（个人） Kuba Gretzky 从事网络安全行业已有 10 多年。进攻性安全软件开发是其爱好。近年来，开发了多种私有和开源安全工具，得到了全球主要红队公司的认可。	2024-05-30 07:32:17
BOT/0.1		2024-05-08 02:45:41
thesis-research-bot		2023-11-08 20:26:29
my-tiny-bot		2023-11-06 05:08:21
TinyTestBot		2023-09-26 06:43:37
clark-crawler		2023-07-15 03:46:41
The Knowledge AI		2022-11-01 02:13:38
Nicecrawler		2022-11-01 01:57:20
Elisabot		2022-10-31 08:37:01
centurybot9		2022-10-31 08:16:45
seolyt		2022-10-31 04:17:44
crawler eb germany		2022-10-27 03:28:12
Spiderbot-Nutch		2022-09-22 01:26:15
DF Bot		2021-08-22 06:38:55
MauiBot		2021-04-09 21:39:57
Monibot		None
AdsTxtCrawler		None
Storm-crawler		None
DownloaderChrome		None
MetaInspector	Jaime Iniesta （个人）	None
YFF35		None
LanaiBot		None
TprAdsTxtCrawler		None
GoodBot		None
Snap URL Preview Service		None
1M7bot		None
Aream Crawler		None
AAABot		None
AnyBot		None
KauaiBot		None
Splash	Zyte	None
voltron	80legs	None
trafilatura	Adrien Barbaresi	None
CF-UC User Agent		None
aranhabot		None
KauaiSlowBot		None
PyWebCopyBot	Raja Tomar	None