This Python module detects if a given HTTP User Agent is a web crawler.
It uses the list of registered robots from the robotstxt.org.
.
This package contains the Python 3 version of the library.
Installed Size: 35.8 kB
Architectures: all