No Proxy¶
The darc.proxy.null
module contains the auxiliary functions
around managing and processing normal websites with no proxy.
-
darc.proxy.null.
fetch_sitemap
(link)¶ Fetch sitemap.
The function will first fetch the
robots.txt
, then fetch the sitemaps accordingly.- Parameters
link (darc.link.Link) – Link object to fetch for its sitemaps.
-
darc.proxy.null.
save_invalid
(link)¶ Save link with invalid scheme.
The function will save link with invalid scheme to the file as defined in
PATH
.- Parameters
link (darc.link.Link) – Link object representing the link with invalid scheme.
-
darc.proxy.null.
PATH
= '{PATH_MISC}/invalid.txt'¶ Path to the data storage of links with invalid scheme.
See also