The source – developers.google.com
Google has updated the DuplexWeb service – the user agent that powers the Duplex on the Internet service. The updated user-agent uses the latest version of Android and Chrome.
Features of work and frequency of scanning:
- No service using the DuplexWeb-Google user agent makes purchases or other important actions when crawling a site.
- DuplexWeb-Google performs crawling from several times a day to several times an hour – depending on the training function, but in such a way as not to overload your site and not disrupt its traffic.
- The data received by this user-agent is not used for indexing in Google Search. Therefore DuplexWeb-Google does not recognize the noindex directive.
- Google Analytics reports do not capture page requests from DuplexWeb-Google during crawling and analysis.
If you do not want the DuplexWeb-Google user agent to crawl your pages, you need to explicitly block access to them using the Disallow directive in your robots.txt file. Disabling training in the property settings in Search Console isn't enough.