Huaweis AspiegelBot / PetalBot Goes on a Crawling and Indexing Spree
https://www.hypernode.comHuawei, the Chinese telecom and smartphone giant, has launched their own search service as they try to bounce back from being blocked from the Google ecosystem. Over the past two months, the Huawei AspiegelBot (now called PetalBot) has been crawling and indexing websites worldwide to gather information for Huawei’s (currently in beta) search engine app.
The activities of the AspiegelBot since bots, crawlers and unwanted traffic can have a negative impact on the performance of online stores. On our Hypernode platform, the traffic of the bot increased with 460% over the past month! There are also some users complaining that it does NOT respect robots.txt, I haven’t tested it so I am not sure about it.
| Product name | User-Agent | Full User-Agent |
| PetalBot(PC) | PetalBot | Mozilla/5.0 (compatible;PetalBot;+https://aspiegel.com/petalbot) |
| PetalBot(mobile) | PetalBot | Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://aspiegel.com/petalbot) |
