Saijo George

Curated by Saijo George

Read more

thursday27 Feb 2020

Googles’ reverse IP check can be easily circumvented by X-Forwarded-For Header Response for Some Sites

https://youtu.be

If some sites are blocking your Googlebot user-agent crawls (Screaming Frog or other crawlers) you might be able to circumvent it by passing off a known Google IP using the X-Forwarded-For Header.

Props to Aarne Salminen for the find, I tried this using Screaming Frog and it does work for some sites. Aarne believes this is a limitation of the AWS platform. Enjoy it while it lasts.

Tips and Tricks


Saijo's tl;dr marketing newsletter has a perfect balance. You are sure to get the best SEO news without being overwhelmed by them.

Facebook Has a Pixel Scorecard That Shows You If You FB Pixel Has Any Issues 1 - Social Media News

Jean-Christophe Chouinard Sr. SEO Specialist Seek