Automated CloudFlare challenge solution with Golang

There is new version of Surf library for Golang has been pushed. This version is able to bypass fresh version of CloudFlare protection. We are using this library in our engine so our users can feel all benefits. Library bypass protection in automated mode, so you dont need to do anything extra. You just loading page as usually, and if there is CloudFlare challenge, it will be resolved and you get content of page you requested.

You are free to use Surf library from our repo for your projects, its under MIT licence and actually forked from headzoo/surf. But we are using own version that fit needs of our web scraping engine.

How to test if its actually works. You can test it by loading this sample page. This site is under CloudFlare. Lets try to use following digger config to get this page and extract website URL:

Data we get will looks like:

Co-founder of cloud based web scraping and data extraction platform Diggernaut

Leave a Reply

Your email address will not be published. Required fields are marked *