The Digger can be started in the active or debug mode (for more details about the modes, see Edit Digger session of the documentation). The digger operating mode is set in your account on the website for each digger separately. While working in debug mode, you do not spend your resources (page requests), but you can not download data, only view the first 20 entries of scraped data. In debug mode you will get more detailed logs about the process of execution of the digger, which will allow you to debug the parsing logic of your diggers in case of problems.
At the moment, 3 levels of debug mode are supported:
- The default level (0) allows you to see in the logs all the operations performed by the digger and state of the register and variables.
- Level 1 also allows you to see the contents of the blocks when the digger enters them by the command Find.
- Level 2 allows you to additionally view the contents of the downloaded pages when the digger loads them with the command Walk.
To set the debug level other than 0, you can use the debug parameter in the settings section of the configuration:
# SETTING DEBUG LEVEL 1 - config: debug: 1 # SETTING DEBUG LEVEL 2 - config: debug: 2