Besides, the compiled diggers take up very little space (about 20 MB), as well as consume very few computer resources (~ 10-30 MB of RAM and 1-3% of the CPU).<\/p>\n
Compilation service stays free during the beta phase, after the release our paid subscribers keep using compile service for free, while free users shall be able to buy compilation credits.<\/p>\n
How does it work? Everything is straightforward. You create a digger, write a configuration for it (or use a ready-made you already have). As you probably already know, there are three ways to do it: use our Excavator app, write a configuration using our meta-language, or hire our or third-party developer to create a configuration for you. After you create or receive a Digger configuration, save it to the digger. Then launch the digger in debug mode to make sure it works correctly. While digger is in debug mode, resources are free, so you need not worry that you spend all the resources you have. If digger is working correctly and the data collected is in a good state, you can proceed with compilation.<\/p>\n
To do it, go to the digger list, find your digger and click the \u201cOptions\u201d button in the \u201cCompile\u201d column.<\/p>\n
![\"compile1\"](\"https:\/\/blog.diggernaut.com\/wp-content\/uploads\/2016\/10\/compile1.png\")
<\/p>\n
There is a new panel with the name In the left pane, you shall see the screen with compilation settings.<\/p>\n First, you have to choose how digger output the data. You can output to a file or the console. To output data to a file, select the File option in the \u201cOutput Type\u201d field, to output to the console \u2013 StdOut. In the Format field, you must select the format you need. There are currently available 4 types: Excel, CSV, JSON, and XML. Excel and CSV do not support nested data structures, so before you use them, make sure that your data is flat (the root objects has no nested objects, only fields). If you need some other format, please contact us, and we try to add it.<\/p>\n Finally, you must choose a platform, where you want to run your digger, in the last field \u201cPlatform\u201d. Currently, we support Windows, MacOS, and Linux for x86(32bit) and x64(64bit). If you need any other platform, please contact us, and if the compiler supports this platform, we add it as soon as possible.<\/p>\n After you configure the compiler, click the Compile button and in a few dozen seconds you shall see a digger compiled in the table on the right side.<\/p>\n You can download it by clicking on \u201cDownload\u201d link and run it at your computer. A download link is valid for 7 days, then compiled digger is removed.<\/p>\n What happens if compiled scraper stops to work correctly, e.g., if the website changed its structure and data scraped wrongly now? You can always get back to your Diggernaut account, launch the digger in debug mode, see what is wrong, fix it and compile revised version. Alternatively, ask one of our or third-party developers to help you.<\/p>\n Happy scrapping!<\/p>","protected":false},"excerpt":{"rendered":" Do you want to scrape some data from the website, but you are not ready to deploy your web scraper to the cloud and pay for page requests? Then, perhaps, the more appropriate option for you is to download the scraper, run it on your computer or server and get the data in the desired […]<\/p>","protected":false},"author":4,"featured_media":75,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-74","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-web-scraping"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts\/74","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/comments?post=74"}],"version-history":[{"count":5,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts\/74\/revisions"}],"predecessor-version":[{"id":682,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts\/74\/revisions\/682"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/media\/75"}],"wp:attachment":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/media?parent=74"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/categories?post=74"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/tags?post=74"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"compile2\"](\"https:\/\/blog.diggernaut.com\/wp-content\/uploads\/2016\/10\/compile2.png\")
<\/p>\n
\nIf you choose the output to a file, you have to specify the file name in the \u201cOutput File Name\u201d field.<\/p>\n![\"compile3\"](\"https:\/\/blog.diggernaut.com\/wp-content\/uploads\/2016\/10\/compile3.png\")
<\/p>\n