Images

Switch to Image Context

Sometimes there is a need to process an image from a website and save the result to the cloud storage or a local drive. To do it, you must switch to image context using the image command. This command only works in block context and requires a base64 encoded image in the register. The following image types are currently supported: jpg, png, gif, webp, tif, bmp.

As a rule, the transition to the context is fairly standard. You need to load the image, go to the block with the encoded image, scrape the contents of the block to the register and switch to the image context:

                        # LOAD IMAGE
- walk:
    to: https://www.diggernaut.com/static/site/images/logo_light_beta.png
    do:
    # FIND THE BLOCK WITH THE BASE64 ENCODED IMAGE
    - find:
        path: imgbase64
        do:
        # SCRAPE THE CONTENT
        - parse
        # SWITCH TO THE IMAGE CONTEXT
        - image:
            do:
            # PROCESS THE IMAGE
                        
                    

Process the Image

While you are in the image context, you can process the image. For example, you can resize and crop the image. The list of available commands and their description is given below:

Command Description
resize resize an image to given size, using given resampling algorithm.
Parameter Description
width The width of the new image, in pixels (if omitted, is set to 0). If 0, then scaling is performed at a given height, keeping the aspect ratio.
height The height of the new image, in pixels (if omitted, is set to 0). If 0, then scaling is done for a given width, keeping the aspect ratio.
resampling algorithm:
lanczos Lanczos (used by default if omitted)
bicubic Bicubic
bilinear Bilinear
box Box
nn Nearest Neighbor
resize_to_fit Resizes the image by fitting it into the specified rectangle, using the specified algorithm, keeping the aspect ratio.
Parameter Description
width The width of the new image, in pixels (if omitted, is set to 0).
height The height of the new image, in pixels (if omitted, is set to 0).
resampling algorithm:
lanczos Lanczos (used by default if omitted)
bicubic Bicubic
bilinear Bilinear
box Box
nn Nearest Neighbor
crop Crops the image to a new image with a given width and height.
Parameter Description
x The coordinate of the origin point for cropping along the X-axis (if omitted, it is set to 0).
y The coordinate of the origin point for cropping along the Y-axis (if omitted, it is set to 0).
width Frame width, in pixels.
height Frame height, in pixels.

Below is an example of how to make a thumbnail of the image with the preservation of the aspect ratio:

                        # LOAD IMAGE
- walk:
    to: https://www.diggernaut.com/static/site/images/logo_light_beta.png
    do:
    # FIND THE BLOCK WITH THE BASE64 ENCODED IMAGE
    - find:
        path: imgbase64
        do:
        # SCRAPE THE CONTENT
        - parse
        # SWITCH TO THE IMAGE CONTEXT
        - image:
            do:
            # RESIZE IMAGE TO THE THUMBNAIL WITH WIDTH OF 50 PX
            - resize:
                width: 50
                        
                    

Save the Image

After processing, we need to save the image. You can save the image back to the register, to a local file on the computer (available only in the compiled digger) or to the cloud storage (currently supported: S3). Save command is used to save the image.

The command supports following parameters:

Parameter Description
ext An extension that defines the type of file being saved. If omitted, the original type and extension are used.                                     The following extensions are currently supported for saving: jpg, png, gif, tif and bmp.
to The type of storage. The following types are currently supported: register, file and s3.

The register type does not use any additional parameters. The image is encoded to Base64 and saved to                         register.

                        # LOAD IMAGE
- walk:
    to: https://www.diggernaut.com/static/site/images/logo_light_beta.png
    do:
    # FIND THE BLOCK WITH THE BASE64 ENCODED IMAGE
    - find:
        path: imgbase64
        do:
        # SCRAPE THE CONTENT
        - parse
        # SWITCH TO THE IMAGE CONTEXT
        - image:
            do:
            # SAVE IMAGE AS JPG TO THE REGISTER
            - save:
                ext: jpg
                to: register
        # NOW WE HAVE AN IMAGE IN THE REGISTER
        # SAVE IT TO THE VARIABLE
        - variable_set: newimage
                        
                    

The file type saves the image to a local drive. This type will work only in compiled scrapers. When using storage of this type, the following parameters are required:

Parameter Description
name Filename without an extension. If not specified, a unique name will be generated.
path A path to the directory where you want to save the file. If not specified, the file will be saved to the current directory.
                        # LOAD IMAGE
- walk:
    to: https://www.diggernaut.com/static/site/images/logo_light_beta.png
    do:
    # FIND THE BLOCK WITH THE BASE64 ENCODED IMAGE
    - find:
        path: imgbase64
        do:
        # SCRAPE THE CONTENT
        - parse
        # SWITCH TO THE IMAGE CONTEXT
        - image:
            do:
            # SAVE IMAGE TO THE FILE (e://myimages/mylogo.png)
            - save:
                to: file
                name: mylogo
                path: 'e://myimages'
                        
                    

The s3 type saves the image to the Amazon S3 cloud storage. When using storage of this type, the following parameters are required:

Parameter Description
key AWS S3 access key. Mandatory.
secret AWS S3 secret. Mandatory.
region AWS S3 region. Mandatory.
bucket AWS S3 bucket name. Mandatory.
token AWS S3 token. Optional.
name Filename without an extension. If not specified, a unique name will be generated.
path A path to the directory where you want to save the file. If not specified, the file will be saved to the root of the bucket.
                        # LOAD IMAGE
- walk:
    to: https://www.diggernaut.com/static/site/images/logo_light_beta.png
    do:
    # FIND THE BLOCK WITH THE BASE64 ENCODED IMAGE
    - find:
        path: imgbase64
        do:
        # SCRAPE THE CONTENT
        - parse
        # SWITCH TO THE IMAGE CONTEXT
        - image:
            do:
            # SAVE IMAGE TO THE S3 STORAGE (/logos/mylogo.png)
            - save:
                to: s3
                key: AWSAJJDJJSJDJDJFK
                secret: AWSSERETTDHFJJJDJSKFJFJSJJFJJGKRI
                region: us-east-1
                bucket: mybucket
                name: mylogo
                path: '/logos'
                        
                    

Please note that when saving images to cloud storages a bandwidth quota is used that corresponds to your subscription plan. For example, on a free plan, the quota is 10 megabytes per month. Upon reaching the quota, the images will no longer be stored in the cloud storage. Also, the images will not be saved to cloud storage if your digger is in debug mode.

Now its time to find out how to work with binary files.