Mikhail Sisin Co-founder of cloud-based web scraping and data extraction platform Diggernaut. Over 10 years of experience in data extraction, ETL, AI, and ML.

Ecommerce scraper for Adidas Hardware online store

8 min read

Ecommerce scraper for Adidas Hardware online store

Adidas Hardware is a store that has large range selection of equipment for sport activities. It is the gym equipment, equipment for practicing yoga, as well as other equipment for fitness, produced by the German company Adidas. This ecommerce scraper is designed to collect information about the products sold on the adidashardware.com website.

Approx number of goods: 200
Approx number of page requests: 200
Recommended subscription plan: Free

PLEASE NOTE! The number of requests can exceed the number of products, because data about variations, images, etc. can be scraped from other resources and will require additional requests. Also part of the product data can be delivered using XHR requests, which also increases the total number of required page requests.

How to use the ecommerce scraper to extract data about goods and prices

To use the ecommerce scraper for Adidas Hardware store’s website, you must have an account with our Diggernaut service. You can just simply follow this comprehensive guide:

  1. Go through this registration link to open free account with Diggernaut
  2. After registering and confirming the email address, you will need to log in to your account
  3. Create a project with any name and description, if you do not know how to do it, please refer to our documentation
  4. Switch to the created project and create a digger with any name, if you do not know how to do it, please refer to our documentation
  5. Copy the following digger configuration to the clipboard and paste it into the digger you created, if you do not know how to do it, refer to our documentation
  6. Switch the mode of the digger from Debug to Active, if you do not know how to do it, please refer to our documentation
  7. Run your digger and wait until the completion, if you do not know how to do it, please refer to our documentation
  8. Download the scraped dataset in the format you need, if you do not know how to do it, please refer to our documentation

You can also setup a schedule for running your scraper and collect data regularly.

Scraping configuration for the digger

---
config:
    debug: 2
    agent: Firefox
do:
- link_add:
    pool: main
    url: 
        - https://www.adidashardware.com/training
        - https://www.adidashardware.com/recovery
        - https://www.adidashardware.com/strength
        - https://www.adidashardware.com/yoga
        - https://www.adidashardware.com/benches
        - https://www.adidashardware.com/cardio
- walk:
    to: links
    pool: main
    do:
    - find:
        path: script:matches(Wix Stores)
        do:
        - parse
        - space_dedupe
        - trim
        - filter:
            args:
                - var\s*rendererModel\s*=\s*(.+)\s*;\s*var\s*publicModel
        - normalize:
            routine: json2xml
        - to_block
        - find:
            path: clientspecmap > *:hasChild(appdefinitionname:matches(Wix Stores))  instance
            do:
            - parse
            - variable_set: instance
    - find: 
        path: body 
        do: 
        - static_get: url
        - normalize:
            routine: replace_substring
            args:
                \s*$: ?_escaped_fragment_=
        - walk:
            to: value
            do:
            - find:
                path: meta[property="og:title"]
                do:
                - parse:
                    attr: content
                    filter:
                        - \s*\|\s*(.+)
                - normalize:
                    routine: lower
                - variable_set: cats
            - find:
                path: li[itemprop="itemListElement"] h3[itemprop="name"] > a
                do:
                - variable_clear: uri
                - parse:
                    attr: href
                - normalize:
                    routine: url
                - variable_set: uri
                - variable_clear: ur
                - filter:
                    args:
                        - product-page\/(.+)
                - variable_set: ur
                - walk:
                    to: https://ecom.wix.com/storefront/product/<%ur%>?instance=<%instance%>&locale=en
                    do:
                    - object_new: product
                    - find: 
                        path: head 
                        do: 
                        - variable_get: cats
                        - object_field_set:
                            object: product
                            field: categories
                        - register_set: Adidas
                        - object_field_set:
                            object: product
                            field: brand
                        - eval:
                            routine: js
                            body: '(function (){var d = new Date(); return d.toISOString()})();'
                        - object_field_set:
                            object: product
                            field: date
                        - variable_get: uri
                        - object_field_set:
                            object: product
                            field: url
                    - find: 
                        path: script:matches(eCom.eComAppConfig) 
                        do: 
                        - parse
                        - space_dedupe
                        - trim
                        - filter:
                            args:
                                - eCom\.eComAppConfig\(\'productPageApp\'\,\s*(.+),\s*\'\/\/
                        - normalize:
                            routine: json2xml
                        - to_block
                        - find:
                            path: body_safe
                            do:
                            - find: 
                                path: product > description 
                                do: 
                                - parse
                                - space_dedupe
                                - trim
                                - normalize:
                                    routine: replace_substring
                                    args:
                                        \\s*\<\/p\>: ''
                                - register_set: 
<%register%>
- to_block - find: path: div do: - attr_remove: selector: '*' - parse - object_field_set: object: product field: description - find: path: product > media:hasChild(mediatype:matches(PHOTO)) > url do: - parse - register_set: https://static.wixstatic.com/media/<%register%> - object_field_set: object: product field: images joinby: "|" - find: path: product > name do: - parse - space_dedupe - trim - object_field_set: object: product field: name - find: path: product options:hasChild(optiontype:matches(COLOR)) description do: - parse - space_dedupe - trim - object_field_set: object: product field: variations joinby: "|" - find: path: product > sku, product > managedproductitems > sku do: - parse - space_dedupe - trim - if: match: (\S{3}) do: - object_field_set: object: product field: sku - find: path: product > prices > price do: - parse: filter: - (\d+\.?\d*) - object_field_set: object: product field: price type: float - register_set: GBP - object_field_set: object: product field: currency - object_save: name: product

Sample of scraped data

Below is a sample of a dataset with several products in JSON format (so you can easily review it and see data structure). The dataset can be downloaded as CSV, XLSX, XML, or any other text format using the templates.

[{
    "product": {
        "brand": "Adidas",
        "categories": "training",
        "currency": "GBP",
        "date": "2017-12-05T01:55:23.339Z",
        "description": "Push-ups are a great exercise to build upper body and core strength, typically performed on the floor. Simple yet effective, they can be incorporated into any workout, anywhere.В Using push-up bars can offer further training benefits beyond the no-equipment push-up. The adidas Push Up Bars are particularly beneficial if you suffer from wrist discomfort when performing your usual push-ups. The apparatus enables you to exercise free of pain and lets you lower yourself closer to the floor which increases the muscle work output required.В The handles are two separate pieces of equipment, two platforms to grip with each hand, allowing you to change the position and direction of your wrists.",
        "images": "https://static.wixstatic.com/media/253a43_564d703bc0eb4b64b330061248d38d05~mv2.jpg|https://static.wixstatic.com/media/253a43_22fdd8de4d794e6ababa9c14c6fe08d5~mv2.jpg|https://static.wixstatic.com/media/253a43_3c7319a7ff244f7ba209d18d226e0ce1~mv2.jpg|https://static.wixstatic.com/media/253a43_cdce72a8f5e54f7e89dfef10ee3be790~mv2.jpg|https://static.wixstatic.com/media/253a43_014bb45c0359428397167cde46b50dec~mv2.jpg|https://static.wixstatic.com/media/253a43_56b56b6e50ec4a569c99dca38d1c454b~mv2.jpg",
        "name": "PUSH UP BARS",
        "price": 18,
        "sku": "ADAC-12231",
        "url": "https://www.adidashardware.com/product-page/push-up-bars"
    }
}
,{
    "product": {
        "brand": "Adidas",
        "categories": "training",
        "currency": "GBP",
        "date": "2017-12-05T01:55:24.397Z",
        "description": "How do you improve your push-up? Push-ups are a great exercise to build upper body and core strength. Simple yet effective, they can be incorporated into any workout, anywhere.В The adidas Swivel Push Up Bars enhance and add variety in performing this staple body weight exercise, transforming the ordinary push-up into an extraordinary push-up.В By adding a unique swivel action, the swivel push-up bars enable you to lock out your wrist and rotate as you lower to the push-up position. This calls upon more chest, shoulder and back muscle than a regular push up meaning bigger, better results in less time. As it's a natural movement, it actively reduces strain on the elbow and wrist joints and the smooth bearing discs help make it one smooth rotation. Alternatively the bars can be locked in position for traditional static push-ups.В Sold as a pair.",
        "images": "https://static.wixstatic.com/media/253a43_fde5f2831d384cf3831f27455c7d9ca4~mv2.jpg|https://static.wixstatic.com/media/253a43_0362df24175a4310b6efdec8e89d52e3~mv2.jpg|https://static.wixstatic.com/media/253a43_aabf3e7a045b4d93b77d3f1a5f85dc55~mv2.jpg",
        "name": "SWIVEL PUSH UP BARS",
        "price": 55,
        "sku": "ADAC-11401",
        "url": "https://www.adidashardware.com/product-page/swivel-push-up-bars"
    }
}
,{
    "product": {
        "brand": "Adidas",
        "categories": "training",
        "currency": "GBP",
        "date": "2017-12-05T01:55:24.631Z",
        "description": "Developing core strength is imperative if you want to become stronger in all other aspects of your training, especially weight training.В Designed to engage the core muscles and strengthen the upper body, the robust and sturdy set of two adidas Core Rollers feature ergonomic hand grips to ensure you can perform your exercises with confidence.В The rolling motion of the equipment triggers hard to target muscles through a variety of exercises such as oblique and pike roll-outs. Having a roller to each hand adds a further level of intensity than using the traditional singular adidas ab wheel.",
        "images": "https://static.wixstatic.com/media/253a43_0452658988ca473e911571421f12317c~mv2.jpg|https://static.wixstatic.com/media/253a43_090f61654ade4e8595175aded4429b27~mv2.jpg|https://static.wixstatic.com/media/253a43_18b97d12fa4244d3bfd477efb0b6b6ac~mv2.jpg|https://static.wixstatic.com/media/253a43_fd0e3755d0494363a5ba0ae4d5982671~mv2.jpg|https://static.wixstatic.com/media/253a43_afdef9f3b5754d7880886ba6af913bfb~mv2.jpg",
        "name": "CORE ROLLERS",
        "price": 38,
        "sku": "ADAC-11604",
        "url": "https://www.adidashardware.com/product-page/core-rollers"
    }
}
,{
    "product": {
        "brand": "Adidas",
        "categories": "training",
        "currency": "GBP",
        "date": "2017-12-05T01:55:24.870Z",
        "description": "The 'roll out' is considered by experts to be one of the most productive (and challenging) abdominal exercises in the gym.В Originally performed by 'rolling out' on a gymball, the adidas Ab Wheel improves and optimises the movement by placing you closer to the floor - this means that every muscle involved in stability and dynamic strength is called in to action.В The first time you roll out, do it on your knees and keep the wheel under your body. As you improve, you can lift your knees off the ground and push the roller further in front of you - at this point your entire bodyweight is held through your abdominals making this one of the most targeted ways of working those particular muscles.В Doing 1000s of sit ups won't give you flat abs - to achieve that you have to work the abdominal muscles as an integrated part of your torso. Roll outs are challenging but highly productive.В Adidas Ab wheel is built to deliver results no matter if you are training for MMA, team sports or integrating it in to your WOD (workout of the day).",
        "images": "https://static.wixstatic.com/media/253a43_40da746d0b0646a9a6d36eb41e532d83~mv2.jpg|https://static.wixstatic.com/media/253a43_d366efdbfe1243b5b47391acd9c2e209~mv2.jpg|https://static.wixstatic.com/media/253a43_ff437ac65cd74e778ebb6e265a19773b~mv2.jpg",
        "name": "AB WHEEL",
        "price": 18,
        "sku": "ADAC-11404",
        "url": "https://www.adidashardware.com/product-page/ab-wheel"
    }
}
,{
    "product": {
        "brand": "Adidas",
        "categories": "training",
        "currency": "GBP",
        "date": "2017-12-05T01:55:25.132Z",
        "description": "Your body is only as strong as the weakest link, so small muscles in your forearms, hands and wrists play an essential role in any exercise that involves lifting weights.В Grip exercises can lead to increased forearm muscularity, hand strength, endurance and dexterity, which in turn leads to better weight training performance.В The pair of adidas Professional Grip trainers feature angled springs which allow for graded resistance to be used to train for hand strength and grip endurance. Take them with you anywhere and build up your grip on the go.",
        "images": "https://static.wixstatic.com/media/253a43_68cee7adecfb4cfab46b0c8b4f216793~mv2.jpg",
        "name": "PROFESSIONAL GRIP TRAINERS",
        "price": 12,
        "sku": "ADAC-11400",
        "url": "https://www.adidashardware.com/product-page/professional-grip-trainers"
    }
}
,{
    "product": {
        "brand": "Adidas",
        "categories": "training",
        "currency": "GBP",
        "date": "2017-12-05T01:55:25.373Z",
        "description": "The adidas Sand Bag enables you to add rotation and resistance to your workouts and perform both single and two handed moves. The bag encourages core muscle activation when held by any of its five grab handles, enabling you to push, pull and even throw the bag to engage multiple muscle groups in every repetition.В Doubling up as a kettlebell, barbell or dumbbell for fast dynamic moves, the adidas Sand Bag has been packed with a double skin liner to ensure it doesn't become unstable or 'sloppy'.",
        "images": "https://static.wixstatic.com/media/253a43_781371b95a404d5995cf58ad82a736d4~mv2.jpg|https://static.wixstatic.com/media/253a43_95863f333eb942539a9b162bf80edaf5~mv2.jpg",
        "name": "SAND BAG",
        "price": 60,
        "sku": "ADAC-11505",
        "url": "https://www.adidashardware.com/product-page/sand-bag"
    }
}
,{
    "product": {
        "brand": "Adidas",
        "categories": "training",
        "currency": "GBP",
        "date": "2017-12-05T01:55:25.601Z",
        "description": "Suspended bodyweight training is one of the latest, most radical and innovative types of exercise you can do to improve strength, power, balance and agility. The adidas 36Zero Suspension Trainer is the perfect piece of fitness equipment to work your entire body without the need for heavy lifting. Utilising a series of different attachments from rings, pads, ab straps and handles, you can perform a multitude of upper and lower body exercises. Its versatile anchor enables you to attach both horizontally and vertically. The hand grips and foot straps are quickly and easily exchanged for rapid effective muscle conditioning and HITT (High Intensity Interval Training). Plus the incorporated ab straps can be used for intense targeted abdominal exercises. The 36Zero training plan has four phases. To make the plan safe, effective, and fun, each phase should be completed in the intended order. Phase 1Phase one introduces fundamental movements to help you build a strong, stable torso—the foundation of all movement to improve your ability to move and perform efficiently. This will be particularly helpful in later phases when workouts become more challenging. Phase 2Everyone has a dominant side of their body. Movement progressions in this phase challenge the right and left sides of your body independently. This helps each side move more efficiently on its own and improves how your entire body moves by decreasing muscle imbalances. Phase 3The third part of this plan challenges the solid foundation built in the previous phases with diverse movements that train your body from every angle. This phase helps solidify your body’s stability and strength Phase 4In the final phase, workouts will challenge you to move more quickly from one exercise to the next and push yourself to maintain proper form the entire time. This phase will help you increase your muscular and cardiovascular endurance",
        "images": "https://static.wixstatic.com/media/253a43_d1b166a1663049ab9b69faba554376a9~mv2.jpg|https://static.wixstatic.com/media/253a43_4771694944254280a987738adae68365~mv2.jpg|https://static.wixstatic.com/media/253a43_d7681fc6de614b6997f511b39951e8df~mv2.jpg|https://static.wixstatic.com/media/253a43_d7cfb67fe7854ae68165a0b08187bec2~mv2.jpg",
        "name": "36ZERO TRAINER",
        "price": 120,
        "sku": "ADAC-12250",
        "url": "https://www.adidashardware.com/product-page/36zero-trainer"
    }
}
]
Mikhail Sisin Co-founder of cloud-based web scraping and data extraction platform Diggernaut. Over 10 years of experience in data extraction, ETL, AI, and ML.

Leave a Reply

Your email address will not be published. Required fields are marked *