{"id":393,"date":"2018-02-11T01:24:32","date_gmt":"2018-02-11T01:24:32","guid":{"rendered":"https:\/\/www.diggernaut.com\/blog\/?p=393"},"modified":"2019-01-12T15:44:49","modified_gmt":"2019-01-12T15:44:49","slug":"extracting-price-product-information-calypso-st-barth-online-store","status":"publish","type":"post","link":"https:\/\/www.diggernaut.com\/blog\/extracting-price-product-information-calypso-st-barth-online-store\/","title":{"rendered":"Extracting price and product information from Calypso St Barth online store"},"content":{"rendered":"

Since the launch of its first store in 1992, Calypso St. Barth grew out of a modest boutique of resort clothes to a luxurious, recognizable worldwide brand. Designers, inspired by the travels, culture and art of different countries, create amazing and feminine models that women of all ages like. If you like a mixture of light elegance and exotic ideas, you will definitely enjoy the assortment of this online store. Our free scraper can be used for extracting price and product information from Calypso St Barth online store. Also it let you extract product images.<\/p>\n

Approx number of goods:<\/strong> 2000
\nApprox number of page requests:<\/strong> 2000
\nRecommended subscription plan:<\/strong> Free<\/p>\n

PLEASE NOTE!<\/strong> The number of requests can exceed the number of products, because data about variations, images, etc. can be scraped from other resources and will require additional requests. Also part of the product data can be delivered using XHR requests, which also increases the total number of required page requests.<\/p>\n

How to use the web scraper to extract data about goods and prices from calypsostbarth.com<\/h3>\n

To use the web scraper for Calypso St Barth store\u2019s website, you must have an account with our Diggernaut service. You can just simply follow this comprehensive guide:<\/p>\n

    \n
  1. Go through this registration link<\/a> to open free account with Diggernaut<\/a><\/li>\n
  2. After registering and confirming the email address, you will need to log in to your account<\/a><\/li>\n
  3. Create a project with any name and description, if you do not know how to do it, please refer to our documentation<\/a><\/li>\n
  4. Switch to the created project and create a digger with any name, if you do not know how to do it, please refer to our documentation<\/a><\/li>\n
  5. Copy the following digger configuration to the clipboard and paste it into the digger you created, if you do not know how to do it, refer to our documentation<\/a><\/li>\n
  6. Switch the mode of the digger from Debug to Active, if you do not know how to do it, please refer to our documentation<\/a><\/li>\n
  7. Run your digger and wait until the completion, if you do not know how to do it, please refer to our documentation<\/a><\/li>\n
  8. Download the scraped dataset in the format you need, if you do not know how to do it, please refer to our documentation<\/a><\/li>\n<\/ol>\n

    You can also setup a schedule for running your scraper and collect data regularly.<\/p>\n

    Scraping configuration for the digger<\/h3>\n
    ---\nconfig:\n    debug: 2\n    agent: Firefox\ndo:\n- walk:\n    to: http:\/\/www.calypsostbarth.com\/new-arrivals\/clothing\n    do:\n    - find: \n        path: .col-side > ul > li \n        do: \n        - variable_clear: cat1\n        - variable_clear: cat2\n        - variable_clear: cat3\n        - find: \n            path: a\n            slice: 0\n            do: \n            - parse\n            - space_dedupe\n            - trim\n            - normalize:\n                routine: lower\n            - variable_set: cat1\n        - parse:\n            format: html\n        - register_set: \n
    • <\/ul><\/div>\n        - to_block\n        - find: \n            path: div > ul > li > ul > li \n            do: \n            - variable_clear: cat2\n            - variable_clear: cat3\n            - find: \n                path: a\n                slice: 0\n                do: \n                - parse\n                - space_dedupe\n                - trim\n                - normalize:\n                    routine: lower\n                - variable_set: cat2\n                - parse:\n                    attr: href\n                - normalize:\n                    routine: url\n                - pool_clear: default\n                - parse:\n                    attr: href\n                - link_add\n                - walk:\n                    to: links\n                    do:\n                    - find: \n                        path: .paging  li > a:matches([Nn]ext) \n                        do: \n                        - parse:\n                            attr: href\n                        - link_add\n                    - find: \n                        path: .item > a \n                        do: \n                        - parse:\n                            attr: href\n                        - walk:\n                            to: value\n                            do:\n                            - object_new: product\n                            - find: \n                                in: doc\n                                path: head \n                                do: \n                                - eval:\n                                    routine: js\n                                    body: '(function (){var d = new Date(); return d.toISOString()})();'\n                                - object_field_set:\n                                    object: product\n                                    field: date\n                                - static_get: url\n                                - object_field_set:\n                                    object: product\n                                    field: url\n                            - find: \n                                path: .tabbed li:nth-of-type(1) \n                                do: \n                                - parse\n                                - space_dedupe\n                                - trim\n                                - if:\n                                    match: (\\S)\n                                    do:\n                                    - object_field_set:\n                                        object: product\n                                        field: description\n                            - find: \n                                path: body \n                                do: \n                                - register_set: Calypso St. Barth\n                                - object_field_set:\n                                    object: product\n                                    field: brand\n                                - variable_get: cat1\n                                - if:\n                                    match: (\\S)\n                                    do:\n                                    - object_field_set:\n                                        object: product\n                                        field: categories\n                                        joinby: "|"\n                                - variable_get: cat2\n                                - if:\n                                    match: (\\S)\n                                    do:\n                                    - object_field_set:\n                                        object: product\n                                        field: categories\n                                        joinby: "|"\n                                - variable_get: cat3\n                                - if:\n                                    match: (\\S)\n                                    do:\n                                    - object_field_set:\n                                        object: product\n                                        field: categories\n                                        joinby: "|"\n                            - find: \n                                path: .product-name > h1 \n                                do: \n                                - parse\n                                - space_dedupe\n                                - trim\n                                - object_field_set:\n                                    object: product\n                                    field: name\n                            - find: \n                                path: img.cloudzoom-gallery, .image > img\n                                do: \n                                - parse:\n                                    attr: data-cloudzoom\n                                - eval:\n                                    routine: js\n                                    body: (function () {var e = {}; return JSON.stringify(e);})();\n                                - normalize:\n                                    routine: json2xml\n                                - to_block\n                                - find: \n                                    path: zoomImage \n                                    do: \n                                    - parse\n                                    - normalize:\n                                        routine: url\n                                    - object_field_set:\n                                        object: product\n                                        field: images\n                                        joinby: "|"\n                            - find: \n                                path: script:matches(var\\s*spConfig) \n                                do: \n                                - parse:\n                                    filter:\n                                        - Product\\.Config\\((.+)\\);\n                                - normalize:\n                                    routine: json2xml\n                                - to_block\n                                - find: \n                                    path: 'attributes > *:hasChild(code:matches(color)) > options > label'\n                                    do: \n                                    - parse\n                                    - space_dedupe\n                                    - trim\n                                    - object_field_set:\n                                        object: product\n                                        field: variations\n                                        joinby: "|"\n                                - find: \n                                    path: body_safe > productid \n                                    do: \n                                    - parse\n                                    - space_dedupe\n                                    - trim\n                                    - object_field_set:\n                                        object: product\n                                        field: sku\n                                - find: \n                                    path: body_safe > baseprice\n                                    do: \n                                    - parse\n                                    - if:\n                                        match: (\\d)\n                                        do:\n                                        - object_field_set:\n                                            object: product\n                                            field: price\n                                            type: float\n                                        - register_set: USD\n                                        - object_field_set:\n                                            object: product\n                                            field: currency\n                            - object_save:\n                                name: product\n            - find: \n                path: ul > li > a \n                do: \n                - parse\n                - space_dedupe\n                - trim\n                - normalize:\n                    routine: lower\n                - variable_set: cat3\n                - parse:\n                    attr: href\n                - normalize:\n                    routine: url\n                - pool_clear: default\n                - parse:\n                    attr: href\n                - link_add\n                - walk:\n                    to: links\n                    do:\n                    - find: \n                        path: .paging  li > a:matches([Nn]ext) \n                        do: \n                        - parse:\n                            attr: href\n                        - link_add\n                    - find: \n                        path: .item > a \n                        do: \n                        - parse:\n                            attr: href\n                        - walk:\n                            to: value\n                            do:\n                            - object_new: product\n                            - find: \n                                in: doc\n                                path: head \n                                do: \n                                - eval:\n                                    routine: js\n                                    body: '(function (){var d = new Date(); return d.toISOString()})();'\n                                - object_field_set:\n                                    object: product\n                                    field: date\n                                - static_get: url\n                                - object_field_set:\n                                    object: product\n                                    field: url\n                            - find: \n                                path: .tabbed li:nth-of-type(1) \n                                do: \n                                - parse\n                                - space_dedupe\n                                - trim\n                                - if:\n                                    match: (\\S)\n                                    do:\n                                    - object_field_set:\n                                        object: product\n                                        field: description\n                            - find: \n                                path: body \n                                do: \n                                - register_set: Calypso St. Barth\n                                - object_field_set:\n                                    object: product\n                                    field: brand\n                                - variable_get: cat1\n                                - if:\n                                    match: (\\S)\n                                    do:\n                                    - object_field_set:\n                                        object: product\n                                        field: categories\n                                        joinby: "|"\n                                - variable_get: cat2\n                                - if:\n                                    match: (\\S)\n                                    do:\n                                    - object_field_set:\n                                        object: product\n                                        field: categories\n                                        joinby: "|"\n                                - variable_get: cat3\n                                - if:\n                                    match: (\\S)\n                                    do:\n                                    - object_field_set:\n                                        object: product\n                                        field: categories\n                                        joinby: "|"\n                            - find: \n                                path: .product-name > h1 \n                                do: \n                                - parse\n                                - space_dedupe\n                                - trim\n                                - object_field_set:\n                                    object: product\n                                    field: name\n\n                            - find: \n                                path: img.cloudzoom-gallery, .image > img\n                                do: \n                                - parse:\n                                    attr: data-cloudzoom\n                                - eval:\n                                    routine: js\n                                    body: (function () {var e = {}; return JSON.stringify(e);})();\n                                - normalize:\n                                    routine: json2xml\n                                - to_block\n                                - find: \n                                    path: zoomImage \n                                    do: \n                                    - parse\n                                    - normalize:\n                                        routine: url\n                                    - object_field_set:\n                                        object: product\n                                        field: images\n                                        joinby: "|"\n                            - find: \n                                path: script:matches(var\\s*spConfig) \n                                do: \n                                - parse:\n                                    filter:\n                                        - Product\\.Config\\((.+)\\);\n                                - normalize:\n                                    routine: json2xml\n                                - to_block\n                                - find: \n                                    path: 'attributes > *:hasChild(code:matches(color)) > options > label'\n                                    do: \n                                    - parse\n                                    - space_dedupe\n                                    - trim\n                                    - object_field_set:\n                                        object: product\n                                        field: variations\n                                        joinby: "|"\n                                - find: \n                                    path: body_safe > productid \n                                    do: \n                                    - parse\n                                    - space_dedupe\n                                    - trim\n                                    - object_field_set:\n                                        object: product\n                                        field: sku\n                                - find: \n                                    path: body_safe > baseprice \n                                    do: \n                                    - parse\n                                    - if:\n                                        match: (\\d)\n                                        do:\n                                        - object_field_set:\n                                            object: product\n                                            field: price\n                                            type: float\n                                        - register_set: USD\n                                        - object_field_set:\n                                            object: product\n                                            field: currency\n                            - object_save:\n                                name: product<\/code><\/pre>\n
      Sample of scraped data<\/h3>\n
      Below is a sample of a dataset with several products in JSON format (so you can easily review it and see data structure). The dataset can be downloaded as CSV, XLSX, XML, or any other text format using the templates.<\/p>\n
      [{\n    "product": {\n        "brand": "Calypso St. Barth",\n        "categories": "new arrivals|clothing",\n        "currency": "USD",\n        "date": "2017-12-27T09:48:47.804Z",\n        "description": "By CALYPSO St. Barth - Welcome cooler weather with this lavish suede jacket. Supple soft and chic, this jacket flatters with a feminine fit and lengthy silhouette. Accented with a ruffled trim, this autumn favorite features hook and eye closure and welt pockets. Wear with anything from casual basics to night out ensembles, a special wardrobe addition!",\n        "images": "http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/d\/i\/diosajacket4_1.1502486097.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/d\/i\/diosajacket3_1.1502486097.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/d\/i\/diosajacket2_1.1502486098.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/d\/i\/diosajacket5_1.1502486098.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/d\/i\/diosajacket_1.1502486099.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/d\/i\/diosajacket.1502486099.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/d\/i\/diosajacket4_1.1502486097.jpg",\n        "name": "Diosa Suede Ruffle Jacket",\n        "price": 1100,\n        "sku": "86060",\n        "url": "http:\/\/www.calypsostbarth.com\/new-arrivals\/clothing\/diosa-suede-ruffle-jacket",\n        "variations": "ATMN GLD"\n    }\n}\n,{\n    "product": {\n        "brand": "Calypso St. Barth",\n        "categories": "new arrivals|clothing",\n        "currency": "USD",\n        "date": "2017-12-27T09:48:48.837Z",\n        "description": "By CALYPSO St. Barth - Transcend into a new dimension of the open cardigan and welcome a versatile piece, essential to both dressy & casual outfitting. Composed of cozy alpaca wool, this sweater stands in for a lavish cold weather scarf, poncho or transitional jacket. Accented with ribbed detailing at the opening flaps, the cardigan\u0432\u0402\u2122s lengthy back center seam connects to inside out seam at waist for figure enhancing structure amidst an oversized silhouette. Wear it over dresses, pants or loungewear \u0432\u0402\u201c it\u0432\u0402\u2122s the perfect piece to carry you into the colder seasons.",\n        "images": "http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artatlaskosencocoon_20.1504818057.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artatlaskosencocoon_35.1504818057.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artatlaskosencocoon_12_ppn_.1504818058.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artatlaskosencocoon_28_ppn_.1504818059.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/s\/e\/serpinadress_11_1.1504818059.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artatlaskosencocoon_20.1504818057.jpg",\n        "name": "Kosen Wool Sweater Cocoon",\n        "price": 395,\n        "sku": "86562",\n        "url": "http:\/\/www.calypsostbarth.com\/new-arrivals\/clothing\/kosen-wool-sweater-cocoon",\n        "variations": "LT GREY"\n    }\n}\n,{\n    "product": {\n        "brand": "Calypso St. Barth",\n        "categories": "new arrivals|clothing",\n        "currency": "USD",\n        "date": "2017-12-27T09:48:49.120Z",\n        "description": "By CALYPSO St. Barth - Casual dressing made easy, this lightweight cotton dress is comfortable, casual and oh so chic. A relaxed silhouette, this effortless mini is graced with ruffled flounce sleeves. Back yoke and square neckline are accented with shirring details that flow into a loose body. Finished with side seam pockets, this dress will your new \u0432\u0402\u045ago-to\u0432\u0402\u045c favorite this season.",\n        "images": "http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artemiadress4.1503599057.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artemiadress2.1503599058.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artemiadress3.1503599058.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artemiadressmain.1503599059.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artemiadress.1503599060.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/e\/m\/emptyname_13.1503599060.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/a\/r\/artemiadress4.1503599057.jpg",\n        "name": "Artemia Flounce Sleeve Dress",\n        "price": 225,\n        "sku": "86442",\n        "url": "http:\/\/www.calypsostbarth.com\/new-arrivals\/clothing\/artemia-flounce-sleeve-dress",\n        "variations": "Black"\n    }\n}\n,{\n    "product": {\n        "brand": "Calypso St. Barth",\n        "categories": "new arrivals|clothing",\n        "currency": "USD",\n        "date": "2017-12-27T09:48:49.430Z",\n        "description": "By CALYPSO St. Barth - Boho chic meets outdoorsy elegance in this fleece poncho. Adorned with a tribal motif in earthy grey hues, it is finished with olive green crochet detailing along the lower trim, front zip and pockets as well as the shoulder sleeves. Gold zippers are enhanced with taupe leather trim for an added touch of bohemian charm. Luxuriously soft and cozy, this is the perfect outerwear piece to take you into the fall season",\n        "images": "http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/b\/a\/bahamutponcho_26.1504799605.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/b\/a\/bahamutponcho_46.1504799605.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/b\/a\/bahamutponcho_85.1504799606.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/b\/a\/bahamutponcho_79.1504799607.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/1\/1\/11_2_1640127.1504799608.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/w\/i\/winterlooks39684.1504799608.jpg|http:\/\/www.calypsostbarth.com\/media\/catalog\/product\/cache\/1\/image\/975x\/040ec09b1e35df139433887a97daa66f\/b\/a\/bahamutponcho_26.1504799605.jpg",\n        "name": "Bahamut Fleece Pullover Poncho",\n        "price": 395,\n        "sku": "86563",\n        "url": "http:\/\/www.calypsostbarth.com\/new-arrivals\/clothing\/bahamut-fleece-pullover-poncho-86639",\n        "variations": "Grey CC"\n    }\n}]\n<\/code><\/pre>","protected":false},"excerpt":{"rendered":"
      Since the launch of its first store in 1992, Calypso St. Barth grew out of a modest boutique of resort clothes to a luxurious, recognizable worldwide brand. Designers, inspired by the travels, culture and art of different countries, create amazing and feminine models that women of all ages like. If you like a mixture of […]<\/p>","protected":false},"author":4,"featured_media":395,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[31,30,2],"tags":[],"class_list":["post-393","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ecommerce-scraping","category-free-scrapers","category-web-scraping"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts\/393","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/comments?post=393"}],"version-history":[{"count":2,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts\/393\/revisions"}],"predecessor-version":[{"id":642,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts\/393\/revisions\/642"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/media\/395"}],"wp:attachment":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/media?parent=393"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/categories?post=393"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/tags?post=393"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}