You\u2019ve probably seen galleries with user-generated content in various online stores that sell clothing, shoes, home products, etc. They are very helpful in selling a product because they allow a potential buyer to see how a particular product sits on a real person rather than on a model and allows the buyer to make a more conscious decision. You probably would like to extract user-generated content but don\u2019t know how to do it with a limited budget.<\/p>\n
Technical implementation of such a mechanism is following: the service aggregator collects user-generated images on the Internet, for example, in Instagram, determines the brand and model of the item or items shown in the photo, and delivers it in a particular feed. It may be costly to connect to such service for a small venue, so mainly large mono and multi-brand online stores can afford it.<\/p>\n
The second option is to create such an aggregation service yourself, but this is a very time-consuming, long-term and expensive process, much more expensive than connecting to a similar service-aggregator for a single online store.<\/p>\n
However, there is a budget option. Many brands and well-known online stores are already customers of such aggregators and have their feeds with user-generated photos and information about corresponding products. Therefore, if you sell products of similar brands, you can get information from these feeds, process the received data and use them in your online store to sell products of this brand.<\/p>\n
You can say that coding scrapers for every site and brand if there are hundreds of them, is quite tedious and takes much time. However, you do not need to scrape the websites. You only need a feed with user content. Moreover, such feeds are provided by a limited set of aggregators. Therefore technically, you need to have only one scraper, with standard logic and use different URLs or parameters to pick up feeds for different stores and brands.<\/p>\n
One such service is Like2Buy<\/strong>, a service provided by Curalate company. They serve more than 6000 online stores and brands. All feeds can be easily googled by typing \u201clike2buy.curalate.com\u201d in the search box and clicking on the link \u201cshow all results.\u201d Also, just for your reference, we\u2019ll list below a few stores and their IDs for use with our free web scraper, which we\u2019ll share in this article.<\/p>\n This data can be useful not only for online stores but also for companies conducting research for brands, as well as companies working in the machine learning area.<\/p>\n So you need a free account with our Diggernaut service. You can follow this comprehensive guide:<\/p>\n You can also set up a schedule for running your scraper and collect data regularly.<\/p>\n The scraper configuration is shown below. You can copy it to any of your diggers, put the ID from the store table (or a few at a time) and start your digger.<\/p>\n As a result, you get a dataset with the following structure:<\/p>\n As you can see, our basic scraper extracts only the URL to the image, the names, and URLs of the products. By changing the scraper logic, you can extract other data available in the feed, as well as perform any manipulations with the extracted data, forming your dataset precisely as you need it. Below is the structure of one source feed object, so you can better navigate to compose CSS selectors to containers with data:<\/p>\n Below, we list the stores and their IDs that use Like2Buy to deliver user-generated content. This list is incomplete, if you did not find the brand or store you are interested in, try to google, or ask us, we are always happy to help :)<\/p>\n You\u2019ve probably seen galleries with user-generated content in various online stores that sell clothing, shoes, home products, etc. They are very helpful in selling a product because they allow a potential buyer to see how a particular product sits on a real person rather than on a model and allows the buyer to make a […]<\/p>","protected":false},"author":4,"featured_media":294,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[31,30,3,2],"tags":[],"class_list":["post-292","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ecommerce-scraping","category-free-scrapers","category-machine-learning","category-web-scraping"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts\/292","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/comments?post=292"}],"version-history":[{"count":4,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts\/292\/revisions"}],"predecessor-version":[{"id":661,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/posts\/292\/revisions\/661"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/media\/294"}],"wp:attachment":[{"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/media?parent=292"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/categories?post=292"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.diggernaut.com\/blog\/wp-json\/wp\/v2\/tags?post=292"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
---\nconfig:\n debug: 2\n agent: Firefox\niterator:\n type: csv\n name: shop\n value: # Set here single store ID or few store IDs separated by comma\ndo:\n- walk:\n to: https:\/\/like2buy.curalate.com\/\/\n do:\n - pool_clear: sub\n - find:\n path: html\n do:\n - eval:\n routine: js\n body: '(function() {return "xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx".replace(\/[xy]\/g, function(e) {var t = 16 * Math.random() | 0, r = "x" === e ? t : 3 & t | 8; return r.toString(16)})})();'\n - variable_set: rid\n - register_set: http:\/\/api.curalate.com\/v1\/like2buy\/\/products.json?rid=\n - link_add:\n pool: sub\n - walk:\n to: links\n pool: sub\n do:\n - find:\n path: qbookmark\n do:\n - parse\n - register_set: http:\/\/api.curalate.com\/v1\/like2buy\/\/products.json?qBookmark=&rid=\n - link_add:\n pool: sub\n - find: \n path: items \n do: \n - object_new: item\n - argument_get: shop\n - object_field_set:\n object: item\n field: shop\n - find:\n path: largephotourl\n slice: 0\n do:\n - parse\n - normalize:\n routine: url\n - object_field_set:\n object: item\n field: image\n - find: \n path: products\n do: \n - parse\n - object_new: product\n - find: \n path: destinationurl\n do:\n - parse\n - object_field_set:\n object: product\n field: url\n - find: \n path: name\n do:\n - parse\n - space_dedupe\n - trim\n - object_field_set:\n object: product\n field: name\n - object_save:\n name: product\n to: item\n - object_save:\n name: item<\/code><\/pre>\n[{\n "item": {\n "image": "https:\/\/d28m5bx785ox17.cloudfront.net\/v1\/img\/PPYWso07RgBC_UHzxcrgAO_Wk0twhD3XHvviHlJ7-ZY=\/d\/l",\n "product": [\n {\n "name": "Marco Faux-Leather Moto Jacket",\n "url": "https:\/\/shop.guess.com\/en\/catalog\/view\/women\/jackets-and-outerwear\/view-all\/marco-faux-leather-moto-jacket\/w74l10r72y1?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&crl8_id=670ce9b5-3465-4372-b0fe-df6a0c71ed4b"\n },\n {\n "name": "CAN: Marco Faux-Leather Moto Jacket",\n "url": "https:\/\/www.guess.ca\/en\/catalog\/view\/women\/jackets-and-outerwear\/view-all\/marco-faux-leather-moto-jacket\/w74l10r72y1?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&utm_content=w74l10r72y1&crl8_id=670ce9b5-3465-4372-b0fe-df6a0c71ed4b"\n }\n ],\n "shop": "guess"\n }\n}\n,{\n "item": {\n "image": "https:\/\/d28m5bx785ox17.cloudfront.net\/v1\/img\/Wn0kXxTmnzmAy6hTP3_bynEdtv9Ph7Y0M9FOVyLen00=\/d\/l",\n "product": [\n {\n "name": "US: Silver-Tone Charm Bracelet Box Set",\n "url": "https:\/\/shop.guess.com\/en\/catalog\/view\/434044G21?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&utm_content=434044G21&crl8_id=a0dd62bd-9024-4224-bcdf-323d6e6e601d"\n },\n {\n "name": "US: Boxed Rose Gold-Tone Charm Bracelet",\n "url": "https:\/\/shop.guess.com\/en\/catalog\/view\/434042G21?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&utm_content=434042G21&crl8_id=a0dd62bd-9024-4224-bcdf-323d6e6e601d"\n },\n {\n "name": "US: GUESS 1981 Eau De Toilette, 3.4 oz.",\n "url": "https:\/\/shop.guess.com\/en\/catalog\/view\/accessories\/women\/fragrance\/guess-1981-eau-de-toilette-3-4-oz\/32667861000?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&utm_content=32667861000&crl8_id=a0dd62bd-9024-4224-bcdf-323d6e6e601d"\n },\n {\n "name": "US: Metallic Mini Backpack Keychain",\n "url": "https:\/\/shop.guess.com\/en\/catalog\/view\/17GUP248?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&utm_content=17GUP248&crl8_id=a0dd62bd-9024-4224-bcdf-323d6e6e601d"\n },\n {\n "name": "CAN: Boxed Gold-Tone Stud Earring Set",\n "url": "https:\/\/guess.ca\/en\/Catalog\/View\/434046GC21\/?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&utm_content=434046GC21&crl8_id=a0dd62bd-9024-4224-bcdf-323d6e6e601d#434046GC21"\n },\n {\n "name": "CAN: GUESS 1981 Eau De Toilette, 3.4 oz.",\n "url": "https:\/\/www.guess.ca\/en\/catalog\/view\/accessories\/women\/fragrance\/guess-1981-eau-de-toilette-3-4-oz\/32667861000?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&utm_content=32667861000&crl8_id=a0dd62bd-9024-4224-bcdf-323d6e6e601d"\n },\n {\n "name": "CAN: Metallic Mini Backpack Keychain",\n "url": "https:\/\/www.guess.ca\/en\/Catalog\/View\/17GUP248\/?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&utm_content=17GUP248&crl8_id=a0dd62bd-9024-4224-bcdf-323d6e6e601d#17GUP248"\n },\n {\n "name": "EU: Holiday Delivery",\n "url": "https:\/\/www.guess.eu\/en\/CustomerCare\/guaranteed-delivery\/?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&crl8_id=a0dd62bd-9024-4224-bcdf-323d6e6e601d"\n }\n ],\n "shop": "guess"\n }\n}\n,{\n "item": {\n "image": "https:\/\/d28m5bx785ox17.cloudfront.net\/v1\/img\/oCSER6z1bD-KgCCgMcbH9Xk9OifDOvwuXgXNwAQmIeI=\/d\/l",\n "product": [\n {\n "name": "CAN: Lily Faux-Fur Coat",\n "url": "https:\/\/www.guess.ca\/en\/catalog\/view\/women\/jackets-and-outerwear\/faux-fur\/lily-faux-fur-coat\/w74l14w9t70?utm_source=instagram&utm_medium=social&utm_campaign=like2buy&utm_content=w74l14w9t70&crl8_id=9a9df613-d531-4252-a63e-2566d16dedd2"\n },\n {\n "name": "EU: Floral Faux-Fur Coat",\n "url": "https:\/\/www.guess.eu\/en\/catalog\/view\/women\/apparel\/coats-and-jackets\/floral-faux-fur-coat\/w74l14w9t70?color=dpid%3FCMP%3DSMC-INSTAGRAM-LIKETOBUY&crl8_id=9a9df613-d531-4252-a63e-2566d16dedd2"\n }\n ],\n "shop": "guess"\n }\n}]\n<\/code><\/pre>\n\n\n
\n \nStore or brand<\/th>\n ID<\/th>\n Store or brand<\/th>\n ID<\/th>\n<\/tr>\n<\/thead>\n \n Aldo<\/td>\n aldo_shoes<\/td>\n Ann Taylor<\/td>\n anntaylor<\/td>\n<\/tr>\n \n Anthropologie<\/td>\n anthropologie<\/td>\n Bed, Bath and Beyond<\/td>\n bedbathandbeyond<\/td>\n<\/tr>\n \n Brilliant Earth<\/td>\n brilliantearth<\/td>\n Cartier<\/td>\n cartier<\/td>\n<\/tr>\n \n CB2<\/td>\n cb2<\/td>\n Champion<\/td>\n champion<\/td>\n<\/tr>\n \n Chobani<\/td>\n chobani<\/td>\n Chumbak<\/td>\n chumbak<\/td>\n<\/tr>\n \n Crate and Barrel<\/td>\n crateandbarrel<\/td>\n Creative Recreation<\/td>\n creativerecreation<\/td>\n<\/tr>\n \n Covergirl<\/td>\n covergirl<\/td>\n David\u2019s Bridal<\/td>\n davidsbridal<\/td>\n<\/tr>\n \n Disney<\/td>\n disney<\/td>\n Dune London<\/td>\n dune_london<\/td>\n<\/tr>\n \n Farfetch<\/td>\n farfetch<\/td>\n Fawn Shoppe<\/td>\n fawn_shoppe<\/td>\n<\/tr>\n \n Forever21<\/td>\n forever21,forever21men<\/td>\n Fossil<\/td>\n fossil<\/td>\n<\/tr>\n \n Free People<\/td>\n freepeople<\/td>\n Gap<\/td>\n gap<\/td>\n<\/tr>\n \n Garage Clothing<\/td>\n garageclothing<\/td>\n Guess<\/td>\n guess<\/td>\n<\/tr>\n \n HauteLook<\/td>\n hautelook<\/td>\n Herbal Essenses<\/td>\n herbalessences<\/td>\n<\/tr>\n \n Hot Topic<\/td>\n hottopic<\/td>\n House of Lashes<\/td>\n houseoflashes<\/td>\n<\/tr>\n \n J. Crew<\/td>\n jcrew<\/td>\n Karl Lagerfeld<\/td>\n karllagerfeld<\/td>\n<\/tr>\n \n Kohl\u2019s<\/td>\n kohls<\/td>\n Laura Mercier<\/td>\n lauramercier<\/td>\n<\/tr>\n \n Lilly Pulitzer<\/td>\n lillypulitzer<\/td>\n Louis Vuitton<\/td>\n louisvuitton<\/td>\n<\/tr>\n \n lululemon<\/td>\n lululemon<\/td>\n Lulus<\/td>\n lulus<\/td>\n<\/tr>\n \n Macy\u2019s<\/td>\n macys<\/td>\n Misspap<\/td>\n misspap<\/td>\n<\/tr>\n \n Neiman Marcus<\/td>\n neimanmarcus<\/td>\n Next Com AU<\/td>\n nextofficial_au<\/td>\n<\/tr>\n \n Nordstrom<\/td>\n nordstrom<\/td>\n Paint Nite<\/td>\n paintnite<\/td>\n<\/tr>\n \n PB Teen<\/td>\n pbteen<\/td>\n Pendleton<\/td>\n pendletonwm<\/td>\n<\/tr>\n \n Pier 1<\/td>\n pier1<\/td>\n Pottery Barn<\/td>\n potterybarn<\/td>\n<\/tr>\n \n Raymour & Flanigan<\/td>\n raymourflanigan<\/td>\n Schoolhouse Electric & Supply Co<\/td>\n schoolhouse<\/td>\n<\/tr>\n \n Schutz<\/td>\n schutzshoes<\/td>\n Sephora<\/td>\n sephora<\/td>\n<\/tr>\n \n Sperry<\/td>\n sperry<\/td>\n Target<\/td>\n target<\/td>\n<\/tr>\n \n The Bump<\/td>\n thebump<\/td>\n The Company Store<\/td>\n thecompanystore<\/td>\n<\/tr>\n \n Topman<\/td>\n topman<\/td>\n TopShop<\/td>\n topshop<\/td>\n<\/tr>\n \n Victoria\u2019s Secret<\/td>\n victoriassecret<\/td>\n Vineyard Vines<\/td>\n vineyardvines<\/td>\n<\/tr>\n \n West Elm<\/td>\n westelm<\/td>\n Williams Sonoma<\/td>\n williamssonoma<\/td>\n<\/tr>\n \n Windsor<\/td>\n windsorstore<\/td>\n Z Gallerie<\/td>\n zgallerie<\/td>\n<\/tr>\n \n Zumiez<\/td>\n zumiez<\/td>\n <\/td>\n <\/td>\n<\/tr>\n<\/tbody>\n<\/table>","protected":false},"excerpt":{"rendered":"