Web Extract Data - nodebotrpa/editor GitHub Wiki

This tutorial introduces NodeBot RPA and creates a flow that demonstrates data extraction and writing to csv file.

How to extract data

1. Add an inject node

2. Add a Web New Session node

New Session node allows you to open a blank browser

3. Add a Web Navigate to node

Navigate to node allows you to navigate a url. Enter https://mufubu.com/collections/jewellery to URL field.

4. Add a Web Extract Data node

Extract Data node allows you to extract data. Enter {"row":"//body[1]/div[3]/div[1]/div[3]/div[3]/div[1]/div[4]/div","header":["Name","Present Price"],"column":["div[2]/a[1]/h3[1]","div[2]/div[1]/div[1]/span[1]"]} to Scrape Data field. You can use Web Extract Tool to get this information. Enter ProductList to result variable field.

5. Add a Write DS to Text File node

Write to text file node allows you to write dataset to text file. Enter filename location to filename field and enter ProductList to Dataset field.

6. Add Web Window Action node

Window Action node allows you to close browser.

7. Connect all nodes by dragging between nodes.

8. Click Deploy button.

9. Click inject node to run.

Import this flow

You can import this flow to NodeRed editor also.

  • Copy below json data to clipboard
  • Run NodeRed editor
  • Click import menu
  • Select Clipboard option and paste from clipboard.
  • Click Import button
[{"id":"28b4398e.2eac76","type":"tab","label":"Tutorial - Web Scrap - 1","disabled":false,"info":"Extracts jewellery products from mufubu.com and writes to csv file"},{"id":"f4ebd7ad.c811c8","type":"inject","z":"28b4398e.2eac76","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":110,"y":60,"wires":["3634c5c.8f11f3a"](/nodebotrpa/editor/wiki/"3634c5c.8f11f3a")},{"id":"42285daa.64c064","type":"nbr-web-extract-data","z":"28b4398e.2eac76","name":"Extract jewellery product table","scrape":"{\"row\":\"//body[1]/div[3]/div[1]/div[3]/div[3]/div[1]/div[4]/div\",\"header\":[\"Name\",\"Present Price\"],\"column\":[\"div[2]/a[1]/h3[1]\",\"div[2]/div[1]/div[1]/span[1]\"]}","variable":"ProductList","waitbefore":"500","waitafter":"500","x":810,"y":60,"wires":["22ab1269.29187e"](/nodebotrpa/editor/wiki/"22ab1269.29187e")},{"id":"3634c5c.8f11f3a","type":"nbr-web-new-session","z":"28b4398e.2eac76","name":"Open browser","browser":"CHROME","implicit":"0","script":"0","pageLoad":"0","maximize":false,"variable":"mufubu","waitbefore":"500","waitafter":"500","x":300,"y":60,"wires":["582f24b5.18362c"](/nodebotrpa/editor/wiki/"582f24b5.18362c")},{"id":"582f24b5.18362c","type":"nbr-web-navigate-to","z":"28b4398e.2eac76","name":"Navigate to mufubu.com","url":"https://mufubu.com/collections/jewellery","pageLoad":false,"waitbefore":"500","waitafter":"500","x":530,"y":60,"wires":["42285daa.64c064"](/nodebotrpa/editor/wiki/"42285daa.64c064")},{"id":"22ab1269.29187e","type":"nbr-write-ds-text-file","z":"28b4398e.2eac76","name":"","filename":"C:\\Temp\\ProductList.csv","data":"ProductList","overwrite":true,"append":false,"header":true,"quote":false,"delimeter":";","waitbefore":"500","waitafter":"500","x":300,"y":140,"wires":["3a428240.6f608e"](/nodebotrpa/editor/wiki/"3a428240.6f608e")},{"id":"3a428240.6f608e","type":"nbr-web-window-action","z":"28b4398e.2eac76","name":"Close browser","action":7,"waitbefore":"500","waitafter":"500","x":500,"y":140,"wires":[](/nodebotrpa/editor/wiki/)}]