Web Extract Data - nodebotrpa/editor GitHub Wiki
This tutorial introduces NodeBot RPA and creates a flow that demonstrates data extraction and writing to csv file.
How to extract data
1. Add an inject node
2. Add a Web New Session node
New Session node allows you to open a blank browser
3. Add a Web Navigate to node
Navigate to node allows you to navigate a url. Enter https://mufubu.com/collections/jewellery to URL field.
4. Add a Web Extract Data node
Extract Data node allows you to extract data. Enter {"row":"//body[1]/div[3]/div[1]/div[3]/div[3]/div[1]/div[4]/div","header":["Name","Present Price"],"column":["div[2]/a[1]/h3[1]","div[2]/div[1]/div[1]/span[1]"]} to Scrape Data field. You can use Web Extract Tool to get this information. Enter ProductList to result variable field.
5. Add a Write DS to Text File node
Write to text file node allows you to write dataset to text file. Enter filename location to filename field and enter ProductList to Dataset field.
6. Add Web Window Action node
Window Action node allows you to close browser.
7. Connect all nodes by dragging between nodes.
8. Click Deploy button.
9. Click inject node to run.
Import this flow
You can import this flow to NodeRed editor also.
- Copy below json data to clipboard
- Run NodeRed editor
- Click import menu
- Select Clipboard option and paste from clipboard.
- Click Import button
[{"id":"28b4398e.2eac76","type":"tab","label":"Tutorial - Web Scrap - 1","disabled":false,"info":"Extracts jewellery products from mufubu.com and writes to csv file"},{"id":"f4ebd7ad.c811c8","type":"inject","z":"28b4398e.2eac76","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":110,"y":60,"wires":["3634c5c.8f11f3a"](/nodebotrpa/editor/wiki/"3634c5c.8f11f3a")},{"id":"42285daa.64c064","type":"nbr-web-extract-data","z":"28b4398e.2eac76","name":"Extract jewellery product table","scrape":"{\"row\":\"//body[1]/div[3]/div[1]/div[3]/div[3]/div[1]/div[4]/div\",\"header\":[\"Name\",\"Present Price\"],\"column\":[\"div[2]/a[1]/h3[1]\",\"div[2]/div[1]/div[1]/span[1]\"]}","variable":"ProductList","waitbefore":"500","waitafter":"500","x":810,"y":60,"wires":["22ab1269.29187e"](/nodebotrpa/editor/wiki/"22ab1269.29187e")},{"id":"3634c5c.8f11f3a","type":"nbr-web-new-session","z":"28b4398e.2eac76","name":"Open browser","browser":"CHROME","implicit":"0","script":"0","pageLoad":"0","maximize":false,"variable":"mufubu","waitbefore":"500","waitafter":"500","x":300,"y":60,"wires":["582f24b5.18362c"](/nodebotrpa/editor/wiki/"582f24b5.18362c")},{"id":"582f24b5.18362c","type":"nbr-web-navigate-to","z":"28b4398e.2eac76","name":"Navigate to mufubu.com","url":"https://mufubu.com/collections/jewellery","pageLoad":false,"waitbefore":"500","waitafter":"500","x":530,"y":60,"wires":["42285daa.64c064"](/nodebotrpa/editor/wiki/"42285daa.64c064")},{"id":"22ab1269.29187e","type":"nbr-write-ds-text-file","z":"28b4398e.2eac76","name":"","filename":"C:\\Temp\\ProductList.csv","data":"ProductList","overwrite":true,"append":false,"header":true,"quote":false,"delimeter":";","waitbefore":"500","waitafter":"500","x":300,"y":140,"wires":["3a428240.6f608e"](/nodebotrpa/editor/wiki/"3a428240.6f608e")},{"id":"3a428240.6f608e","type":"nbr-web-window-action","z":"28b4398e.2eac76","name":"Close browser","action":7,"waitbefore":"500","waitafter":"500","x":500,"y":140,"wires":[](/nodebotrpa/editor/wiki/)}]