Data import tasks - Waiviogit/waivio GitHub Wiki

Additional information

  • [import / products] Amazon can only have one product under one ASIN, datafiniti often has duplicates. When importing records with one ASIN, you must select the most recent record, ignore the rest. #4742
  • [import / product] #4744
    1. the names of departments are best left in the register as they are;
    2. description to collect and, if possible, run through chatGPT or equivalent;
    3. avatar - take from PrimaryImage (or 1st), bring to a square (can be a transparent background);
    4. upload other pictures to gallery
  • [import / books] #4749
    1. Brand is listed as a publisher; there are as many as 3(!) publishers in the Publisher field - apparently, take only the first one;
    2. Departments - sometimes written as Categories or taxonomy
  • [import / books] Datafiniti announced that the MostRecentPriceAmount field contains the actual price of the product, if it exists, then it makes sense to use it for our Price #4751
  • [import / books] weight must be imported #4750
  • [data import / price] apparently MostRecentPrice is not available in all records on Datadfiniti. It makes sense in its absence to take the price from the secondary fields #4775
  • [data import / group iD] looks like, we can use features / Parent ASIN as the Group ID. For this group of goods, he performs this role (B08RD2P6MJ) #4776
  • [data import / dialog] the user must consciously choose the object type selection field himself (no default value). #4799
  • [data import / book] quite often there is a situation when importing books - two options appear simultaneously: "paperback" and "paperbackpaperback". It would be nice to catch such an option on pre-processing and not add an erroneous option. #4886
  • [data import / RC] an import was initiated and after 1,500 updates the account ran out of RCs (cannot vote although there is still enough VP). We need to add import inhibition when RC goes below, for example, 10% (at this stage this parameter is just hardcoded). #4848
  • [data import / options] it turns out that sometimes options (color, size) are hidden in the prices section and, apparently, they should be taken into account too (possibly if the options are not found in the usual place) #4847
  • [data import / gallery] in many galleries there are empty pictures, also video icons are too small, it also makes sense to ignore them when importing #4841
  • [data import / descriptions] 1. in this case, the product description was under Product Features, points 2 and 3 are already described in other tasks #4780
  • Add Set option categories if objects with the same groupId have no options, when importing (rename to "version") #4910
  • [data import / default] By default, set the voice power limit for imports to 30%. In practice, 75% is reached very quickly. #4969
  • [data import / votes] user WAIV vote value: $0.0034 (according to my estimates, this should have been enough for 120 votes at $0.001), but he was denied import due to low voting power. Probably, we need to cut off only those who have WAIV vote less than $0.0011 (5 updates). #4957
  • [data import / options] when importing options, their names must be capitalized as a heading (capital letters on all words) #5031
  • Add ChatGPT to rewrite description in import #5119
  • Option will be pulled from amazona (and not from import if there is no flag).
  • Added mechanism to check images on duplicates
  • If there was no department from the main shop in the import, then one department is added from the main ones that matched with the keywords #5057
  • [data import / books] when creating pages of authors, the description for them is easy to get with chatGPT for the query "tell me more about [person name], the author of [book title]". The results are very accurate. #5156
  • [data import / brand] if the values in the brand / manufacturer / merchant fields are the same, then write update for only one field (in the specified sequence) #5207
  • [data import / book] for products, a rule is introduced that an option can have only one value. The same rule needs to be introduced for importing books. #5155
  • [author / chatGPT] there are times when chatGPT doesn't know the author. These cases should be rejected and the description field should be left empty. #5203
  • [data import / chatGPT] change "Crafted by ChatGPT" to "Description by ChatGPT." and put it as a continuation of the last paragraph, and not on a separate line (too striking, and "crafted" sometimes reads that this product was designed by ChatGPT.) #5218
  • fix to import book options to look for the shortest option #5239
  • It is necessary that regional asins be perceived as asins #5239
  • Add socket for Data import page #5113
  • [data import / large file] an error occurs when uploading a file (35Mb) #5228
  • [data import / book] there was an object book, it had asins. When trying to update a product using this ASIN, a new record was created instead of update and there is no asins in it, although it was in JSON. #5154
  • [data import / dialog] on large files there is a danger of pressing the Submit button again. It is proposed to add a loading indicator to it (similar to ASIN scanner), as well as change the name from Confirm to Submit. #5227
  • [data import / upgrade] there was an old record with an asin number, importing a record with the same asin led to the creation of a new record, instead of upgrading an existing record #5170
  • [data import / type] user mistakenly imported books as products. It would be good to check to avoid such cases. #5299
  • [data import / descriptions] if chatGPT gives an error (As an AI language model...), then there is no need to save such a description, it is better to fill the field with data without specification via chatGPT or, if there is no data, leave the field empty #5369
  • [data import / options] it is desirable to have one value for one kind of options. There is no task to pull out all possible options, you need to understand which value is the most probable. #5043
  • Add rating when importing a file (type product - add to Value category, type book - Rating) from Features field "Overall Rating" #5280
  • [data import / ratings] in this case, there is a rating (recorded in Features), but there is no vote on the rating. From feature "Average Overall Rating"#5396
  • Import: If Chat GPT does not respond, pull description from file #5400
  • Import: If the hive engine is not working, that is, the import is not possible, then you can show the appropriate notification when submitting the import #5403
  • Add additional modal window for import, when user are trying to import books with product type #5348
  • [data import / duplicates] when you re-upload the same json, 4 updates are added. Two updates visible (avatar, description) and two hidden. First, you need to avoid duplicate avatars and descriptions (for example, for chatGPT, do not create a new description if it already exists). Secondly, we need to think about the other two entries. The fact is that in the future we may have automatic gateways and it is impossible to produce records on each check. #5412
  • [data import / errors] Sometimes generating descriptions via ChatGPT causes errors. These errors must be recognized and then: 1. try a different query formulation, 2. use the description from the imported file, 3. leave the field empty #5306
  • Import: The product name is constantly repeated in the text if the description is via ChatGPT #5499
  • Add additional status for import - pending #5482
  • [data import / ratings] ratings need to be imported only for the voice (Value), there is no need to save the value in Features #5384
  • [extension / gallery] many key fields were not included in Json: gallery items, description, weight, dimensions, manufacturer. Some values, such as "Item model number", can also be saved as one of the Product IDs (perhaps this number can be used to find the product in other stores, for example, Wallmart, Target, etc) #5384
  • [data import / failed] the user's HP dropped below 10% and the import stopped. Part of the imports for which there was not a single update, failed instead of slowing down the import. #5699
  • [extension / books] When importing books, not only the selected option is transferred, but also the names of all others #6174