Tools - TravelingRobot/NAI_Community_Research GitHub Wiki
Tokenizer Tools
NovelAI's own tokenizer tool let's you see directly in the browser how things are tokenized. If you prefer, you could also use The Ermela tool or the OpenAI tokenizer tool.
(Tokens are how the AI processes the text. Read more about tokens on the UKB; thanks to ShinCore for bringing the OpenAI tool to my attention!)
Bran's aid-cross-compat-tool
allows converting of ai dungeon content to novelai .scenario and vice versa. also known as AIDCCT/AID-CCT/it's not AIDCAT please don't get them mixed up
Find it here
Unofficial NovelAI APK For Android
This is a simple webview app to make it easier to use Novel AI on a Android phone.
This should work for Android from version 4.4.4.
nrt
NovelAI Research Tool - nrt
was written by OccultSage and allows you to specify your testing parameters and then automatically generates a series of outputs for you. Strongly recommended for almost any type of sampling you want to do in NAI! Read more about the tool and download it here.
nrt JSON Builder & Launcher (outdated)
Currently outdated - should still work, but does not support newer nrt
features
A companion tool to nrt
. It features a GUI to more easily build your input JSONs and can take care of starting up nrt
for you. More importantly, it enables you do use nrt
for mass recursive testing. Read more about the tool and download it here.
Formatting module training data
These tools can be used to clean text data for your own modules. In general it is recommended to make a backup before applying any automatic cleaning tool. Occasionally they can lead to unintended changes to your files.
The UKB has an excellent list of these as part of an excellent guide for custom module training.
Datesetting Tool for ao3 Stories
Convert any ao3 story to a module with a single click of a button Find it here.
APIs
novelai-api
Python API for the NovelAI REST API
The module is intended to be used by developers as a help for using NovelAI's REST API.
NovelAPI
An C# API for NovelAI. Based on work from wbrown's NovelAI Research Tool.
Built against .Net Framework 4.6.1 Requires Konscious's Argon2 (v1.2.0 known good) and RestSharp (v106.12.0.0 known good) available via NuGet.
Link.
Taleir regex helper
- Taleir wrote a tool to help producing regex keys
Here's my initial contribution to the NovelAI community. This is an advanced lorebook generator script, run on your local computer with NodeJS (NovelAI's engine still does not support scripting), that allows complex keyword composition to make managing larger lorebooks easier and with more focused entries.
It spits out a .lorebook file that you can then upload into your story.
I unfortunately have not taken much time to explain how to use it and the example lorebook provided is pretty short (I need a break, but wanted to get this shared), so consider it a more "advanced" tool.
- From what I can gather it lets you combine keys with different matching modes (can be set to match word exactly, as prefix, postfix, or anywhere in a word) with each other or your own regex expressions?
Check out the tool here
Rando's tool to extract entries from wikia
Here's another tool I just whipped up to help someone else extract data from Fallout wikia articles. I hope you can see how to make tools for your own extractions.
Find it here
NAI-Chatbot by rosemash
This is a web server written in Lua that serves a web chatbot interface for use with NovelAI. It lets you talk to multiple concurrent AI chat partners at once, saving persistent chat memory to disk so you can resume your conversations on any device on your network.
Find it here