Tools - TravelingRobot/NAI_Community_Research GitHub Wiki

Tokenizer Tools

NovelAI's own tokenizer tool let's you see directly in the browser how things are tokenized. If you prefer, you could also use The Ermela tool or the OpenAI tokenizer tool.

(Tokens are how the AI processes the text. Read more about tokens on the UKB; thanks to ShinCore for bringing the OpenAI tool to my attention!)

Bran's aid-cross-compat-tool

allows converting of ai dungeon content to novelai .scenario and vice versa. also known as AIDCCT/AID-CCT/it's not AIDCAT please don't get them mixed up

Find it here

Unofficial NovelAI APK For Android

This is a simple webview app to make it easier to use Novel AI on a Android phone.
This should work for Android from version 4.4.4.

Link

NovelAI Research Tool - nrt

nrt was written by OccultSage and allows you to specify your testing parameters and then automatically generates a series of outputs for you. Strongly recommended for almost any type of sampling you want to do in NAI! Read more about the tool and download it here.

nrt JSON Builder & Launcher (outdated)

Currently outdated - should still work, but does not support newer nrt features

A companion tool to nrt. It features a GUI to more easily build your input JSONs and can take care of starting up nrt for you. More importantly, it enables you do use nrt for mass recursive testing. Read more about the tool and download it here.

Formatting module training data

These tools can be used to clean text data for your own modules. In general it is recommended to make a backup before applying any automatic cleaning tool. Occasionally they can lead to unintended changes to your files.

The UKB has an excellent list of these as part of an excellent guide for custom module training.

Datesetting Tool for ao3 Stories

Convert any ao3 story to a module with a single click of a button Find it here.

APIs

novelai-api

Python API for the NovelAI REST API

The module is intended to be used by developers as a help for using NovelAI's REST API.

Link

NovelAPI

An C# API for NovelAI. Based on work from wbrown's NovelAI Research Tool.

Built against .Net Framework 4.6.1 Requires Konscious's Argon2 (v1.2.0 known good) and RestSharp (v106.12.0.0 known good) available via NuGet.

Link.

Taleir regex helper

  • Taleir wrote a tool to help producing regex keys

Here's my initial contribution to the NovelAI community. This is an advanced lorebook generator script, run on your local computer with NodeJS (NovelAI's engine still does not support scripting), that allows complex keyword composition to make managing larger lorebooks easier and with more focused entries.

It spits out a .lorebook file that you can then upload into your story.

I unfortunately have not taken much time to explain how to use it and the example lorebook provided is pretty short (I need a break, but wanted to get this shared), so consider it a more "advanced" tool.

  • From what I can gather it lets you combine keys with different matching modes (can be set to match word exactly, as prefix, postfix, or anywhere in a word) with each other or your own regex expressions?

Check out the tool here

Rando's tool to extract entries from wikia

Here's another tool I just whipped up to help someone else extract data from Fallout wikia articles. I hope you can see how to make tools for your own extractions.

Find it here

NAI-Chatbot by rosemash

This is a web server written in Lua that serves a web chatbot interface for use with NovelAI. It lets you talk to multiple concurrent AI chat partners at once, saving persistent chat memory to disk so you can resume your conversations on any device on your network.

Find it here