🤖 LLM

After tinkering with Stable Diffusion, I found out that LLM can do role playing. Now I can create pictures and chat with OC characters.

The LLM scene has been getting better every month, it's crazy to realize GPT4 was only released last year 2023.

Setup

For roleplay, I use SillyTavern and AWS Claude.

As for work, I use Google AI Studio because it's free.

Roleplay

I'm using Claude 3 Opus and Sonnet 3.6 for RP, their writings feel more natural than GPT4. Sonnet has some problems following prompts and its proses are a bit dry. Opus is the best LLM I have ever used, but it's really expensive.

I'm still learning to create character cards in SillyTavern. So far, my endeavor has created interesting results.

Hallucination

As the number of tokens in a prompt increases, LLMs becomes slower and lose track of the earliest conversation. There are a few features I found on SillyTavern which can mitigate this issue.

Vector Database

The link points to smart context, but vector db can be run locally. It automatically stores chat messages, indexed by some keywords, from past prompts. If the recent prompts have some matching keywords in the db, the relevant messages will be inserted into the chain of conversation.

In my testing, Claude 2 repeated the old messages in a loop.

World Info

Users manually insert relevant data into a dictionary and choose the associated keyword. Info from the dictionary will be inserted when relevant keywords are triggered. This takes time to set up, but yields the most benefits.

Author's note

Users can insert texts into the prompt at selected position. I used this before settling with World Info entries.

Summary

Same as Vector DB, it can be run locally.

Users create a custom prompt to summarize the chat history, using a LLM. This can be set to run on schedule. My prompt regularly fails in silence, so the summary doesn't get updated at scheduled intervals.

Current Workflow

I use STscript to generate a prompt asking for updated character profile or summary.

I was originally using a script to update WI and Author's Note directly, but I want to compare returned responses with current saved texts before saving new version.

/input Update character |
/setinput Pause RP and give me updated character profile of {{pipe}}. This info is for you to resume RP later.

/setinput Pause RP and give me summary of the story. This info is for you to resume RP later.

Work

Google AI Studio with Gemini 1206 is free so it's hard to complain. It does decently well with my technical Q&A sessions.

Study

I'm following this guide to learn about LLM. I have to start from the basics, working on linear algebra. I barely remember high-school math, even calculating sides in a triangle with sin(), cos() stump me.