⚙️Thought Machine⚙️

Search

SearchSearch

LLM

Nov 24, 20243 min read

After tinkering with Stable Diffusion, I found out that LLM (ChatGPT is the most popular) can do role playing. Now I can create pictures and chat with OC characters.

The LLM scene has been getting better every month, it’s crazy to realize GPT4 was only released last year 2023.

Setup

Roleplay

I’m using Claude 3 Opus and Sonnet for RP, their writings feel more natural than GPT4. Sonnet has some problems following prompts and its proses are a bit dry. Opus is the best LLM I have ever used, but it’s really expensive. I hope GPT5 will be better but cheaper than Opus.

I’m still learning to create character cards in SillyTavern . So far, my endeavor has created interesting results.

Claude breaking 4th wall

I’m going to try out Nand2Tetris with one of my OCs soon.

Context

As the number of tokens in a prompt increases, LLMs becomes slower, and lose memory of the earliest conversation. There are a few features I found on SillyTavern which somewhat mitigate this issue.

  • Vector Database (The link points to smart context, but vector db can be run locally)
    • A database which automatically creates chat messages, indexed by some keywords, from past prompts.
    • If the latest prompt has some matching keywords with the database, the relevant messages will be inserted as well.
    • In my testing, LLMs regularly repeat the old messages in a loop.
  • World Info
    • Users manually insert relevant data into a dictionary, choosing the associated keyword.
    • Info from the dictionary will be inserted when relevant keywords are triggered.
    • This takes time to set up, but yields the most benefits.
  • Author’s note
    • A section of texts which can be inserted into the prompt.
    • I use this before creating WI entries.
  • Summary (Same as Vector DB, can be run locally)
    • Users create a custom prompt to summarize the chat history, using an LLM.
    • This can be set to run on schedule.
    • My prompt regularly fails in silence, so the summary doesn’t get updated at scheduled intervals.

Current Workflow

I use STscript to generate a prompt asking for updated character profile or summary.

I was originally using a script to update WI and Author’s Note directly, but I want to compare returned responses with current saved texts before saving new version.

/input Update character |
/setinput Pause RP and give me updated character profile of {{pipe}}. This info is for you to resume RP later.

/setinput Pause RP and give me summary of the story. This info is for you to resume RP later.

Work

Phind Search is absolutely time-saver, their search results and summaries are more useful to me than Google. I have tried Phind Chat which is also a decent alternative to Chat GPT.

Study

I’m following this guide to learn about LLM. I have to start from the basics, working on linear algebra. I barely remember high-school math, even calculating sides in a triangle with sin(), cos() stump me. I do have a lot of holes in my knowledge, so it’s a good opportunity to improve.

Current progress:

Backlinks