5 ms·
I did a fine tuning and embedding on a large LLM that is based on 50 years of daily journal entries, extensive daily notebooks usually measuring in the hundreds
by justinlloyd 3y ago
I did a fine tuning and embedding on a large LLM that is based on 50 years of daily journal entries, extensive daily notebooks usually measuring in the hundreds to thousands of words per day, and personal writings across a half-dozen different blogs and websites and various social media feeds. Social media posts and comments (including this one) are also put into my notebooks with a snippet of context about why I posted that.
In my journal and daily notebooks I write out some pretty deep and dark thoughts too. It's about as close to a snapshot of my inner thoughts as you can imagine.
And the LLM digested all of that. It took quite a while.
It has provided some interesting insight into who I am as a person, and also how I see myself.
- Exuma 3y agoYou can’t leave us hanging like that … so what happened. What did you learn? Was it weird reading it? How do you see yourself?
- justinlloyd 3y agoI discovered that I can be a bit of a tease.
- romwell 3y agoGoddamit.
- ImHereToVote 3y ago+
- jakkos 3y agooutstanding
- cptcobalt 3y agowell done.
- legendofbrando 3y agoHaving done some of this myself, I’m curious your results on fine tuning vs embeddings. I’ve found the latter much more performant, but perhaps I’m thinking about fine tuning wrong.
- justinlloyd 3y agoI used fine tuning to approximate my style. Which is especially important around my logging style as I tend to break it down in to sections, and it is stream of thought writing, example of what I mean is here: https://www.github.com/justinlloyd/retro-chores https://www.github.com/justinlloyd/retro-chores. In my logging and journals I'll crank out anywhere between a couple of hundred words and a few thousand words per day. I used embedding for adding new knowledge. I also did a little work in letting it scan through my notebooks (they are OneNote and you can access and search via a Python API) via keyword search because it can point directly to something I've written in the past, and not just rely on model weights.