3 ms·
Is the difference between this and a frontier model that the scripture is guaranteed to be real? I'm on a team that develops a Bible study app, and we're all r
by ComputerPerson 2mo ago
Is the difference between this and a frontier model that the scripture is guaranteed to be real?
I'm on a team that develops a Bible study app, and we're all relatively content with how the basic models converse regarding scripture. Even as far back as GPT-4 was excellent. They occasionally have minor hallucinations (a dealbreaker for a production app), but they do an excellent job with theology and Bible scholarship, given reasonable guardrails.
I'll admit I'm coming from the perspective of "should we be implementing this?" It seems, on the surface, that a strong embedding-based verse retrieval covers the bases at a microfraction of the cost.
If you're interested, check out the development server where we're working on this. You navigate to the search (magnifying glass) and then hit "Meaning". Sorry for the confusing route; we're still deciding on back-end details and haven't focused on the front yet.
[1] https://ai.stepbible.org https://ai.stepbible.org
- vmt-man 2mo ago> Is the difference between this and a frontier model that the scripture is guaranteed to be real? flash is suitable only for a toy apps, not for production environments :)
- willchis 2mo agoI hope the irony of being concerned about hallucinations in a bible study app is not lost on you or others reading this!
- swingboy 2mo agoSir, you dropped your fedora.
- kfse 2mo agoAll models hallucinate. Everything in production with a model will too. It is not a "deal breaker"
- jmathai 2mo agoPhilosophically, I don’t believe we should outsource the accuracy of scripture to any single entity (let alone a for profit secular one). So it’s less about model choice and more about governance of scripture. I will check out the link you sent for sure!
- ComputerPerson 2mo agoI was under the impression you were getting BOOK.CHAPTER.VERSE references from the model and then sourcing them from a ground truth db. Anyways, impressive app! We haven't tackled such an ambitious project just for it being daunting.
- irthomasthomas 2mo agoIt can still generate a wrong reference and select the wrong verse.
- jmathai 2mo agoThat’s okay, IMO. Ultimately it’s the readers responsibility to apply their own intelligence. But the app should not misquote scripture itself.
- jmathai 2mo agoTook a look at StepBible. Looks really cool. My email is in my profile if you wanted to connect.