3 ms·

It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing. It looks like the model takes as input a state (structured

It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing. It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence). Edit: On the AI primer page, it looks like they do the RLCD on a pre-trained base model? [0]:https://docs.typesafe.ai/concepts/system-one https://docs.typesafe.ai/concepts/system-one

CompleteSkeptic 20d ago
CEO here - that is right! I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable). But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)
- ianbutler 20d ago
  I see this super interestingly as the "subconscious" to the llms "conscious" for lack of better terms. I'm super interested in this for broad and rapid decision making in the context of consumer agents so will be signing up for sure.
  - CompleteSkeptic 20d ago
    1. I am extremely on the same page 2. I do think that subconscious is not only much smarter than we give it credit for, but also much more robust than the "jagged frontier" of current LLMs (shilling my blog post on that jaggedness: https://www.completeskeptic.com/p/lies-damned-lies-and-benchmarks https://www.completeskeptic.com/p/lies-damned-lies-and-bench...)
  - cheesecakegood 19d ago
    I wonder if a good analogy would be, they are trying to tune the “gut feeling” of a model more directly
- deleted 20d ago
  [deleted]
- Flere-Imsaho 20d ago
  Hi - first congratulations, System One looks really promising. The Doom demo really help me, at least, to understand how System One differs from LLMs. However the first demo (Side-by-side demonstration) - I'm struggling to understand what is going on here!
  - potatoman22 20d ago
    I think that's to demonstrate its speed
  - davideg 20d ago
    I was confused at first too, but it makes more sense when you read about their primitives. E.g. https://docs.typesafe.ai/primitives/noul https://docs.typesafe.ai/primitives/noul The demo is showing System One producing its output in parallel very quickly and for little cost compared to an LLM generating its answers token-by-token. The "noul" type is used to evaluate a yes/no question and return the probability that the answer is yes. So this demo is showing System One offering much more nuanced responses and specific probabilities compared to an LLM's more crude responses (e.g. LLM shows "true" or "false" compared to "0.9" or "0.07" probabilities that the answer to some question is true).
- mckngbrd 20d ago
  here is how I attempted to explain it to my company's AI group chat, is this roughly accurate? "instead of autoregressive string output it instead outputs structured type-safe 'decisions' with probabilities/confidence scores, each generated in parallel so sort of more like a Large Classification Model than a Large Language Model? or, maybe better to think of it as a sort of "shift left" in the LLM's transformer architecture, allowing you to replace the predefined token vocabulary of an LLM with a prescribed set of 'decisions' that need to be made based off the input context; and exposing those probabilities directly so they can be integrated into the system logic, instead of just sampling from top-K. all of this while still being instruction-tuned (!!!)" It's always been possible to build classification pipelines using LLM embeddings as the input. seems like this is a much more sophisticated / useful application of that concept
  - CompleteSkeptic 20d ago
    very accurate! the one nuance I'd get into is I'd call it "zero-shot" over "instruction-tuned" (the latter often implies a particular distribution), but very safe for sharing
- ttul 20d ago
  For many day-to-day computing use cases, Jev seems far better suited than an autoregressive language model, if for no other reason than it is not wasting compute thinking about anything other than how to spit out a decision. Do you have an architectural explainer yet for Jev or are you holding that close to your chest and letting the magic rip for now?
  - CompleteSkeptic 20d ago
    architecture is close to the chest for now, but we have talked about writing a paper I don't want to shill my blog too much, but I will say data is probably far most interesting than architecture: https://www.completeskeptic.com/p/the-bitterest-lesson https://www.completeskeptic.com/p/the-bitterest-lesson
    - ttul 20d ago
      Well, I'm stoked to try it out. We have about a billion reasons a day to call a model like this to rid the world of spam and phishing.
- animan 20d ago
  It's unclear if the context extends as the conversation grows?
- mijoharas 20d ago
  Small request, can we get an explanation of the naming of "noul" in the docs[0]. I tried googling, and searching the docs and didn't understand why it was called that. (I'd also argue something like p_yes or just probability might be a simpler name, but I'm sure there's a better reason behind Bernoulli maps). [0] https://docs.typesafe.ai/primitives/noul#noul https://docs.typesafe.ai/primitives/noul#noul
  - safeandsound 20d ago
    Im going to guess bernoulli
    - mijoharas 20d ago
      It is Bernoulli (mentioned in the comment I replied to). I just failed to guess that myself :)
- sedev 16d ago
  You really, really, really, really, REALLY REALLY REALLY should explain neologisms in your documentation! Really really! Having "here's a weird unexplained difficult-to-guess-impossible-to-Google-for term" in your documentation is aggressively user-hostile. Please don't. The actual term is fine. Lack of an inline explanation is a blow-it move. Don't do that.
zenlikethat 20d ago
> the model takes as input a state (structured text? not sure if multi-modal) Input, and criteria/instructions can both be defined as structured input (JSON). This ends up being pretty powerful because the model is trained to understand structure. e.g.: https://docs.typesafe.ai/primitives/advanced#structured-instructions https://docs.typesafe.ai/primitives/advanced#structured-inst... > not sure if multi-modal just JSON... for now :) > outputs the question's answers as appropriate correct!
- big_toast 20d ago
  I assume this isn't really for consumers/individuals currently? Kinda feels like an improved magic 8 ball. I can't really intuit how I should think about when the model will be accurate. Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
  - zenlikethat 20d ago
    > I assume this isn't really for consumers/individuals currently? Unless they're hackers, no. It's not really a chat interface, it's meant for consumption by machines and composing into higher level systems (pairs great with LLMs). > Is there somewhere to read more about that? I assume customers would just have some tests or talk to you. We're going to release some more info on evaluations over time, and yeah, join the waitlist! We offer faster access in exchange for good memes

CompleteSkeptic 20d ago

CEO here - that is right! I do agree that the comparison to LLM tokens is hard to understand (also because output tokens are not comparable). But yes, text or structured state (like a JSON with multiple pieces of text in) -> decisions out (e.g. choice maps to "match" statement, "score" maps to sorting, "noul" short for bernoulli maps to if-statements)

ianbutler 20d ago
I see this super interestingly as the "subconscious" to the llms "conscious" for lack of better terms. I'm super interested in this for broad and rapid decision making in the context of consumer agents so will be signing up for sure.
- CompleteSkeptic 20d ago
  1. I am extremely on the same page 2. I do think that subconscious is not only much smarter than we give it credit for, but also much more robust than the "jagged frontier" of current LLMs (shilling my blog post on that jaggedness: https://www.completeskeptic.com/p/lies-damned-lies-and-benchmarks https://www.completeskeptic.com/p/lies-damned-lies-and-bench...)
- cheesecakegood 19d ago
  I wonder if a good analogy would be, they are trying to tune the “gut feeling” of a model more directly
deleted 20d ago
[deleted]
Flere-Imsaho 20d ago
Hi - first congratulations, System One looks really promising. The Doom demo really help me, at least, to understand how System One differs from LLMs. However the first demo (Side-by-side demonstration) - I'm struggling to understand what is going on here!
- potatoman22 20d ago
  I think that's to demonstrate its speed
- davideg 20d ago
  I was confused at first too, but it makes more sense when you read about their primitives. E.g. https://docs.typesafe.ai/primitives/noul https://docs.typesafe.ai/primitives/noul The demo is showing System One producing its output in parallel very quickly and for little cost compared to an LLM generating its answers token-by-token. The "noul" type is used to evaluate a yes/no question and return the probability that the answer is yes. So this demo is showing System One offering much more nuanced responses and specific probabilities compared to an LLM's more crude responses (e.g. LLM shows "true" or "false" compared to "0.9" or "0.07" probabilities that the answer to some question is true).
mckngbrd 20d ago
here is how I attempted to explain it to my company's AI group chat, is this roughly accurate? "instead of autoregressive string output it instead outputs structured type-safe 'decisions' with probabilities/confidence scores, each generated in parallel so sort of more like a Large Classification Model than a Large Language Model? or, maybe better to think of it as a sort of "shift left" in the LLM's transformer architecture, allowing you to replace the predefined token vocabulary of an LLM with a prescribed set of 'decisions' that need to be made based off the input context; and exposing those probabilities directly so they can be integrated into the system logic, instead of just sampling from top-K. all of this while still being instruction-tuned (!!!)" It's always been possible to build classification pipelines using LLM embeddings as the input. seems like this is a much more sophisticated / useful application of that concept
- CompleteSkeptic 20d ago
  very accurate! the one nuance I'd get into is I'd call it "zero-shot" over "instruction-tuned" (the latter often implies a particular distribution), but very safe for sharing
ttul 20d ago
For many day-to-day computing use cases, Jev seems far better suited than an autoregressive language model, if for no other reason than it is not wasting compute thinking about anything other than how to spit out a decision. Do you have an architectural explainer yet for Jev or are you holding that close to your chest and letting the magic rip for now?
- CompleteSkeptic 20d ago
  architecture is close to the chest for now, but we have talked about writing a paper I don't want to shill my blog too much, but I will say data is probably far most interesting than architecture: https://www.completeskeptic.com/p/the-bitterest-lesson https://www.completeskeptic.com/p/the-bitterest-lesson
  - ttul 20d ago
    Well, I'm stoked to try it out. We have about a billion reasons a day to call a model like this to rid the world of spam and phishing.
animan 20d ago
It's unclear if the context extends as the conversation grows?
mijoharas 20d ago
Small request, can we get an explanation of the naming of "noul" in the docs[0]. I tried googling, and searching the docs and didn't understand why it was called that. (I'd also argue something like p_yes or just probability might be a simpler name, but I'm sure there's a better reason behind Bernoulli maps). [0] https://docs.typesafe.ai/primitives/noul#noul https://docs.typesafe.ai/primitives/noul#noul
- safeandsound 20d ago
  Im going to guess bernoulli
  - mijoharas 20d ago
    It is Bernoulli (mentioned in the comment I replied to). I just failed to guess that myself :)
sedev 16d ago
You really, really, really, really, REALLY REALLY REALLY should explain neologisms in your documentation! Really really! Having "here's a weird unexplained difficult-to-guess-impossible-to-Google-for term" in your documentation is aggressively user-hostile. Please don't. The actual term is fine. Lack of an inline explanation is a blow-it move. Don't do that.

zenlikethat 20d ago

> the model takes as input a state (structured text? not sure if multi-modal) Input, and criteria/instructions can both be defined as structured input (JSON). This ends up being pretty powerful because the model is trained to understand structure. e.g.: https://docs.typesafe.ai/primitives/advanced#structured-instructions https://docs.typesafe.ai/primitives/advanced#structured-inst... > not sure if multi-modal just JSON... for now :) > outputs the question's answers as appropriate correct!

big_toast 20d ago
I assume this isn't really for consumers/individuals currently? Kinda feels like an improved magic 8 ball. I can't really intuit how I should think about when the model will be accurate. Is there somewhere to read more about that? I assume customers would just have some tests or talk to you.
- zenlikethat 20d ago
  > I assume this isn't really for consumers/individuals currently? Unless they're hackers, no. It's not really a chat interface, it's meant for consumption by machines and composing into higher level systems (pairs great with LLMs). > Is there somewhere to read more about that? I assume customers would just have some tests or talk to you. We're going to release some more info on evaluations over time, and yeah, join the waitlist! We offer faster access in exchange for good memes