8 ms·
DeepThought-8B: A small, capable reasoning model
- esteban78 2y ago[dead]
- 19911991 2y ago[flagged]
- tkgally 2y agoThere's been a rush of releases of reasoning models in the past couple of weeks. This one looks interesting, too. I found the following video from Sam Witteveen to be a useful introduction to a few of those models: https://youtu.be/vN8jBxEKkVo https://youtu.be/vN8jBxEKkVo
- wongarsu 2y agoA bit off-topic, but that comparison graph is a great example why you should buy your designer a cheap secondary screen. I was viewing it on my second monitor and had to lean in to make out the off-white bar for Model D on the light-grey background. Moved the window over to my main screen and it's clear as day, five nice shades of coffee on a light-gray background.
- AYBABTME 2y agoEven on a high quality screen, it's a bit much.
- peterkos 2y agoThat's a pretty egregious mistake for a designer to make -- and that's not even mentioning the lack of accessibility. WebAIM's contrast checker says it's a 1:1 contrast ratio! If someone is releasing a model that claims to have a level of reasoning, one would hope that their training dataset was scrutinized and monitored for unintended bias (as any statistical dataset is susceptible to: see overfitting). But if the graph on the announcement page is literally unreadable to seemingly anyone but the creator... that's damning proof that there is little empathy in the process, no?
- parhamn 2y ago> that's damning proof that there is little empathy in the process, no? No.
- hmottestad 2y agoI wouldn’t say it’s implied, but there’s a reason people put on nice clothes for an interview. I’m looking at the graphs on my phone and I’m pretty sure that there are 5 graphs and 3 labels. And their 8B model doesn’t seem to be very good, looks like a 20B model beats it in every single benchmark.
- pavlov 2y agoThe body text is also quite hard to read because the font has a tall x-height and line spacing is very tight. This makes paragraphs look very dense, almost like it was set in uppercase only, because the lowercase letters don’t create a varying flow between lines for the eye to follow. The model may be good, but the web design doesn’t win any prizes.
- xena 2y agoI have an iPhone 15 Pro Max and it took me five glances to see the last bar. Use black outlines or something idk
- imachine1980_ 2y agoi sadly don't feel this is a mistake, the transparent once are the two that beat the model in one category or more, its feels more like scam than error, if not please fix it
- lolinder 2y agoAlso, is it standard practice to obfuscate which models you're benchmarking against? They're just labeled Model A-D, with sizes but no additional information.
- sigmoid10 2y agoGiven the context, it appears they are not benchmarking against other models but comparing differently sized versions of the same model. The 8B one is just the one they decided to give a catchy name. The other ones are probably also just fine tuned Llama models. But without information on the total compute budget (i.e. nr. of trained tokens), this kind of plot is pretty useless anyways.
- deleted 2y ago[deleted]
- kgeist 2y agoNot bad, asked it to count Rs in "strawberry" and Ns in "international", it answered correctly, and it was fast.
- magicalhippo 2y agoDoes it still count correctly if you ask for variations like "strawerberry"?
- wongarsu 2y agoI found the following transcript of the thought process to that question equally enlightening and funny: Thought for 4 seconds 1 problem understanding The task is to count the number of instances of the letter 'r' in the word 'strawerberry'. This requires identifying each occurrence of 'r' in the given word and keeping a count. 2 data gathering The word to analyze is 'strawerberry'. I need to examine each character in this word. 3 hypothesis formation Based on a quick glance, I hypothesize that there are 2 instances of 'r' in 'strawerberry'. 4 analysis Let's break down the word into individual characters: s, t, r, a, w, e, r, b, e, r, r, y 5 calculation Now, let's count the 'r's: 1) First 'r' at position 3, 2) Second 'r' at position 7, 3) Third 'r' at position 10. Total count: 3 6 verification Double-checking the count: s(0), t(0), r(1), a(0), w(0), e(0), r(2), b(0), e(0), r(3), r(3), y(0). Confirmed: 3 instances of 'r'. 7 error checking The initial hypothesis was incorrect. The actual count (3) differs from the initial guess (2). This highlights the importance of careful analysis rather than quick assumptions. 8 conclusion drawing The letter 'r' appears 3 times in the word 'strawerberry'. 9 implementation The final response should be the number 3, as this is the correct count of 'r' instances in 'strawerberry'.
- Legend2440 2y agoI get: >Alright, I need to figure out how many 'r's are in the word 'strawerberry'. Let's look at it carefully. I'll write it down: s-t-r-a-w-e-r-b-e-r-r-y. Now, I'll go through each letter one by one. >Starting from the beginning: s, t, r – that's one 'r'. Then a, w, e, r – that's the second 'r'. Next, b, e, r – that's the third 'r'. And finally, another r before the y, making it the fourth 'r'. So, I count four 'r's in total. >The word "strawberry" contains four 'r's.
- jb_briant 2y agoAm I wrong to think that "reasoning model" is a misleading marketing term? Isn't it a LLM with an algo wrapper?
- viraptor 2y agoWhether you bake the behaviour in or wrap it in an external loop, you need to train/tune the expected behaviour. Generic models can do chain of thought if asked for, but will be worse than the specialised one.
- benchmarkist 2y agoThey're not baking anything in. Reasoning, as it is defined by AI marketing departments, is just beam search.
- jb_briant 2y agoCould you educate me on what is beam search ? Or link a good ressource EDIT: https://www.width.ai/post/what-is-beam-search https://www.width.ai/post/what-is-beam-search So the wider the beam, the better the outcome? Yep, no reasoning, just a marketing term to say "more accurate probabilities"
- benchmarkist 2y agoBeam search just traverses different paths and assigns each path a probability of being correct. The paths with the higher probabilities are kept and the ones with lower probabilities are pruned until the search terminates with an "answer". The marketing department calls it "reasoning" and "test-time compute" because the average consumer does not care whether it's beam search or something else. Your link seems to do a good job of explaining beam search but it's a classic algorithm in state space exploration so most books on search algorithms and discrete optimization will have a section about it.¹ 1: https://books.google.com/books?id=QzGuHnDhvZIC&q=%22beam%20search%22#v=snippet&q=%22beam%20search%22&f=false https://books.google.com/books?id=QzGuHnDhvZIC&q=%22beam%20s...
- CGamesPlay 2y agoIn what way did they "release" this? I can't find it in hugging face or ollama, and they only seem to have a "try online" link in the article. "Self-sovereign intelligence", indeed.
- underlines 2y agothey didn't
- wongarsu 2y agoThey released it in the same sense OpenAI released GPT4. There is an online demo you can chat with, and a form to get in touch with sales to get API access
- lowyek 2y agoI asked it 'find two primes whose sum is 123' .. it is in deep thought from 5 minutes just looping and looping over seemingly repeated hallucinations of right path. (btw, chatgpt immediately answers 61 and 62 lol.. so much for intelligence)
- utdiscant 2y agoHere is what I got from o1: To find two prime numbers that sum to 123, let’s consider the properties of prime numbers and the nature of their sums: 1. Prime Numbers Greater Than 2 Are Odd: Except for 2, all prime numbers are odd. 2. Sum of Two Odd Numbers Is Even: Adding two odd numbers results in an even number. 3. Sum of an Even and an Odd Number Is Odd: Adding an even number (like 2) and an odd number results in an odd sum. Given that 123 is an odd number, the only way two primes can sum to 123 is if one of them is 2 (the only even prime) and the other is an odd prime. Let’s test this possibility: • 2 + x = 123 ⇒ x = 121 However, 121 is not a prime number (since ). Next, consider the possibility of adding two odd primes: • Odd Prime + Odd Prime = Even Number Since adding two odd primes yields an even number, it’s impossible for two odd primes to sum to 123 (an odd number). Conclusion: There are no two prime numbers that sum to 123. Answer: There are no two prime numbers whose sum is 123; such primes do not exist.
- kenjackson 2y agoWell explained!
- meta_x_ai 2y ago[dead]
- alwayslikethis 2y agoqwq went through a two screenfuls of tokens to try all primes under 123 twice, but eventually gave the right answer.
- lowyek 2y agoI think may be i am not with the right model. the reasoning looks so good here. May be the default model in chatgpt free is not the one which you used.
- reissbaker 2y ago"Model A 13B", "Model B 20B" etc are pretty vapid claims. Which actual models? There are plenty of terrible high-param-count models from a year or two ago. The benchmark seems meaningless without saying what models are actually being compared against... And "13B" in particular is pretty sketchy: are they comparing it against Llama 2 13B? Even an untuned Llama 3.1 8B would destroy that in any benchmark. Smells a little grifty to me...
- chvid 2y agoIs the source code available for this? And who is behind the company?
- alexvitkov 2y agoPlease don't call model weights source code. Code is something you can (usually) read and understand, if anything weights are closer to a very obfuscated compiled binary - although even that can be analyzed by a good enough reverse engineer. "Open Source" models are the latest in a long series of attempts to take advantage of FOSS's reputation without actually releasing something that adheres to the principles.
- littlestymaar 2y agoTo run a language model you need both the model weights and the source code of the implementation!
- rkagerer 2y agoI would think truly open source means also including the initial training data. i.e. Everything needed to build[/tweak/customize] the thing from the ground up.
- littlestymaar 2y agoI agree, but I'm responding to someone confusing source code and model weights…
- exe34 2y agollama.cpp is open source!
- alexvitkov 2y agoNeat! But if we're sticking to the source code analogy, the matrix multiplier probably maps closer to a CPU or an interpreter - I wouldn't say every Java program is open source because you have an open-source JVM.
- sushidev 2y agoIt’s just a web page. How to try the model?
- Timwi 2y agoIt's a bit buried, but there's a link further down that takes you to an interface similar to ChatGPT. However, I have not been able to get any output from it; it seems to just think forever. I must be giving it the wrong prompts...
- nyoomboom 2y agoThe reasoning steps look reasonable and the interface is simple and beautiful, though Deepthought-8b fails to disambiguate the term "the ruliad" as the technical concept from Wolfram physics, from this company's name Ruliad. Maybe that isn't in the training data, because it misunderstood the problem when asked "what is the simplest rule of the ruliad?" and went on to reason about the company's core principles. Cool release, waiting for the next update.
- segalord 2y agoXd, Gotta love how your first question to a test a model is about a “ruliad”. It’s not even in my ios dictionary
- tanakai24 2y agoLegally, you cannot name the llama3 based models like that, YOu have to use, llama in the name https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct/blob/main/LICENSE https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct/blo...
- alexvitkov 2y agoToo bad :) Facebook trained the model on an Internet's worth of copyrighted material without any regard for licenses whatsoever - even if model weights are copyrightable, which is an open question, you're doing the exact same thing they did. Probably not a bulletproof legal defense though.
- tourmalinetaco 2y agoAt least Zuck had the decency to release model weights, unlike these worthless clowns.
- littlestymaar 2y agoCan't wait until Meta sue them so we can have a judgment on whether or not models weights are subject to copyright.
- euroderf 2y agoModel weights are (abstractly speaking) a very intensive, concentrated form of website scraping, yes ? What does the (USA) law say about scraping ? Does "fair use" play a role ?
- ranger_danger 2y agoYes, and there have already been court cases that ruled AI training of copyrighted data to be fair use, because it's technically no different than any other form of art, everything is based off of seeing other ideas elsewhere, there are no new ideas anymore.
- igleria 2y agoI asked it "Describe how a device for transportation of living beings would be able to fly while looking like a sphere" and it just never returned an output
- Timwi 2y agoI asked it to just count letters in a long word and it never returned an output (been waiting for 30 minutes now)
- m3kw9 2y agoIt isn’t pleased you ask it such questions
- ConspiracyFact 2y agoBlaine is a pain
- sans_souse 2y agoIt looks nice, but my chrome browser on android has all sorts of trouble with rendering the animated bits, so it ends up skipping frames throughout my navigating and clicks. Add to that; the model doesn't respond at all in my multiple attempts, it's a waste of time until it's remedied.
- codetrotter 2y agoGiven the name they gave it, someone with access should ask it for the “Answer to the Ultimate Question of Life, The Universe, and Everything” If the answer is anything other than a simple “42”, I will be thoroughly disappointed. (The answer has to be just “42”, not a bunch of text about the Hitchhikers Guide to the Galaxy and all that.)
- vintermann 2y agoDeep Thought didn't answer right away either.
- lowbloodsugar 2y ago“Right away”. lol.
- euroderf 2y agoI am very impressed. I asked chat.ruliad.co Beginning from the values for fundamental physical constants, is it possible to derive the laws of entropy ? and then based on its response to that I asked it Based on this analysis, can you identify and describe where the dividing line is between (a) the region where (microscopic/atomic) processes are reversible, and (b) the region where macroscopic processes are irreversible ?
- rkagerer 2y agoFor those of us who can't try it out ourselves, could you post the results somewhere and share a link?
- euroderf 2y agoIt's not giving me a link, and its responses are multi-step and wordy. I can post some pullquotes... Yes, it is possible to derive the laws of entropy from the fundamental physical constants. Entropy is a measure of the number of possible microscopic configurations of a system, and it is related to the fundamental physical constants through the Boltzmann constant (k_B) and the laws of thermodynamics. The second law of thermodynamics states that the total entropy of an isolated system can only increase over time, or remain constant if the system is in equilibrium. This law can be derived from the fundamental physical constants and the microscopic behavior of particles in a system. One way to derive the laws of entropy is by considering the statistical mechanics of a system. In statistical mechanics, entropy is related to the number of possible microscopic configurations (microstates) of a system, which can be described by the Boltzmann constant (k_B). The entropy (S) of a system is given by the Boltzmann formula: S = k_B * ln(Ω) where Ω is the number of possible microstates. The fundamental physical constants, such as the speed of light (c), the gravitational constant (G), and the Planck constant (h), are related to the other constants through various equations. For example, the Boltzmann constant (k_B) can be expressed in terms of the Planck constant (h), the speed of light (c), and the gravitational constant (G): k_B = (h * P / 2π) * (c^2 / G) where P is the pressure. By using these relationships and considering the microscopic behavior of particles in a system, it is possible to derive the laws of entropy from the fundamental physical constants. However, it is important to note that entropy is a macroscopic property, and its laws are not directly derived from the fundamental physical constants but rather from the microscopic behavior of particles and the laws of thermodynamics. [Q: Based on this analysis, can you identify and describe where the dividing line is between (a) the region where (microscopic/atomic) processes are reversible, and (b) the region where macroscopic processes are irreversible ?] The dividing line between the region where microscopic/atomic processes are reversible and the region where macroscopic processes are irreversible lies in the scale at which the processes occur and the interactions between particles. At the microscopic level (a), processes are generally considered reversible because the interactions between individual particles can be easily reversed. For example, when two particles collide, they can exchange momentum and energy, and the process can be considered reversible. In this region, the system is in thermodynamic equilibrium, and the second law of thermodynamics does not apply. However, as we move to the macroscopic level (b), the number of particles involved in the processes increases, and the interactions between them become more complex. In this region, the processes are considered irreversible because the energy and momentum exchanges between particles lead to an increase in entropy. This increase in entropy is a result of the second law of thermodynamics, which states that the total entropy of an isolated system can only increase over time or remain constant if the system is in equilibrium. The dividing line between reversible and irreversible processes can be identified by examining the scale at which the system operates and the interactions between particles. At the microscopic level, processes are generally reversible, while at the macroscopic level, processes are irreversible due to the increase in entropy resulting from the interactions between a large number of particles. -end-
- rkagerer 2y agoIs it possible to try it without logging in? Can you log in with anything other than a Google account? I was excited by the tagline "Self-Sovereign", but it appears this is not.
- andai 2y ago[flagged]
- lowbloodsugar 2y agoOh fuck me, it’s a fellow autistic. You’d think after all these years on the planet my brain would be able to immediately respond with “Fine. You?” but no, this is what my brain does before eventually replying.
- deleted 2y ago[deleted]
- alcover 2y agoDeepMasking
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- asah 2y ago"what is the population of manhattan below central park" ChatGPT-o1-preview: 647,000 (based on 2023 data, breaking it down by community board area): https://chatgpt.com/share/674b3f5b-29c4-8007-b1b6-5e0a4aeaf0e9 https://chatgpt.com/share/674b3f5b-29c4-8007-b1b6-5e0a4aeaf0... (this appears to be the most correct, judging from census data) DeepThought-8B: 200,000 (based on 2020 census data) Claude: 300-350,000 Gemini: 2.7M during peak times (strange definition of population !) I followed up with DeepThought-8B: "what is the population of all of manhattan, and how does that square with only having 200,000 below CP" and it cut off its answer, but in the reasoning box it updated its guess to 400,000 by estimating as a fraction of land area.