5 ms·

This problem strikes me as relatively simple. What about more complex math problems? Are there good benchmarks for that? I would dearly love to have an AI tool

by s1mon 2y ago

This problem strikes me as relatively simple. What about more complex math problems? Are there good benchmarks for that? I would dearly love to have an AI tool that I could trust to help with math. What is the state of the art? My math skills are very rusty (the last math class I took was calculus almost 40 years ago), and I find myself wanting to do things which would require a PhD level understanding of computer aided geometric design. If I had the magical AI which really understood a ton of math and/or could be fed the appropriate research papers and could help me, that would be amazing. So far all my attempts with ChatGPT 4 and 4o have been confusing because I don't really trust or fully understand the results. This simple example and the frequency of wrong answers drives home the fact that I shouldn't trust ChatGPT for math help.

barkingcat 2y ago
Have you tried actual math tools like wolfram alpha? My response to attempts to use llms to do math is usually, use the right tool for the job.
- slushy-chivalry 2y ago
  "a PhD level understanding of computer aided geometric design" could you elaborate? I found chatgpt to pretty useful in high level understanding of where the topics stand in the grand scheme of things kinda like a pre-google -- it helps me navigate what to google let's ask gpt4o You're an expert math researcher. - Give me an overview of topos theory - What are the main unsolved problems - What is the point of it? - What research directions are most promising? - What are the math prerequisites to understanding it? output Overview of Topos Theory Topos theory is a branch of mathematics that generalizes set theory and category theory, providing a framework for analyzing and understanding mathematical structures. A topos can be seen as a generalized space that behaves much like the category of sets but with more flexible and abstract properties. Topos theory was developed by Alexander Grothendieck and his collaborators in the 1960s, initially to solve problems in algebraic geometry. Key concepts in topos theory include: Categories: Collections of objects and morphisms (arrows) between them. Sheaves: Structures that systematically organize data over a topological space. Topoi (plural of topos): Categories that satisfy certain axioms making them similar to the category of sets. Main Unsolved Problems Some of the main unsolved problems in topos theory involve: Classifying all possible topoi: Understanding the full landscape of possible topoi and their interrelationships remains a complex challenge. Connections to other areas of mathematics: Fully elucidating the connections between topos theory and other fields such as logic, algebraic geometry, and mathematical physics. Homotopy theory in the context of topoi: Developing a deeper understanding of homotopy theory within the framework of higher topos theory. The Point of Topos Theory The point of topos theory is to provide a unifying and flexible framework for various mathematical concepts and structures. It offers a generalized setting in which both geometric and logical ideas can be expressed and related. This has significant implications for: Algebraic geometry: Offering new tools and perspectives. Mathematical logic: Providing a categorical framework for logic and set theory. Theoretical computer science: Influencing the development of type theory and the semantics of programming languages. Promising Research Directions Some of the most promising research directions in topos theory include: Higher Topos Theory: Extending the classical theory to higher categories and understanding its implications in algebraic geometry and homotopy theory. Connections with Logic: Exploring further connections between topos theory and logic, particularly in relation to intuitionistic logic and type theory. Topos-theoretic Approaches to Physics: Investigating how topos theory can be applied to quantum mechanics and other areas of theoretical physics. Descent Theory and Stacks: Further developing the applications of topos theory in descent theory and the study of stacks in algebraic geometry. Math Prerequisites to Understanding Topos Theory To understand topos theory, one should have a solid background in the following areas: Category Theory: Fundamental concepts such as categories, functors, natural transformations, limits, and colimits. Set Theory: Basic set-theoretic concepts and operations. Algebraic Topology: Basic knowledge of topological spaces, sheaves, and cohomology. Algebra: Basic group theory, ring theory, and module theory. Logic: Understanding of basic logical systems, particularly intuitionistic logic. With these prerequisites, one can start delving into the more advanced and abstract concepts of topos theory. not perfect but good enough to get started
  - mvdtnz 2y ago
    Please don't pollute comment sections with gpt output.
  - Chinjut 2y ago
    That's fine but it's about the same as you'll get from an encyclopedia also, which makes sense as that's just where GPT got it from anyway. Nothing revolutionary in the ability to read encyclopedia articles. We've had that forever.
    - slushy-chivalry 2y ago
      sure, but with like a 100x improvement in usability -- chatgpt is helpful in figuring out what stuff to read (at least for me) so that when I go to the actual paper or a book I know what to focus on otherwise you can say "why do you need google, it's the same as you'll get from the website" moreover, I found that chatgpt is pretty decent at rephrasing a convoluted concept or a paragraph in a research paper, or even giving me ideas on the research directions I mean, same with coding -- I treat it as a smart autocomplete I could go to google and look for a .csv containing a list of all US States Or, I can write const US_STATES = [ and let copilot complete it for me -- 5 minutes saved?
  - s1mon 2y ago
    Specifically, I was trying to get help from ChatGPT to give a simple formula for the location of the P3 control point of a degree 3 (or higher) Bézier curve in order to maintain G3 continuity (given the derivatives at the end of the adjacent curve). There's a very straightforward equation for the P2 control point for G2 continuity, but I've been struggling to understand the math for G3 continuity. I've found a ton of research papers and information, but most of it is quickly beyond my ability to digest. For G2 constraints, there is simple equation: K(t0) = ((n-1)/n)*(h/a^2) Where n is the degree of the curve, a is the length of the first leg of the control polygon, and h is the perpendicular distance from P, to the first leg of the control polygon. K(t0) is the curvature at the end point of the adjacent curve. Depending on what you want to do, it's easy to solve for K(t0), a or h. I would like something this simple for G3.
- s1mon 2y ago
  I have tried to use Wolfram Alpha inside of ChatGPT, but that didn't get me very far. It seems like I would need to understand a lot more math to be able to do anything useful with Wolfram Alpha, and perhaps it would be better to run it stand alone not as a plugin.
  - jiggawatts 2y ago
    Ask it to write you the Wolfram language code and then verify it and execute it yourself. I’ve found that I can work 100x faster with Mathematica this way and solve problems that I wouldn’t have bothered to attempt otherwise. This is particularly effective for quickly visualising things, I’m too lazy to figure out all the graphing options for esoteric scenarios but GPT 4 can quickly iterate over variants given feedback.
xanderlewis 2y ago
ChatGPT has an amazing ability to write, but you shouldn't trust it for any form of mathematics aside from providing vague descriptions of what various topics are about (and even that tends to result in a word soup that is more flowery than descriptive). When it comes to solving specific problems, or even providing specific examples of mathematical objects, it falls down really quickly. I'll inevitably be told otherwise by some ChatGPT-happy hypebro, but LLMs are hopeless when it comes to anything requiring reasoning. Scaling it up will lessen the chance of a cock-up, but anything vaguely out of distribution will result in the same nonsense we're all used to by now. Those who say otherwise very likely just lack the experience or knowledge necessary to challenge the model enough or interpret the results. As a test of this claim: please comment below if you, say, have a degree in mathematics and believe LLMs to be reliable for 'math help' (and explain why you think so). We need a better technology! And when this better technology finally comes along, we'll look back at pure LLMs and laugh about how we ever believed we could magic such a machine into existence just by pouring data into a model originally designed for machine translation.
- fragmede 2y ago
  > ChatGPT-happy hypebro Rude. From the guidelines: > Please don't sneer, including at the rest of the community. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html "math help" is really broad, but if you add "solve this using python", chatgpt will generate code and run that instead of trying to do logic as a bare LLM. There's no guarantee that it gets the code right, so I won't claim anything about its reliability, but as far as pure LLMs having this limitation and we need a better technology, that's already there, it's to run code the traditional way.
  - xanderlewis 2y ago
    You’re right, but I get frustrated by the ignorance and hubris of some people. Too late to edit now.
- nurple 2y ago
  I'm with you. The thing I find baffling is how anyone with any logical sense finds chatGPT useful for anything that requires precision, like math and code. If you do indeed follow the caveats that the LLM companies require placing alongside any output: to not rely on it, and verify it yourself, then you already have to be skilled enough to detect problems, and if you are that skilled, the only way to check the output is to do the work again yourself! So, umm, where's the savings? You can't not do the work to check the output, and a novice just can't check at all... I have personally been brought into a coding project created by a novice using GPT4, and I was completely blown away by how bad the code was. I was asked to review the code because the novice dev just couldn't get the required functionality to work fully. Turns out that since he didn't understand the deployment platform, or networking, or indeed the language he was using, that there was actually no possible way to accomplish the task with the approach him and the LLM had "decided" on. He had been working on that problem for three weeks. I leveraged 2 off-the-shelf tools and had a solve from scratch in under a full day's work, including integration testing.

by s1mon 2y ago

barkingcat 2y ago

Have you tried actual math tools like wolfram alpha? My response to attempts to use llms to do math is usually, use the right tool for the job.

slushy-chivalry 2y ago
"a PhD level understanding of computer aided geometric design" could you elaborate? I found chatgpt to pretty useful in high level understanding of where the topics stand in the grand scheme of things kinda like a pre-google -- it helps me navigate what to google let's ask gpt4o You're an expert math researcher. - Give me an overview of topos theory - What are the main unsolved problems - What is the point of it? - What research directions are most promising? - What are the math prerequisites to understanding it? output Overview of Topos Theory Topos theory is a branch of mathematics that generalizes set theory and category theory, providing a framework for analyzing and understanding mathematical structures. A topos can be seen as a generalized space that behaves much like the category of sets but with more flexible and abstract properties. Topos theory was developed by Alexander Grothendieck and his collaborators in the 1960s, initially to solve problems in algebraic geometry. Key concepts in topos theory include: Categories: Collections of objects and morphisms (arrows) between them. Sheaves: Structures that systematically organize data over a topological space. Topoi (plural of topos): Categories that satisfy certain axioms making them similar to the category of sets. Main Unsolved Problems Some of the main unsolved problems in topos theory involve: Classifying all possible topoi: Understanding the full landscape of possible topoi and their interrelationships remains a complex challenge. Connections to other areas of mathematics: Fully elucidating the connections between topos theory and other fields such as logic, algebraic geometry, and mathematical physics. Homotopy theory in the context of topoi: Developing a deeper understanding of homotopy theory within the framework of higher topos theory. The Point of Topos Theory The point of topos theory is to provide a unifying and flexible framework for various mathematical concepts and structures. It offers a generalized setting in which both geometric and logical ideas can be expressed and related. This has significant implications for: Algebraic geometry: Offering new tools and perspectives. Mathematical logic: Providing a categorical framework for logic and set theory. Theoretical computer science: Influencing the development of type theory and the semantics of programming languages. Promising Research Directions Some of the most promising research directions in topos theory include: Higher Topos Theory: Extending the classical theory to higher categories and understanding its implications in algebraic geometry and homotopy theory. Connections with Logic: Exploring further connections between topos theory and logic, particularly in relation to intuitionistic logic and type theory. Topos-theoretic Approaches to Physics: Investigating how topos theory can be applied to quantum mechanics and other areas of theoretical physics. Descent Theory and Stacks: Further developing the applications of topos theory in descent theory and the study of stacks in algebraic geometry. Math Prerequisites to Understanding Topos Theory To understand topos theory, one should have a solid background in the following areas: Category Theory: Fundamental concepts such as categories, functors, natural transformations, limits, and colimits. Set Theory: Basic set-theoretic concepts and operations. Algebraic Topology: Basic knowledge of topological spaces, sheaves, and cohomology. Algebra: Basic group theory, ring theory, and module theory. Logic: Understanding of basic logical systems, particularly intuitionistic logic. With these prerequisites, one can start delving into the more advanced and abstract concepts of topos theory. not perfect but good enough to get started
- mvdtnz 2y ago
  Please don't pollute comment sections with gpt output.
- Chinjut 2y ago
  That's fine but it's about the same as you'll get from an encyclopedia also, which makes sense as that's just where GPT got it from anyway. Nothing revolutionary in the ability to read encyclopedia articles. We've had that forever.
  - slushy-chivalry 2y ago
    sure, but with like a 100x improvement in usability -- chatgpt is helpful in figuring out what stuff to read (at least for me) so that when I go to the actual paper or a book I know what to focus on otherwise you can say "why do you need google, it's the same as you'll get from the website" moreover, I found that chatgpt is pretty decent at rephrasing a convoluted concept or a paragraph in a research paper, or even giving me ideas on the research directions I mean, same with coding -- I treat it as a smart autocomplete I could go to google and look for a .csv containing a list of all US States Or, I can write const US_STATES = [ and let copilot complete it for me -- 5 minutes saved?
- s1mon 2y ago
  Specifically, I was trying to get help from ChatGPT to give a simple formula for the location of the P3 control point of a degree 3 (or higher) Bézier curve in order to maintain G3 continuity (given the derivatives at the end of the adjacent curve). There's a very straightforward equation for the P2 control point for G2 continuity, but I've been struggling to understand the math for G3 continuity. I've found a ton of research papers and information, but most of it is quickly beyond my ability to digest. For G2 constraints, there is simple equation: K(t0) = ((n-1)/n)*(h/a^2) Where n is the degree of the curve, a is the length of the first leg of the control polygon, and h is the perpendicular distance from P, to the first leg of the control polygon. K(t0) is the curvature at the end point of the adjacent curve. Depending on what you want to do, it's easy to solve for K(t0), a or h. I would like something this simple for G3.
s1mon 2y ago
I have tried to use Wolfram Alpha inside of ChatGPT, but that didn't get me very far. It seems like I would need to understand a lot more math to be able to do anything useful with Wolfram Alpha, and perhaps it would be better to run it stand alone not as a plugin.
- jiggawatts 2y ago
  Ask it to write you the Wolfram language code and then verify it and execute it yourself. I’ve found that I can work 100x faster with Mathematica this way and solve problems that I wouldn’t have bothered to attempt otherwise. This is particularly effective for quickly visualising things, I’m too lazy to figure out all the graphing options for esoteric scenarios but GPT 4 can quickly iterate over variants given feedback.

xanderlewis 2y ago

ChatGPT has an amazing ability to write, but you shouldn't trust it for any form of mathematics aside from providing vague descriptions of what various topics are about (and even that tends to result in a word soup that is more flowery than descriptive). When it comes to solving specific problems, or even providing specific examples of mathematical objects, it falls down really quickly. I'll inevitably be told otherwise by some ChatGPT-happy hypebro, but LLMs are hopeless when it comes to anything requiring reasoning. Scaling it up will lessen the chance of a cock-up, but anything vaguely out of distribution will result in the same nonsense we're all used to by now. Those who say otherwise very likely just lack the experience or knowledge necessary to challenge the model enough or interpret the results. As a test of this claim: please comment below if you, say, have a degree in mathematics and believe LLMs to be reliable for 'math help' (and explain why you think so). We need a better technology! And when this better technology finally comes along, we'll look back at pure LLMs and laugh about how we ever believed we could magic such a machine into existence just by pouring data into a model originally designed for machine translation.

fragmede 2y ago
> ChatGPT-happy hypebro Rude. From the guidelines: > Please don't sneer, including at the rest of the community. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html "math help" is really broad, but if you add "solve this using python", chatgpt will generate code and run that instead of trying to do logic as a bare LLM. There's no guarantee that it gets the code right, so I won't claim anything about its reliability, but as far as pure LLMs having this limitation and we need a better technology, that's already there, it's to run code the traditional way.
- xanderlewis 2y ago
  You’re right, but I get frustrated by the ignorance and hubris of some people. Too late to edit now.
nurple 2y ago
I'm with you. The thing I find baffling is how anyone with any logical sense finds chatGPT useful for anything that requires precision, like math and code. If you do indeed follow the caveats that the LLM companies require placing alongside any output: to not rely on it, and verify it yourself, then you already have to be skilled enough to detect problems, and if you are that skilled, the only way to check the output is to do the work again yourself! So, umm, where's the savings? You can't not do the work to check the output, and a novice just can't check at all... I have personally been brought into a coding project created by a novice using GPT4, and I was completely blown away by how bad the code was. I was asked to review the code because the novice dev just couldn't get the required functionality to work fully. Turns out that since he didn't understand the deployment platform, or networking, or indeed the language he was using, that there was actually no possible way to accomplish the task with the approach him and the LLM had "decided" on. He had been working on that problem for three weeks. I leveraged 2 off-the-shelf tools and had a solve from scratch in under a full day's work, including integration testing.