2 ms·

Large Language Models from scratch (2022)

peter_d_sherman 3y ago
Consider the following frame/time from the video: https://youtu.be/lnA9DMvHtfI?t=311 https://youtu.be/lnA9DMvHtfI?t=311 At this point in time in the video, we see the following 3 things: 1) Fourier Series 2) Taylor Series 3) Neural Network The remainder of the video goes on to show how Neural Networks can approximate functions -- much like the Fourier Series and Taylor Series can... Now, I don't hold an advanced Ph.D. degree in Math, and prior to watching this video, I had never made the connection between Fourier Series, Taylor Series, and Neural Networks... I knew about each of those independently -- but prior to watching the video, I never made the connection that they can be used to approximate one another. And that's the one of the key words -- approximate. The other key word is Identity -- as in Mathematical Identity... That's because IF two Mathematical Objects (functions, algorithms, call them what-you-will) are approximately equal -- then there should be a set of conditions (i.e., "If and only if...", etc.), a set of rigorously defined mathematical constraints which, if those conditions/constraints are met, then the two Mathematical Objects are exactly precisely equivalent -- under those rigorously defined conditions/constraints.... In other words, under those rigorously defined conditions/constraints -- one Mathematical Object could be substituted for another -- with no loss of precision or result. In other words, under those rigorously defined conditions/constraints -- those two completely apparently different Mathematical Objects -- function as an Identity. In other words, under those rigorously defined conditions/constraints -- you can use one to understand the other... You see, I really want to understand what ChatGPT and other AI models and their "hidden layers" -- are doing behind the scenes -- in terms of the most granular, most exacting, mathematical precision possible... Mathematical Identities -- for example, if we could swap out a Neural Network and replace it with a Taylor Series -- under specific conditions which guarantee exact, not approximate equality, would probably help a lot of people understand Neural Networks (and AI in general) better... I know that would be the case for me...

Consider the following frame/time from the video: https://youtu.be/lnA9DMvHtfI?t=311 https://youtu.be/lnA9DMvHtfI?t=311 At this point in time in the video, we see the following 3 things: 1) Fourier Series 2) Taylor Series 3) Neural Network The remainder of the video goes on to show how Neural Networks can approximate functions -- much like the Fourier Series and Taylor Series can... Now, I don't hold an advanced Ph.D. degree in Math, and prior to watching this video, I had never made the connection between Fourier Series, Taylor Series, and Neural Networks... I knew about each of those independently -- but prior to watching the video, I never made the connection that they can be used to approximate one another. And that's the one of the key words -- approximate. The other key word is Identity -- as in Mathematical Identity... That's because IF two Mathematical Objects (functions, algorithms, call them what-you-will) are approximately equal -- then there should be a set of conditions (i.e., "If and only if...", etc.), a set of rigorously defined mathematical constraints which, if those conditions/constraints are met, then the two Mathematical Objects are exactly precisely equivalent -- under those rigorously defined conditions/constraints.... In other words, under those rigorously defined conditions/constraints -- one Mathematical Object could be substituted for another -- with no loss of precision or result. In other words, under those rigorously defined conditions/constraints -- those two completely apparently different Mathematical Objects -- function as an Identity. In other words, under those rigorously defined conditions/constraints -- you can use one to understand the other... You see, I really want to understand what ChatGPT and other AI models and their "hidden layers" -- are doing behind the scenes -- in terms of the most granular, most exacting, mathematical precision possible... Mathematical Identities -- for example, if we could swap out a Neural Network and replace it with a Taylor Series -- under specific conditions which guarantee exact, not approximate equality, would probably help a lot of people understand Neural Networks (and AI in general) better... I know that would be the case for me...