6 ms·

The commit messages across the entire project are... alarmingly insightful. There's some detailed stream of consciousness stuff in there.

by kgbier 8y ago

stochastic_monk 8y ago
I’m quite happy to see random projections getting some love, but I hope more people start using Choromanski et al.’s 2016 Structured Orthogonal Random Features, which has provably higher accuracy while reducing runtime to linearithmic and memory to linear (or constant) from quadratic for each. I’ve verified this experimentally in my implementation here [0]. As a shameless plug, it’s quite fast, is written in C++, and comes with Python bindings for both kernel projections and orthogonal JL transforms. [0]: https://github.com/dnbaker/frp https://github.com/dnbaker/frp
- wild_preference 8y ago
  How is this relevant to the upstream comment?
  - jpwagner 8y ago
    https://github.com/mrakgr/The-Spiral-Language/commit/d75e30403987df156e9f0de271abae5447e4e9b6 https://github.com/mrakgr/The-Spiral-Language/commit/d75e304...
- abstractcontrol 8y ago
  The reason why I am using random projections in the latest test is because I am testing an algorithm that iteratively calculates the inverse Cholesky factor of the covariance matrix and am testing it on Mnist images. The cov matrices made from raw Mnist images are non-invertible, but projecting them to a much smaller dimension allows me to actually test the algorithm on non-synthetic data. I do not actually need more than I have, but I'll keep your link in mind if I ever need random projections though.
  - stochastic_monk 8y ago
    That makes sense. Thank you for explaining! I hadn’t fully pieced it together from the commit. For dimensionality reduction, if one were to use my library, the way to go would be the orthogonal JL transform. The key to all these methods is the idea that multiplication by a diagonal matrix and subsequent F[HF]T is equivalent to multiplication by a matrix, which is what allows it to do what random matrix multiplication provides without instantiating or using the full matrix. As an aside, I admire your work and think it’s both very exciting and highly valuable.
jpwagner 8y ago
https://github.com/mrakgr/The-Spiral-Language/commit/51fe8b7f286590a83db5aa47d1da7fb11a96af56 https://github.com/mrakgr/The-Spiral-Language/commit/51fe8b7...
- jessep 8y ago
  If you haven't clicked through to read the above commit message, please do. It is amazing. This is a glimpse into the mind of a mad scientist (and I respect and love mad scientists more than anything and aspire to be one)
  - Razengan 8y ago
    > It is a tragedy, but I cannot imagine a language better than Spiral. Not within real world constraints facing me. A language better than it would pretty much have to be magic. > So, in this respect, my programming skill has reached it's limit and the most I will be able to do is refine it. > And it is embarrassing to me because I want to do better. I am ashamed, but the one to come will have to follow up on what I could not manage with my meager human abilities. This reminds me of the fictional inspirational process of dwarves in the Drizzt Do'Urden series of books [0] by R. A. Salvatore [1]. In that setting a dwarf craftsman gets only one moment of exceptional divine inspiration in his life to forge a legendary artifact. It is both a blessing and curse for them, because they realize that they will never be able to surpass that work for the remainder of their long lives. I wonder if other people here have had that moment in their life as well? [0] https://en.wikipedia.org/wiki/The_Icewind_Dale_Trilogy https://en.wikipedia.org/wiki/The_Icewind_Dale_Trilogy [1] https://en.wikipedia.org/wiki/R._A._Salvatore https://en.wikipedia.org/wiki/R._A._Salvatore
    - zwkrt 8y ago
      The dwarves should be thankful. Many people would have this moment and not realize it has passed them by.
  - s3m4j 8y ago
    https://github.com/mrakgr/The-Spiral-Language/commit/1dee010d0efe665d976ae619597997d87eba6161 https://github.com/mrakgr/The-Spiral-Language/commit/1dee010... That one is really in the "mad scientist" mindset. I love it, I wish him and this project good luck.
caseymarquis 8y ago
I'm inspired.I'm going to start writing little things about my day in my commit messages.
HerrMonnezza 8y ago
`git commit -m` as a blogging platform...

The commit messages across the entire project are... alarmingly insightful. There's some detailed stream of consciousness stuff in there.

by kgbier 8y ago

The commit messages across the entire project are... alarmingly insightful. There's some detailed stream of consciousness stuff in there.

stochastic_monk 8y ago

I’m quite happy to see random projections getting some love, but I hope more people start using Choromanski et al.’s 2016 Structured Orthogonal Random Features, which has provably higher accuracy while reducing runtime to linearithmic and memory to linear (or constant) from quadratic for each. I’ve verified this experimentally in my implementation here [0]. As a shameless plug, it’s quite fast, is written in C++, and comes with Python bindings for both kernel projections and orthogonal JL transforms. [0]: https://github.com/dnbaker/frp https://github.com/dnbaker/frp

wild_preference 8y ago
How is this relevant to the upstream comment?
- jpwagner 8y ago
  https://github.com/mrakgr/The-Spiral-Language/commit/d75e30403987df156e9f0de271abae5447e4e9b6 https://github.com/mrakgr/The-Spiral-Language/commit/d75e304...
abstractcontrol 8y ago
The reason why I am using random projections in the latest test is because I am testing an algorithm that iteratively calculates the inverse Cholesky factor of the covariance matrix and am testing it on Mnist images. The cov matrices made from raw Mnist images are non-invertible, but projecting them to a much smaller dimension allows me to actually test the algorithm on non-synthetic data. I do not actually need more than I have, but I'll keep your link in mind if I ever need random projections though.
- stochastic_monk 8y ago
  That makes sense. Thank you for explaining! I hadn’t fully pieced it together from the commit. For dimensionality reduction, if one were to use my library, the way to go would be the orthogonal JL transform. The key to all these methods is the idea that multiplication by a diagonal matrix and subsequent F[HF]T is equivalent to multiplication by a matrix, which is what allows it to do what random matrix multiplication provides without instantiating or using the full matrix. As an aside, I admire your work and think it’s both very exciting and highly valuable.

jpwagner 8y ago

https://github.com/mrakgr/The-Spiral-Language/commit/51fe8b7f286590a83db5aa47d1da7fb11a96af56 https://github.com/mrakgr/The-Spiral-Language/commit/51fe8b7...

jessep 8y ago
If you haven't clicked through to read the above commit message, please do. It is amazing. This is a glimpse into the mind of a mad scientist (and I respect and love mad scientists more than anything and aspire to be one)
- Razengan 8y ago
  > It is a tragedy, but I cannot imagine a language better than Spiral. Not within real world constraints facing me. A language better than it would pretty much have to be magic. > So, in this respect, my programming skill has reached it's limit and the most I will be able to do is refine it. > And it is embarrassing to me because I want to do better. I am ashamed, but the one to come will have to follow up on what I could not manage with my meager human abilities. This reminds me of the fictional inspirational process of dwarves in the Drizzt Do'Urden series of books [0] by R. A. Salvatore [1]. In that setting a dwarf craftsman gets only one moment of exceptional divine inspiration in his life to forge a legendary artifact. It is both a blessing and curse for them, because they realize that they will never be able to surpass that work for the remainder of their long lives. I wonder if other people here have had that moment in their life as well? [0] https://en.wikipedia.org/wiki/The_Icewind_Dale_Trilogy https://en.wikipedia.org/wiki/The_Icewind_Dale_Trilogy [1] https://en.wikipedia.org/wiki/R._A._Salvatore https://en.wikipedia.org/wiki/R._A._Salvatore
  - zwkrt 8y ago
    The dwarves should be thankful. Many people would have this moment and not realize it has passed them by.
- s3m4j 8y ago
  https://github.com/mrakgr/The-Spiral-Language/commit/1dee010d0efe665d976ae619597997d87eba6161 https://github.com/mrakgr/The-Spiral-Language/commit/1dee010... That one is really in the "mad scientist" mindset. I love it, I wish him and this project good luck.

caseymarquis 8y ago

I'm inspired.I'm going to start writing little things about my day in my commit messages.

HerrMonnezza 8y ago

`git commit -m` as a blogging platform...