4 ms·
I started to learn machine learning relatively seriously. But I've always had issues with frameworks where the team goes out of their way to be clever rather th
by corodra 7y ago
I started to learn machine learning relatively seriously. But I've always had issues with frameworks where the team goes out of their way to be clever rather than straightforward. ML frameworks to me are plagued with that. And I hate python. A lot. Mostly because of white space formatting.
So, I decided to make my own neural net in C#. For fun, it'll never be released. I spent a solid month learning anything and everything I could about how brains work in the animal kingdom. Then I built out a neural net according to what I learned. My cells aren't really similar to most of the conventional types out there. But it does work fairly well with numerical data. If I spent more time, like a solid year instead of spare time over 2 months, I think it could be respectable.
What I really learned from this project was optimization to the extreme. I spent a hell of a lot of time testing different ways to accomplish the same math and pull out as much performance as possible. I'd guess for every hour of code, I spent 4 or 5 hours research, testing and optimizing. Mostly because it's all CPU instead of GPU. I never got into cuda and I never will. It's not like I've never optimized before. The difference now, I spent time finding out if conventional wisdom was correct. Also, I discovered a bunch of methods in C# that I never knew about.
I dont do development anymore for work (and God willing, never will), so this was just a distraction/curiosity project for me. In reality, I wish I took the time early on in my career to do a project like this. Anyone fresh in dev needs to do a 3 to 6 month pure optimization project learning, for themselves, what works and what doesn't. Conventional wisdom really is only the tip of the iceberg.
- stormtroper1721 7y agoIs this on GitHub? I'd love to take a look since I'm thinking of making my own ml library to learn more about how frameworks like pytorch and tensorflow work
- corodra 7y agoNah. Like I said, it was a distraction project and I don't do dev for work anymore. Honestly, making a basic ML framework is not hard. I'm not the greatest programmer. Hell, I'd barely say I'm even decent. So if I could get something working, just about anyone can. My recommendation though, it must be kept neat and tidy. 100%. Plus, keep paper notes. It gets complex real fast. Spend your time on good naming conventions and being VERY well organized. Obviously all projects should be that way. But this is one where it's not really a recommendation. It's a flat out must. Don't fall into the trap of "hacking" in something to make it work. Not even once. If something is wrong, fix it completely the moment you notice it. My experience, neural net hacks compound really fast into trouble. Plus, I had fun with my naming conventions. My hidden layer cell class is called a "centralized understanding neural tracker" with the input cells called "data input command kontroller". As you can tell, I'm not a very mature adult. Pretty sure it wouldn't be appreciated in github.
- JamesBarney 7y agoI work with c# all the time and would love to hear some of your lessons learned on performance improvement.
- moritonal 7y agoOoh, could you write up a piece on this? Few people really know how to write speedy C# code!
- corodra 7y agoI guess if there's interest in it. Honestly, it's not like I'm that good at programming. Literally, all I did was second guess every line of code and operation I put down. I would try to think of alternatives and research for alternatives. Then test them in their basic forms. It's just grinding work with a giant spreadsheet, nothing special.