8 ms·
Tinn: A tiny neural network library written in C99
- ptr 8y agoJust curious, is there a reason for passing around the Tinn struct by-value? Will it get optimized?
- rjeli 8y agoIt’s only a few bytes, pointers to buffers
- yvdriess 8y agoIt's commonly optimized out, yes. Rule of thumb is typically to pass pointers to a struct if it gets fairly large and complex. Even then, you should really profile to see that the becomes a bottleneck, compilers inline aggressively and a pointer might defeat some other optimizations down the call chain.
- tom_mellior 8y agoIt might get optimized by inlining, these functions are pretty small and would make good candidates. Passing small structs by value isn't as expensive as it once was, modern ABIs pass the initial fields in registers. However, this struct does have ten pointer/integer fields, and I'm not aware of a mainstream ABI that allows that many. x86-64 allows up to six on Linux, IIRC. A quick check of the AArch64 ABI says it takes up to eight. So yeah, there will be stack traffic involved if these function calls are not inlined.
- kbumsik 8y agoI'm curious too. Even if it will be optimized isn't it still be better to pass-by-reference?
- skrebbel 8y agoIn C?
- kbumsik 8y agoI mean pointer.
- tom_mellior 8y agoIt might be easier to answer if you explained why you think it would be better. There may be cases where by-reference (yes, that's an absolutely acceptable thing to say, even in C) is more efficient: If the struct is a kind of "this pointer" (as in object-oriented programming) that is passed to a bunch of functions but relatively rarely used, then passing it by reference causes much less register pressure. Passing it by value would mean occupying a lot of registers with values that are probably not accessed. On the other hand, if you expect the callee to do actual computation on all or most of the fields of the struct you pass in, passing it by value should probably be the way to go. This way, the values to be processed are already in registers and don't have to be stored to the stack by the caller and loaded back from the stack by the callee. There is no general answer that is uniformly best, it depends on the characteristics of the actual program and their interactions with the compiler's optimizations. With inlining in particular, which should often allow the compiler to generate the same code for both versions.
- foo101 8y ago> There may be cases where by-reference (yes, that's an absolutely acceptable thing to say, even in C) I don't want to be nit-picking. But I would like to know why "by-reference" is an acceptable thing to say. When I learnt C, C++, Java, etc. I was repeatedly told that pass by reference exists only in C++. In C, a pointer is passed by value. In Java, a reference is passed by value. Doesn't calling "pass pointer by value" as "pass by reference" lead to confusion and distortion of the true meaning of "by reference"?
- pubby 8y agoWhat do you do when you want to find the value a pointer points to? You dereference it of course! C++ and Java have their own specific language-lawyer definitions of the word reference, but outside of those languages, "reference" is a catch-all term that includes pointers, handles, and indexes.
- robotresearcher 8y agoC implements the concept 'pass by reference' using the mechanism of 'pointer passed by value'. You have no problem using the terms 'loop' or 'recurse' to describe C code, right? But these high-level ideas are implemented without using those terms in the code. Same with 'pass by reference'.
- cubano 8y agoI would expect it would depend on what the function actually did with the data.
- glouwbug 8y agoHi, I'm glouw. Structs on the stack I pass by value. Structs on the heap I pass by pointer. It's a mental thing that I find helps with large codebases.
- offbytwo 8y agoUnrelated but thanks for making this! The code is clean and easy to follow. - a student trying to get a better grasp on neural networks
- sanxiyn 8y agoHow about in 12 kilobytes of assembly? More featureful implementation here: https://github.com/dfouhey/caffe64 https://github.com/dfouhey/caffe64
- p1esk 8y agoThis is a lot more impressive! I'm puzzled by this bit in FAQ though: Q. It seems that it only does 1D feature vectors? A. Convolution and other inductive biases are only necessary when you have small data.
- Annatar 8y ago“Installing a new neural network library is typically a tremendous pain because of onerous dependencies, python version hell, complicated Makefiles, and massive code bloat.” Finally people have started speaking up and doing something about it! May we see more of such mentality in the future, and computer industry will be a better place to work!
- p1esk 8y agoSingle core? No AVX? What's the point of writing it in C?
- sanxiyn 8y agoIt's a demo, like https://github.com/rswier/c4 https://github.com/rswier/c4.
- deleted 8y ago[deleted]
- Annatar 8y agoMinimal dependencies; people are starting to get fed up with having to bootstrap and/or compile the planet just to be able to run some software. The situation has gotten completely out of hand, especially if one is not on GNU/Linux.
- AstralStorm 8y agoThis does nor preclude the use of threads or SIMD. Essentially this library is a demo toy that will need a lot to work to make useful in real conditions.
- purerandomness 8y agoThat's exactly the point.
- taneq 8y ago> Essentially this library is a demo toy It's 200 lines of C, what were you expecting?
- didymospl 8y agoGenerally when I see links to github projects on HN I expect something useful or extraordinary and this looks like my Neural Networks 101 assignment, only far more polished. But people seem to like it(over 400 stars right now) so maybe I am missing something.
- chestervonwinch 8y agoTo the author: in your github topics, "propogation" -> "propagation".
- glouwbug 8y agoThankyou, my friend ;)
- stochastic_monk 8y agokann [0] is a similarly small neural network library written in pure C. It's quite fast for CPU-only, but only applicable for rather small problems. [0] https://github.com/attractivechaos/kann https://github.com/attractivechaos/kann
- attractivechaos 8y agoIt is fair to say KANN "only applicable for rather small problems". However, KANN is not so "similar" to Tinn. Tinn only implements MLP with a single hidden layer. KANN supports a lot more features such as 1D/2D convolution, RNN/LSTM, weight sharing and mini-batching. It is also more optimized.
- stochastic_monk 8y agoYou're absolutely right that kann is more optimized and full-featured. I thought to mention your library because it is also a very portable neural network library written in C. kann is a stunning piece of engineering, and your code is magnificent, and that is precisely why I mentioned it. I apologize, I did not clearly express my intent.
- attractivechaos 8y agoI just meant to add some clarification. I actually do appreciate that you mentioned it. Thank you!
- deleted 8y ago[deleted]
- Fethbita 8y agogenann is also a similar small neural network library written in C. https://github.com/codeplea/genann https://github.com/codeplea/genann
- dvdplm 8y agoWhat exactly is the output of the example in `test.c`? I.e. what do the numbers mean? The error in the prediction I guess, but the prediction of what? What is the first entry in the semion data set? a "0"?
- silval 8y agoNo kidding. Does the output mean "hot dog" or "not hot dog"?
- zimpenfish 8y agoThe first row of output is the vector of numerals 0-9 where 1.0 indicates "it was this digit." The second row is the probability of each given the input. e.g. This output means it's 98.6% certain that the input represents the digit 7. 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 1.000000 0.000000 0.000000 0.000009 0.000005 0.000000 0.000000 0.003899 0.012933 0.000436 0.986013 0.000005 0.000000
- lavabender 8y agoCranium is another alternative that uses BLAS for efficient matrix operations. https://github.com/100/Cranium https://github.com/100/Cranium