4 ms·

What kind of insight into whether deep networks are interesting or not does the fact that they are (supposedly) not used in the industry bring to the table? To

by stiff 14y ago

What kind of insight into whether deep networks are interesting or not does the fact that they are (supposedly) not used in the industry bring to the table? To me, insight would be something like "they do not generalize well unless you have really a lot of data" or anything concretely related to the research done on them. By the way, I am not asking anyone what their degree is, I am just asking to bring arguments and experiences or stay neutral.

rm999 14y ago
You missed the point of my comment. I was speaking about the hype, I wasn't putting down deep networks or making a technical analysis of the algorithms. Anyway, here are my major reservations, including my technical analysis. You're oddly demanding in a casual non-technical online discussion about a topic you don't seem qualified in, so if this isn't enough for you we really aren't going to find common ground. There isn't great library support for deep networks. This is a big deal, I don't want to spend tons of time building my own library or working with buggy/poorly supported/infant libraries. In production systems we prefer extremely well-established libraries that work in our language/environment of choice. Also deep belief networks are a couple orders of magnitude slower than linear models (probably the most commonly used type of model in industry). They require more parameter selection. They're not even useful for a lot of tasks - if I'm already spending tons of time building useful features (often a requirement in industry for non-technical reasons, like reporting or legal constraints) deep networks aren't going to be very useful. Much of their utility is taking raw, unstructured data and creating useful features for a supervised model. You can't easily interpret them as models unless you are working with visual data, they are a black box.

by stiff 14y ago

rm999 14y ago

You missed the point of my comment. I was speaking about the hype, I wasn't putting down deep networks or making a technical analysis of the algorithms. Anyway, here are my major reservations, including my technical analysis. You're oddly demanding in a casual non-technical online discussion about a topic you don't seem qualified in, so if this isn't enough for you we really aren't going to find common ground. There isn't great library support for deep networks. This is a big deal, I don't want to spend tons of time building my own library or working with buggy/poorly supported/infant libraries. In production systems we prefer extremely well-established libraries that work in our language/environment of choice. Also deep belief networks are a couple orders of magnitude slower than linear models (probably the most commonly used type of model in industry). They require more parameter selection. They're not even useful for a lot of tasks - if I'm already spending tons of time building useful features (often a requirement in industry for non-technical reasons, like reporting or legal constraints) deep networks aren't going to be very useful. Much of their utility is taking raw, unstructured data and creating useful features for a supervised model. You can't easily interpret them as models unless you are working with visual data, they are a black box.