Certainly more than 6. It depends on what you're going for. If you're just trying out a new technique (e.g. a new activation function, etc.) super-deep nets don

Certainly more than 6. It depends on what you're going for. If you're just trying out a new technique (e.g. a new activation function, etc.) super-deep nets don't really matter. But if you're trying to get the best performance possible on a task (for a competition, say), you can tack on many more layers (MSR did ~150 layers).