4 ms·

I agree with what you said. I worked with .net since the 1.0 days and after years of staunch support, I got a job last year working with python/javascript on a

I agree with what you said. I worked with .net since the 1.0 days and after years of staunch support, I got a job last year working with python/javascript on a unix stack and kissed microsoft technologies goodbye. You're absolutely right about Microsoft being behind the curve. There's a pattern of denial when it comes to new technologies, and a total ineptitude at spotting new trends and adapting to them. I watched with envy for years as all the rails kids played with their new toys until Microsoft got its shit together and came up with an MVC solution. The whole world is using flash? Lets come up with a knock-off 5 years too late. Interactivity is a big deal? Let's try to convince them that webforms is MVC until its too late. Who needs javascript anyways? Distrubuted software you say? Let's not jump on the REST bandwagon, lets make WSDL defined web services and make a shitty API on top! Why have an ORM solution when putting all of your business logic in SQL is SO EASY with microsoft? See, I can drag and drop a database connection from Visual Studio! I love C# as much as the next person. It's a great language. SQL Server is a pretty good relational database. But if you choose to devote all of your energy to Microsoft development and don't learn anything else, you have nobody to blame expect for yourself when you find your skill-set behind the curve. .Net developers would never agree about the big-data part, because to be honest they haven't invested time into understanding what data science is. There are so many IT shops developing on the Microsoft stack that make less than say 50 million a year that have tons of data (billions of records they say!). Maybe they make CRM software, or school software, or an inventory system for a small grocery store. They have tons of test scores, purchase records, waste numbers, demographic info, yet if you ask them to develop something that gives you insight into what factors influence student performance, or to visualize perchasing trends, or to predict sales numbers so that the company doesn't over purchase perishables, you'll get a blank stare. The idea that data can be transformed into useful information, and that this isn't simply a matter of CRUD hasn't occurred to them. They probably don't even replicate their db for their half ass attempt at reporting, so they wouldn't begin to understand what something like hadoop even does. You simply CANNOT explain to a .net zealot what big data is. They just don't get it. "It's stored in the database, and its a few terabytes...how big does it need to get? We aren't Google afterall!"

stirno 13y ago
Your statement is an incredible generalization of a large group of developers. It may be your experience, but I have to counter it with my own. While the Windows/.NET stacks may not provide as many pre-built solutions to handle 'big data' (a term I'm growing to hate), I have worked with tons of competent developers who have been able to build solutions to answer exactly the kinds of questions you propose. I've seen it done with SQL Server Reporting Services, Analysis Services and even straight custom C# code. I've seen it done with Hadoop + streaming API, StreamInsight (CEP rather than post processing) and custom Workflows in WF. As for your other statements that are just built to jump on the 'hate Microsoft' bandwagon, just because a single large company doesn't iterate fast enough to keep up with the rest of the OSS landscape doesn't mean that .NET didn't have options available to it. Nancy provides great REST capabilities and has existed since 2010. Devs were building REST services with Microsoft's ASP.NET MVC during the same period. Others were using WCF REST (not great, but usable) at the same point as well. WCF Data Services was another option. ORMs.. Theres the ever-expanding Entity Framework (which I don't love), NHibernate and many many others. Lets not just jump in and bash MS because you've only worked with bad developers using their stuff.
graycat 13y ago
> what factors influence student performance, or to visualize perchasing trends, or to predict sales numbers so that the company doesn't over purchase perishables "Big data"? Those are all just everyday, common, ordinary, vanilla-pure, old-fashioned exercises in statistics with maybe a little optimization thrown in, that is, the 'mathematical sciences' or, if you will, with a lot of overlap of 'operations research'. But long the standard remarks were that "operations research is dead" and "statistics died long ago except for some of biomedical statistics", e.g., for doing FDA approved 'trials'. Generally the amount of relevant data available and needed for such work is not very 'big'; yes, maybe the relevant data comes from a database of 10 TB, but a simple SQL query should be able to extract into, say, just a simple 'flat' file, maybe comma separated values or name-type-value triples, what is relevant. From the OP > 4. We need 10x or 100x more data scientists and AI specialists Again not a prediction. We are already seeing this. So perhaps I can sharpen some of my old skills in computer vision. Ah, come ON! So, all of a sudden college courses in undergraduate, junior level in mathematical statistics -- weak/strong laws of large numbers, Lindeberg-Feller central limit theorem, Neyman-Pearson lemma, Cramer-Rao lower bound, minimum variance, unbiased estimation, confidence intervals, hypothesis testing, experimental design and analysis of variance, etc. are packed? I doubt it! And course in optimization -- unconstrained sufficient conditions, roles for convexity, Kuhn-Tucker conditions, the simplex algorithm, Lagrange multipliers and Lagrangian relaxation, nonlinear duality, the simplex algorithm and how to use it to solve nonlinear/integer problems, integer linear programming, etc. -- is also getting going, say, if only for maximum likelihood estimation in 'machine learning'? Why do I doubt it? Guys, yes, for much of anything practical in statistics or optimization will require computing and software to do the data manipulations. Alas, that does not mean that .NET or computer science is what is central to the work. I may be that all that is needed from computing is SAS, R, some open source statistics/optimization code, etc. .NET has a big role in computing, but it's not intended for statistics/optimization. Who the heck put the $10 into the hype machine?

stirno 13y ago

Your statement is an incredible generalization of a large group of developers. It may be your experience, but I have to counter it with my own. While the Windows/.NET stacks may not provide as many pre-built solutions to handle 'big data' (a term I'm growing to hate), I have worked with tons of competent developers who have been able to build solutions to answer exactly the kinds of questions you propose. I've seen it done with SQL Server Reporting Services, Analysis Services and even straight custom C# code. I've seen it done with Hadoop + streaming API, StreamInsight (CEP rather than post processing) and custom Workflows in WF. As for your other statements that are just built to jump on the 'hate Microsoft' bandwagon, just because a single large company doesn't iterate fast enough to keep up with the rest of the OSS landscape doesn't mean that .NET didn't have options available to it. Nancy provides great REST capabilities and has existed since 2010. Devs were building REST services with Microsoft's ASP.NET MVC during the same period. Others were using WCF REST (not great, but usable) at the same point as well. WCF Data Services was another option. ORMs.. Theres the ever-expanding Entity Framework (which I don't love), NHibernate and many many others. Lets not just jump in and bash MS because you've only worked with bad developers using their stuff.

graycat 13y ago

> what factors influence student performance, or to visualize perchasing trends, or to predict sales numbers so that the company doesn't over purchase perishables "Big data"? Those are all just everyday, common, ordinary, vanilla-pure, old-fashioned exercises in statistics with maybe a little optimization thrown in, that is, the 'mathematical sciences' or, if you will, with a lot of overlap of 'operations research'. But long the standard remarks were that "operations research is dead" and "statistics died long ago except for some of biomedical statistics", e.g., for doing FDA approved 'trials'. Generally the amount of relevant data available and needed for such work is not very 'big'; yes, maybe the relevant data comes from a database of 10 TB, but a simple SQL query should be able to extract into, say, just a simple 'flat' file, maybe comma separated values or name-type-value triples, what is relevant. From the OP > 4. We need 10x or 100x more data scientists and AI specialists Again not a prediction. We are already seeing this. So perhaps I can sharpen some of my old skills in computer vision. Ah, come ON! So, all of a sudden college courses in undergraduate, junior level in mathematical statistics -- weak/strong laws of large numbers, Lindeberg-Feller central limit theorem, Neyman-Pearson lemma, Cramer-Rao lower bound, minimum variance, unbiased estimation, confidence intervals, hypothesis testing, experimental design and analysis of variance, etc. are packed? I doubt it! And course in optimization -- unconstrained sufficient conditions, roles for convexity, Kuhn-Tucker conditions, the simplex algorithm, Lagrange multipliers and Lagrangian relaxation, nonlinear duality, the simplex algorithm and how to use it to solve nonlinear/integer problems, integer linear programming, etc. -- is also getting going, say, if only for maximum likelihood estimation in 'machine learning'? Why do I doubt it? Guys, yes, for much of anything practical in statistics or optimization will require computing and software to do the data manipulations. Alas, that does not mean that .NET or computer science is what is central to the work. I may be that all that is needed from computing is SAS, R, some open source statistics/optimization code, etc. .NET has a big role in computing, but it's not intended for statistics/optimization. Who the heck put the $10 into the hype machine?