5 ms·

It's inevitable you will have multiple languages / platforms comprising your stack. Bash scripts for admin. Golang to write servers. Data science in Python. JS

by indescions_2018 8y ago

It's inevitable you will have multiple languages / platforms comprising your stack. Bash scripts for admin. Golang to write servers. Data science in Python. JS / D3 on the frontend. C# / Unity 3D modelling and scripting. Even low level C / Assembly if you need performance optimizations. I think mastery of cloud development and deployment is crucial going forward. Start experimenting locally right on your laptop with minikube, microk8s, snaps, prometheus. And get a sense of designing scalable systems. You should have the ability to immediately estimate project costs. So you can give potential stakeholders a ballpark figure to open negotiations. Billion item recommendation system running on spark that serves 100K+ requests per second deployed to 1000 node cluster. Will it cost under $100 per hour? Or do we jump to tensorflow and a few large gpu instances for half that. Once you gain sufficient mastery. It will become second nature. And you can concentrate on design and building. Best of luck :)

lmm 8y ago
> It's inevitable you will have multiple languages / platforms comprising your stack. Bash scripts for admin. Golang to write servers. Data science in Python. JS / D3 on the frontend. C# / Unity 3D modelling and scripting. Even low level C / Assembly if you need performance optimizations. It's not at all inevitable. I would - indeed do - use Scala for all of the use cases you listed.
- passivepinetree 8y ago
  Scalia for frontend work? How do you do it?
  - lmm 8y ago
    https://www.scala-js.org/ https://www.scala-js.org/
    - 11235813213455 8y ago
      well, for anything complex, it'll be difficult with that approach (lack of ecosystem, libraries, ..) much more confident with a full JS approach
- Brockenstein 8y ago
  Hey, someone found a golden hammer! Better throw out all those other tools, who needs 'em.
  - sghiassy 8y ago
    Let’s rewrite the Linux kernel in Scala!!
    - preordained 8y ago
      I know we're just having fun here...but if we really could standardize on one language (or mostly one language) the development benefits would be insane.
  - lmm 8y ago
    Programming is not carpentry. The best programming languages (IMO) are general-purpose, and the gains from being able to know a single language in depth and reuse the same tools and techniques across different domains outweigh those from using different languages in different domains.
    - sghiassy 8y ago
      I agree with Imm Check out this boat car btw! (http://yachtpals.com/boat-car-9158 http://yachtpals.com/boat-car-9158) It’s a boat. And it’s a car. It’s amazing. People are so stupid that they’re still buying cars that can’t go in water. When will they learn?
      - cpursley 8y ago
        I first read that as "bloat" car ;)
    - AnimalMuppet 8y ago
      Depends. Yes, learn a general-purpose language, and learn it well. But also, learn some other things well enough to be able to use them. Use them when they make the job easier. For example, my main language is C++, and my second language is Java. But when the problem I have is picking snippets of text out of a text file, I reach for Perl, because Perl is far better at that than either C++ or Java. I don't want to "reuse the same [C++/Java] tools and techniques" in that domain - it's considerably more painful than using Perl to do it.
- sghiassy 8y ago
  Web browsers only run JavaScript, so in fact if you wanted to write front end code, then you would indeed need to know JS. Even if there’s a Scala transpiler or WASM target - if you think you’re going to ship to production and blissfully never need to know JS...
  - lmm 8y ago
    I "know" javascript in the same way that I "know" SQL, or JVM assembly. It's part of the stack, you do want to be able to read and understand it, but you don't use it for programming (that is, expressing business logic) in.
    - sghiassy 8y ago
      Which proves the commentor’s first point: “It's inevitable you will have multiple languages / platforms comprising your stack.”
      - lmm 8y ago
        Different senses of "stack". It's inevitable that there will be multiple technologies involved in the running of your code, which will involve multiple programming languages, and you will need to understand the behaviour of those technologies (though less and less as they mature - I wouldn't be surprised if it were possible to debug a compile-to-js language without understanding JS these days). But that's not the same thing as being able to program in those languages.
  - mi100hael 8y ago
    https://www.scala-js.org/ https://www.scala-js.org/
- hobls 8y ago
  People who want jobs don’t usually have the luxury of picking them by the languages used.
  - lmm 8y ago
    People usually get hired to use a particular primary language (though even then, I've worked in jobs where I was hired for Java that ended up switching to Scala mainly due to developer preference), but often have broad discretion over which languages to use for secondary tasks. Certainly in a lot of "Java shops" the pattern I've seen is that devs will not be second-guessed (or at least, will contribute to team consensus) in their choice of bash/python/ruby/scala/... for things like testing scripts or data migration tools, nor in their choice of coffeescript/typescript/... for frontend parts.
- projectramo 8y ago
  That is an interesting response. I wonder if it were equally possible with any language. Could a Python or Java programmer say the same? I think there are probably a handful of languages that you could use for "everything". The limiting constraint is the presence of a reliable javascript compiler, followed by data science libraries (I think).
  - lmm 8y ago
    I think most languages could get close - see e.g. https://www.teamten.com/lawrence/writings/java-for-everything.html https://www.teamten.com/lawrence/writings/java-for-everythin... . Of course if you want to actually execute code in the browser then you need to compile to javascript, but I'm not convinced that's required as often as people think it is (Wicket offers a very interesting AJAX model where you re-render UI fragments on the server in response to changes - all the code that runs is Java, but the behaviour ends up very similar to if you had client-side javascript making a server call to fetch data). "Data science" means many things to many people; you wouldn't want to write actual linear algebra in Java (or any language without operator overloading), but again the cases where you can't reuse a prebuilt model from your language of choice are rarer than one might think. Scala was explicitly designed as a "scalable language" with both the strong safety guarantees that you need for large-scale development and the lightweight syntax that you need for throwaway scripts. But a lot of modern language design is converging on the same set of features (e.g. static typing for safety but type inference for convenience, pattern matching for the safety of the visitor pattern but the convenience of an if/else) and would have the same abilities. E.g. in a Reddit post about Rust a couple of days ago people were talking about writing web services in Rust, and part of the rationale there is that while Rust may not be the perfect web service language, it's good enough that if you already have Rust in your stack you might well write web services in Rust rather than bringing in a new language. I haven't seen anyone doing data science in Rust yet, but I see no fundamental reason why the language couldn't be adapted for it.
    - _glass 8y ago
      Rust is better suited as universal language than Scala for me as I am writing also music applications, which really hate the garbage collector needed in Java/Scala, things that run on the JVM. I never really needed recursive types, but that's also something limiting on the JVM.
- 11235813213455 8y ago
  JS and bash here
- dangom 8y ago
  Data science?!
  - lmm 8y ago
    Oh yeah. Spark has lots of good aspects (though also some bad ones), and personally I find it much easier to think in terms of code with map/filter/aggregate/... than in terms of tables.
- ralphc 8y ago
  Python is the de facto language for the deep learning part of data science, the interface to TensorFlow, PyTorch and Keras. Others may be possible but all the examples, books, etc. use Python.

lmm 8y ago

> It's inevitable you will have multiple languages / platforms comprising your stack. Bash scripts for admin. Golang to write servers. Data science in Python. JS / D3 on the frontend. C# / Unity 3D modelling and scripting. Even low level C / Assembly if you need performance optimizations. It's not at all inevitable. I would - indeed do - use Scala for all of the use cases you listed.

passivepinetree 8y ago
Scalia for frontend work? How do you do it?
- lmm 8y ago
  https://www.scala-js.org/ https://www.scala-js.org/
  - 11235813213455 8y ago
    well, for anything complex, it'll be difficult with that approach (lack of ecosystem, libraries, ..) much more confident with a full JS approach
Brockenstein 8y ago
Hey, someone found a golden hammer! Better throw out all those other tools, who needs 'em.
- sghiassy 8y ago
  Let’s rewrite the Linux kernel in Scala!!
  - preordained 8y ago
    I know we're just having fun here...but if we really could standardize on one language (or mostly one language) the development benefits would be insane.
- lmm 8y ago
  Programming is not carpentry. The best programming languages (IMO) are general-purpose, and the gains from being able to know a single language in depth and reuse the same tools and techniques across different domains outweigh those from using different languages in different domains.
  - sghiassy 8y ago
    I agree with Imm Check out this boat car btw! (http://yachtpals.com/boat-car-9158 http://yachtpals.com/boat-car-9158) It’s a boat. And it’s a car. It’s amazing. People are so stupid that they’re still buying cars that can’t go in water. When will they learn?
    - cpursley 8y ago
      I first read that as "bloat" car ;)
  - AnimalMuppet 8y ago
    Depends. Yes, learn a general-purpose language, and learn it well. But also, learn some other things well enough to be able to use them. Use them when they make the job easier. For example, my main language is C++, and my second language is Java. But when the problem I have is picking snippets of text out of a text file, I reach for Perl, because Perl is far better at that than either C++ or Java. I don't want to "reuse the same [C++/Java] tools and techniques" in that domain - it's considerably more painful than using Perl to do it.
sghiassy 8y ago
Web browsers only run JavaScript, so in fact if you wanted to write front end code, then you would indeed need to know JS. Even if there’s a Scala transpiler or WASM target - if you think you’re going to ship to production and blissfully never need to know JS...
- lmm 8y ago
  I "know" javascript in the same way that I "know" SQL, or JVM assembly. It's part of the stack, you do want to be able to read and understand it, but you don't use it for programming (that is, expressing business logic) in.
  - sghiassy 8y ago
    Which proves the commentor’s first point: “It's inevitable you will have multiple languages / platforms comprising your stack.”
    - lmm 8y ago
      Different senses of "stack". It's inevitable that there will be multiple technologies involved in the running of your code, which will involve multiple programming languages, and you will need to understand the behaviour of those technologies (though less and less as they mature - I wouldn't be surprised if it were possible to debug a compile-to-js language without understanding JS these days). But that's not the same thing as being able to program in those languages.
- mi100hael 8y ago
  https://www.scala-js.org/ https://www.scala-js.org/
hobls 8y ago
People who want jobs don’t usually have the luxury of picking them by the languages used.
- lmm 8y ago
  People usually get hired to use a particular primary language (though even then, I've worked in jobs where I was hired for Java that ended up switching to Scala mainly due to developer preference), but often have broad discretion over which languages to use for secondary tasks. Certainly in a lot of "Java shops" the pattern I've seen is that devs will not be second-guessed (or at least, will contribute to team consensus) in their choice of bash/python/ruby/scala/... for things like testing scripts or data migration tools, nor in their choice of coffeescript/typescript/... for frontend parts.
projectramo 8y ago
That is an interesting response. I wonder if it were equally possible with any language. Could a Python or Java programmer say the same? I think there are probably a handful of languages that you could use for "everything". The limiting constraint is the presence of a reliable javascript compiler, followed by data science libraries (I think).
- lmm 8y ago
  I think most languages could get close - see e.g. https://www.teamten.com/lawrence/writings/java-for-everything.html https://www.teamten.com/lawrence/writings/java-for-everythin... . Of course if you want to actually execute code in the browser then you need to compile to javascript, but I'm not convinced that's required as often as people think it is (Wicket offers a very interesting AJAX model where you re-render UI fragments on the server in response to changes - all the code that runs is Java, but the behaviour ends up very similar to if you had client-side javascript making a server call to fetch data). "Data science" means many things to many people; you wouldn't want to write actual linear algebra in Java (or any language without operator overloading), but again the cases where you can't reuse a prebuilt model from your language of choice are rarer than one might think. Scala was explicitly designed as a "scalable language" with both the strong safety guarantees that you need for large-scale development and the lightweight syntax that you need for throwaway scripts. But a lot of modern language design is converging on the same set of features (e.g. static typing for safety but type inference for convenience, pattern matching for the safety of the visitor pattern but the convenience of an if/else) and would have the same abilities. E.g. in a Reddit post about Rust a couple of days ago people were talking about writing web services in Rust, and part of the rationale there is that while Rust may not be the perfect web service language, it's good enough that if you already have Rust in your stack you might well write web services in Rust rather than bringing in a new language. I haven't seen anyone doing data science in Rust yet, but I see no fundamental reason why the language couldn't be adapted for it.
  - _glass 8y ago
    Rust is better suited as universal language than Scala for me as I am writing also music applications, which really hate the garbage collector needed in Java/Scala, things that run on the JVM. I never really needed recursive types, but that's also something limiting on the JVM.
11235813213455 8y ago
JS and bash here
dangom 8y ago
Data science?!
- lmm 8y ago
  Oh yeah. Spark has lots of good aspects (though also some bad ones), and personally I find it much easier to think in terms of code with map/filter/aggregate/... than in terms of tables.
ralphc 8y ago
Python is the de facto language for the deep learning part of data science, the interface to TensorFlow, PyTorch and Keras. Others may be possible but all the examples, books, etc. use Python.