5 ms·
Go, Python, Rust, and production AI applications
- niux 3y agoI have spent the majority of my life programming in Python, starting from the very first program I wrote. However, recently I've begun learning Go, and in many respects, it feels more Zen-like compared to Python. In Python, it seems there are a million ways to accomplish a single task, whereas Go often presents a singular, correct approach. As projects grow in size, the expressiveness of Python and the weak type safety eventually become a liability.
- givemeethekeys 3y agoWhen I first programmed in Python over 15 years ago, Python very much made me feel like there was one clear way to do a particular thing. Maybe as Go gets older, it too will have multiple ways to do the same thing.
- neomantra 3y agoI've had these strong feelings and the OP describes it really well. Despite being a polyglot programmer, I really struggle with Python, both in expression and performance (unless it's just config for GPUs). Some of this frustration was recently an "Unpopular Opinion" on the Go Time Podcast regarding Python being great for "data exploration" but not for "data engineering": https://changelog.com/gotime/304#t=3196 https://changelog.com/gotime/304#t=3196 I've been yearning for better interactive tooling and ML-related libraries to bridge this gap and started using some even in just the last week: * GoNB (Golang-support for Jupyter notebooks, also from a Googler) https://github.com/janpfeifer/gonb https://github.com/janpfeifer/gonb * That uses Go-Plotly for graphs/UI: https://github.com/MetalBlueberry/go-plotly https://github.com/MetalBlueberry/go-plotly * GoMLX (GoNB author is also on that project, many thanks Jan!) https://github.com/gomlx/gomlx https://github.com/gomlx/gomlx * Hidden at the end of OP is LangChainGo for LLMs, which I haven't used yet: https://github.com/tmc/langchaingo https://github.com/tmc/langchaingo Pick those up and let's make the Go community stronger together!
- mountainriver 3y agoGo will sadly never be viable for ML tasks due to its C interop. Maybe calling an LLM api but often you have a mix of models. Rust also struggles here a bit due to its poor C++ interop. Written but someone who has built ML libraries from scratch in Go
- Art9681 3y agoCan you elaborate on this or point me to some more information? I am primarily a Go dev that has been integrating AI/ML into a personal project. Like many other apps, I wrap llama.cpp and various other popular ML projects. Recently I have been getting the itch to go lower level and see if it would be feasible to write an LLM backend in pure Go, or a Stable Diffusion backend. Just for curiosity and learning. But if this is a dead-end I would like to know from someone more experienced than I. Thanks for any advice you can offer in advance.
- nerdix 3y agoDo you know of any resources with details on using llama.cpp with Go?
- neomantra 3y agoI'm starting to explore this for Time-Series analysis -- still at the beginning as I mostly used C++ before. Torch.ch (so old now!) used LuaJIT which has excellent FFI. What is the realm of the interop problem there? That one wants to assemble data in Go and then feed it to another API? Or even at a lower level, getting data to a GPU/TPU requires C interop so it's more about the memory sharing? Those libraries come from Google, so hopefully they can help battle these issues in the nether realms.
- LettuceSand12 3y agoWhy not elixir then? Good c interop, functional, and good performance.
- 3y ago
- d357r0y3r 3y agoA missing tech here feels like TypeScript. It feels like the perfect mix of expressiveness and type safety for AI/ML work. I wish there was more tooling out there for it. For me, Go feels too verbose, but I appreciate the static typing and performance. Python with its robust ecosystem and optional types feels okay, but the performance is really awful.
- whalesalad 3y agoWhat about Python performance is awful in AI? All of the hard stuff is delegated to a C library or the GPU.
- d357r0y3r 3y agoThere's tons of glue code that gets written in Python.
- tl 3y agoPython makes it hard to get this right. Simple idioms result in copies of large collections or other work shifting from C to Python. The state-of-the-art of Python profiling [1] has to target this problem to the exclusion of other concerns. [1]: https://www.thestrangeloop.com/2022/python-performance-matters.html https://www.thestrangeloop.com/2022/python-performance-matte...
- fridder 3y agoHonestly I think Elixir has made some great strides in the past few years to also be an interesting alternative. Bumblebee, Livebook, the NX project are also awesome pieces of tech
- J_Shelby_J 3y agoI switched from python to rust for my AI stuff. Honestly, I don't care about the things people say rust is used for. I like it because the package manager, testing, and typings being built into the ecosystem by default makes it so easy to build. VS Python where it all can be done, but you need to then maintain all of those separate tools. The overhead of writing Rust is less than the overhead of dealing with the Python ecosystem. And then you have all the benefits of Rust everyone mentions more often... one other thing no one mentions is the feedback loop between a strongly typed language and copilots ability to more accurately generate code. That being said, there is a real shortage of Rust software for Rust only projects. I ended up writing a wrapper for Llama.cpp and open ai API [0] because I needed it and couldn't find anything out there. Eventually, I do intend to implement Hugging Face's Candle library [1] (A rust version of Torch). There is something appealing about doing everything in a single lang especially as the monopoly of CUDA inevitably gets chipped away. [0] https://github.com/ShelbyJenkins/llm_client https://github.com/ShelbyJenkins/llm_client [1] https://github.com/huggingface/candle https://github.com/huggingface/candle
- drBonkers 3y ago> no one mentions is the feedback loop between a strongly typed language and copilots ability to more accurately generate code. Dang, I never thought of this.
- rhodysurf 3y agoI am the same, and for me the biggest reason to use rust for this stuff is simply how if i leave projects, I know that the type system is so strict that I know in a few months things should still work even as I just jump in and change stuff. With python projects they just feel like they rot. Also as copilot generates code for me, if it compiles in rust I know it will usually work well. With python the iteration loop is so much less trustworthy
- throwaway63467 3y agoI like Go and use it a lot but for AI, Rust seems to have better cards in conjunction with Python. Go’s runtime makes it very impractical to build bindings for Python (it’s e.g. not possible to load two Golang-based libraries simultaneously as the runtimes will overwrite each others memory, or at least that was the case a while ago), Rust is much better suited to that task. And I think by now Rust already has a head start for data science & AI tooling.
- CuriouslyC 3y agoI think there are 2 layers in an AI application that can be considered somewhat orthogonal: 1. Making your models network accessible in a configured way 2. RAG and any chaining code for model calls For #1, this is probably going to start with pytorch since that's typically what research code uses, and get hand optimized over time as your application matures. For #2, it seems like the most popular tools so far are in the Python space, of course that doesn't mean other languages don't have robust ecosystems for AI already and of course new things get developed all the time, but ecosystem size is important to consider.
- planetis 3y agoNim has also a powerful deep learning library called Arraymancer. It's selling point is that you don't have to rewrite your code from research to production. It's used in various machine learning projects, but one recent one that caught my eye was https://github.com/amkrajewski/nimCSO https://github.com/amkrajewski/nimCSO "Composition Space Optimization" https://github.com/mratsim/Arraymancer https://github.com/mratsim/Arraymancer