3 ms·
I'm so glad the AI community is finally starting to ditch Python. It has held progress back for far too long.
by IshKebab 3y ago
I'm so glad the AI community is finally starting to ditch Python. It has held progress back for far too long.
- dbmikus 3y agoHow has Python held it back? Most of the heavy computation lifting is done by C extensions/bindings and the models are compiled to run on CUDA, etc. What am I missing?
- lhl 3y agoPresumably what you're missing is that it's that IshKebab probably doesn't work in AI/ML at all (no links in his profile, but you can judge his post history yourself). Anyone can have voice opinion, but that doesn't mean it's particularly well informed.
- IshKebab 3y agoI worked for an AI startup for 5 years until recently. Nice try though.
- IshKebab 3y agoSetting up and deploying models in production or on edge devices is much much more complex if you have to deal with Python and Conda and whatnot.
- dbmikus 3y agoYou can compile the models to something that runs on edge though, right? For example, Tensorflow is a C++ framework that has Python bindings and a Python library, but when the models are served they are running on C++. Maybe the act of compilation is an extra step, but I'd much rather have my development be in a high level language that is very suited to experimentation, probing, and testing, and then compile the final result down to something performant. EDIT: I don't know much about the IOT world, and Tensorflow is likely a bad example as it's not designed to run on edge. So, I could understand that things like llama.cpp, GGML and GGUF are making strides towards easier runtimes. But I still think for dev-time, Python makes sense!
- shepardrtc 3y agoCloudflare lets you just upload the model itself: https://blog.cloudflare.com/introducing-constellation/ https://blog.cloudflare.com/introducing-constellation/ No idea what they're using to run it though. But there's no way I'll stop using Python for working with ML code lol. It just makes life easy.
- IshKebab 3y ago> Tensorflow is a C++ framework that has Python bindings and a Python library, but when the models are served they are running on C++ Sure, and it's only a simple 20 step process that involves building Tensorflow from source. Yeay! https://medium.com/@hamedmp/exporting-trained-tensorflow-models-to-c-the-right-way-cf24b609d183 https://medium.com/@hamedmp/exporting-trained-tensorflow-mod... Let me see what the process for compiling a LLM written in Rust is.... https://github.com/rustformers/llm https://github.com/rustformers/llm cargo install llm-cli Oh look, it doesn't make me immediately want to give up.
- dbmikus 3y agollm-cli looks like it loads model files and it doesn't help with model development. It runs GGML model files. The models aren't written in Rust. Besides the point, GGUF is successor to GGML. There's a variety of ways to convert Pytorch, Keras, etc models to GGML or GGUF. I dunno, maybe we're talking about different things. I'm saying it's better to do model development in a high level language and then export the training or runtime to a lower level framework, multiple of which exist and have existed. It's becoming simpler to use low-level runtimes (llama.cpp vs Tensorflow). Is that the point you're making?
- lelag 3y agoThe AI community is nowhere close to ditching Python. Most model development and training still use python based toolchains (torch, tf...). The new trends is for popular and useful models to be ported to more efficient stack like C++/GGML for easier usage and inference speed on consumer hardware. Another popular optimisation is to port models to WASM + GPU because it makes them easy to support a variety of platforms (desktop, mobile...) with a single API and it can still offer great performance (see Google's mediapipe as an exemple of that).
- IshKebab 3y agoThat's why I said "starting to" not "close to".
- Havoc 3y agoI’d say discovery and innovation would be slower in a less relaxed language. And speed end up comparable thanks to the compiled parts of python
- fsloth 3y agoIn general if you don't know what you are doing, it's much faster to first figure out a good strategy for a solution in a language that does not suffer from all of the encumbrance C++ brings in. Python is really great for fast prototyping. It can be argued most AI products so far are result of fast prototyping. So not sure if there is anything wrong with that. As practical models emerge, at that point it indeed makes sense to port them to C++. But I would not in my wildest dreams suggest prototyping a data model in C++ unless absolutely necessary.
- jebarker 3y agoThis is exactly the wrong way around. We've seen the progress we've seen because of the adoption of Python. Even now there are relatively few people that can write code like this and have the ML and math experience to push forward the research.