3 ms·
The python pytorch ecosystem is such a dependency hell that that you literally need a dependency manager manager to set up the container for making the entirely
by superkuh 3y ago
The python pytorch ecosystem is such a dependency hell that that you literally need a dependency manager manager to set up the container for making the entirely new and dedicated python install with a dependency manager just to run pytorch stuff. System python will never work. It is a huge hassle and simply infeasible on most not-cutting edge systems.
Unfortunately if you want to use the tokenizer python script included with llm.c you have to pull in and satisfy the dependencies for nearly the entire pytorch chain anyway. So the ease of use of just "make" for llm.c is lost.