3 ms·

Newbie question: When would you use dataframes (e.g Pandas, Polars...) and when would you use tensors (Pytorch, TF...)? Are the usecases completely distinct o

• codyvoda 3y ago
  fairly distinct, there is of course some overlap. you could technically do (mostly?) everything with tensors that you could with dataframes, but generally dataframes are for analyzing and transforming data for ETL/analytic workloads. tensors are how machine learning models understand data, i.e. before training a neural network (or LLM) at some point text is converted to numbers in tensors you still transform data in tensors, but generally that's one-hot encoding or transposing or other transformation done right before model training. before that, you might use a dataframe to cleanup strings, aggregate timeseries data, etc. hope that makes sense. so yes there's some overlap, but generally they're distinct toolsets that would be used together for an end-to-end ML project