3 ms·

They both let you build and train a model of nn layers. Tensorflow via keras and pytorch via torch.nn.Module. How is that not functionally the same?

refulgentis 3y ago
You won't learn anything, and attemping to derive any info is likely to be extremely unfair to Vulkan, which was the stated purpose of said comparison. PyTorch and TensorFlow have been developed and fine-tuned over many years by large teams of expert engineers. They incorporate numerous performance optimizations, including advanced memory management, kernel fusion, and auto-tuning of hyperparameters. Richard, as a single-person educational project, simply cannot match this level of optimization. The scope and feature set of the projects are vastly different. PyTorch and TensorFlow are comprehensive ecosystems that include not just the core computational graphs, but also high-level APIs, pre-trained models, data loading and augmentation pipelines, distributed training support, and more. PyTorch and TensorFlow are often evaluated on large, complex models with millions of parameters, trained on massive datasets. Richard's examples, as shown in the README, use much smaller models and datasets. I wish I could think of a good analogy off the top of my head...I've only gotten as far as testing the viability of carbon fiber tires for a highway-bound semi via someone's hobby pinewood derby car where they lovingly crafted novel carbon fiber tires from scratch. No one did anything wrong and both are achievements. It's just a category error.
- nurettin 3y ago
  I do implementations from scratch/paper myself. I know the value of this. I do my own personal research using graph neural networks recreationally. I was replying to someone who thought the two ubiquitous libraries people use for almost identical functionality were somehow different.
  - refulgentis 3y ago
    Not sure what you mean, maybe they weren't either.

refulgentis 3y ago

You won't learn anything, and attemping to derive any info is likely to be extremely unfair to Vulkan, which was the stated purpose of said comparison. PyTorch and TensorFlow have been developed and fine-tuned over many years by large teams of expert engineers. They incorporate numerous performance optimizations, including advanced memory management, kernel fusion, and auto-tuning of hyperparameters. Richard, as a single-person educational project, simply cannot match this level of optimization. The scope and feature set of the projects are vastly different. PyTorch and TensorFlow are comprehensive ecosystems that include not just the core computational graphs, but also high-level APIs, pre-trained models, data loading and augmentation pipelines, distributed training support, and more. PyTorch and TensorFlow are often evaluated on large, complex models with millions of parameters, trained on massive datasets. Richard's examples, as shown in the README, use much smaller models and datasets. I wish I could think of a good analogy off the top of my head...I've only gotten as far as testing the viability of carbon fiber tires for a highway-bound semi via someone's hobby pinewood derby car where they lovingly crafted novel carbon fiber tires from scratch. No one did anything wrong and both are achievements. It's just a category error.

nurettin 3y ago
I do implementations from scratch/paper myself. I know the value of this. I do my own personal research using graph neural networks recreationally. I was replying to someone who thought the two ubiquitous libraries people use for almost identical functionality were somehow different.
- refulgentis 3y ago
  Not sure what you mean, maybe they weren't either.