4 ms·
Talaria: Interactively Optimizing Machine Learning Models for Efficient Inferenc
- jgoertler 2y agoHi, I’m Jochen, one of the authors. We recently did a Show HN (https://news.ycombinator.com/item?id=41463916 https://news.ycombinator.com/item?id=41463916) which did not get much traction, so I’m posting this again here: We just released Mycelium, the library that powers Talaria’s graph viewer. You can check it out and play around with it here: https://apple.github.io/ml-mycelium https://apple.github.io/ml-mycelium I’m happy to answer any questions about Talaria or Mycelium!
- efnx 2y agoHow does this compare to TVM?
- jgoertler 2y agoDisclaimer upfront: I have no direct experience with TVM. I would imagine that the model compilation works quite similar, but I'm not sure if TVM supports palletization. What I believe is unique to Talaria, is that it can make recommendations for optimizations to the user for each of the layer in the network. The system allows the user to quickly identify "problematic" layers either through the table view or the graph viewer. This works based on simulated metrics (energy consumption, latency, ...) that are collected for each layers. It then gives optimization choices for each layer, together with the implied changes to the overall (total) metrics. I'm not sure if TVM collects / exposes similar metrics. So a large part of the system focus on the user-in-the-loop aspect of optimizing a network for inference, which is also why this paper was presented at a conference on human-computer interaction (SIGCHI).
- efnx 2y agoOk, thanks :)
- bobosha 2y agoCould you give us a tl;dr on this project? and how could I use something like this work for on-device applications, think "smart home" style applications?
- SaBaAg 2y agoAre inference metrics like latency and power measured live from device? To which devices can Talaria be applied?