3 ms·

From the article: > Using Sony’s suite of AI tools, existing neural network models using frameworks such as TensorFlow or PyTorch can be converted to run effic

From the article: > Using Sony’s suite of AI tools, existing neural network models using frameworks such as TensorFlow or PyTorch can be converted to run efficiently on the AI Camera. Alternatively, new models can be designed to take advantage of the AI accelerator’s specific capabilities. From Sony's website [1] > Deploy your AI model to IMX500 in three steps. > 1. Convert your own quantized and trained deep neural network model into an optimized binary file. > 2. Package your AI model as a cryptographically signed package. > 3. Deploy your packaged model securely to IMX500. and [2] > Memory size 8388480 bytes for firmware, network weight file, and working memory 8.39mb sounds a lot smaller than 8388480 bytes. It is not clear how much is available for the model and how much memory and firmware. Does anyone know how well these really quantized image models perform? I have never tried or needed to reduce a model to <8.4 mb. [1] https://developer.sony.com/imx500 https://developer.sony.com/imx500 [2] https://developer.sony.com/imx500/imx500-key-specifications https://developer.sony.com/imx500/imx500-key-specifications

CalRobert 2y ago
"8.39mb sounds a lot smaller than 8388480 bytes. " I don't follow? (Assuming you meant MB and not millibits - if the latter I definitely agree!)
fidotron 2y ago
> Does anyone know how well these really quantized image models perform? IME it's better if they are quantized during training/refinement as it can definitely affect model performance if not. > I have never tried or needed to reduce a model to <8.4 mb. Same. The limit has been inference speed before model size. It's quite unusual to run video inference at full resolution due to this (having spent too much of my life up-scaling segmentation masks) but maybe their accelerator is actually capable of that speed? They don't seem to say what resolution of image their model is optimized for. EDIT: The Sony specs say input tensor size is 640x480.
- hcfman 2y ago
  Really terribly. To put it into context, from memory, yolov3 tiny model was around 6 million parameters. How many parameters will this model be using? Indeed, no one is saying what it is quantized to, but you can be pretty sure that it's 8 bits. I deployed yolov3 tiny model back in 2019 with 32 bit weights. It thought that the drain pipe in the driveway was a person. Whenever people show hype about this this show pre-selected images where there are no candidates for false positives. In reality you would not want to be woken in the middle of the night on the output of these things.

CalRobert 2y ago

"8.39mb sounds a lot smaller than 8388480 bytes. " I don't follow? (Assuming you meant MB and not millibits - if the latter I definitely agree!)

fidotron 2y ago

> Does anyone know how well these really quantized image models perform? IME it's better if they are quantized during training/refinement as it can definitely affect model performance if not. > I have never tried or needed to reduce a model to <8.4 mb. Same. The limit has been inference speed before model size. It's quite unusual to run video inference at full resolution due to this (having spent too much of my life up-scaling segmentation masks) but maybe their accelerator is actually capable of that speed? They don't seem to say what resolution of image their model is optimized for. EDIT: The Sony specs say input tensor size is 640x480.

hcfman 2y ago
Really terribly. To put it into context, from memory, yolov3 tiny model was around 6 million parameters. How many parameters will this model be using? Indeed, no one is saying what it is quantized to, but you can be pretty sure that it's 8 bits. I deployed yolov3 tiny model back in 2019 with 32 bit weights. It thought that the drain pipe in the driveway was a person. Whenever people show hype about this this show pre-selected images where there are no candidates for false positives. In reality you would not want to be woken in the middle of the night on the output of these things.