4 ms·

Probably 1,000 for the system prompt, 400 for the audio speech-to-text, 8 for the query, 180 for the thinking, 12 for the tool call, 33 for the response with a

by a2128 1y ago

Probably 1,000 for the system prompt, 400 for the audio speech-to-text, 8 for the query, 180 for the thinking, 12 for the tool call, 33 for the response with a useless follow-up question

godelski 1y ago
All to achieve something that could be done with a Raspberry Pi. You could do all this locally too. https://gist.github.com/mgarratt/afb3b57a08e2eb2479eb6083a86d8a64 https://gist.github.com/mgarratt/afb3b57a08e2eb2479eb6083a86... https://www.xda-developers.com/ollama-ai-comparison-raspberry-pi-5/ https://www.xda-developers.com/ollama-ai-comparison-raspberr... https://www.xda-developers.com/raspberry-pi-voice-assistant-how/ https://www.xda-developers.com/raspberry-pi-voice-assistant-... https://www.youtube.com/watch?v=o1sN1lB76EA https://www.youtube.com/watch?v=o1sN1lB76EA
- Sean-Der 1y ago
  This project isn’t tightly coupled with anything. Any service that supports WebRTC should work! Also I was hoping to push people toward a RTOS. Better experience then a raspberry pi, I can cycle power and be back way quicker. Also cheaper/more power efficient. I also think I unfairly like ESP because it’s an excuse to write C :)
  - godelski 1y ago
    Home Assistant integrates with WebRTC btw[0]. Also, why make the ESP32 the the hotspot? Why not just connect to the same network? Then you're not really range limited. > I also think I unfairly like ESP because it’s an excuse to write C :) Is the comment about Home Assistant being python? Yeah, I can get that. Feels weird to be using slow scripting languages on lean hardware. Though of course you can write whatever routines in C and just patch it in to the interface. The ESPs are cheaper (here's the non-dev kit which has WiFi[1]), but way less powerful. I don't think you could get away with doing things on device. Though I wouldn't call that dev kit cheaper and that price point was context of my comment. FWIW, I don't think there's really anything wrong with the project other than just that it comes off as doing things that have already been done before but presenting as if something novel was done. I'm all for reinventing the wheel. For fun, education, or even to improve. Just if I'm being honest, it came off with some weird vibes because of that. I imagine that's how some people are responding as well. [0] https://www.home-assistant.io/integrations/homeassistant/ https://www.home-assistant.io/integrations/homeassistant/ [1] https://shop.m5stack.com/products/m5stamp-esp32s3-module https://shop.m5stack.com/products/m5stamp-esp32s3-module
    - regularfry 1y ago
      > Also, why make the ESP32 the the hotspot? Why not just connect to the same network? Then you're not really range limited. Because then they don't have to include the ability to configure wifi, which (while not that hard) is one more thing to do and for a hackathon that's not really contributing to the end goal.
      - Sean-Der 1y ago
        I couldn't get on WiFi at the office at all. Corporate WiFi had a bunch of hoops to jump through that made ESP32 hard. Once I got it working it felt really cool though. As a user I don't want to configure WiFi on the microcontroller at all. I would be really cool if I could walk up to a 'smart device' and set my phone next to it and do zero configuration.
        regularfry 1y ago
        Hah, that doesn't surprise me either. You're really hoping for guest WiFi in that situation.
        godelski 1y ago
        That makes a lot more sense now. Thanks
    - Sean-Der 1y ago
      I thought about your comment a lot. I worry that most people just say nice things (but think the opposite) so I appreciate you being direct. ----- I don't expect you to know anything about me. It made me feel like you have written me off/dismissed me when you mention HomeAssistant + WebRTC. HomeAssistant uses Go2RTC and the WebRTC library it uses is Pion[0]. I created that and maintain it. Getting WebRTC on tiny devices is something I have been working on for years and always doing it Open Source/making it best for users. ----- > comes off as doing things that have already been done before but presenting as if something novel was done. I don't think 'Hardware AI Assistant' is a novel idea. What I hoped was a novel idea was putting it in an easy to use package for others to build upon and use. WebRTC + Hardware is something I have been trying to make easier for a long time https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c https://github.com/awslabs/amazon-kinesis-video-streams-webr... [1] I wrote this code, but things still felt too difficult. When ESP32s got powerful enough to do WebRTC I wrote [2]. Reflect inherits from that. So I am proud of that. I have been on this journey to make RTC + cheap hardware possible so that others can build upon that. ----- Again I really appreciate your comment, and sorry to be so defensive. Someone I really respected (and I thought they respected me) said the same thing about my work not being novel. They said people have been building security cameras for years that use WebRTC, you are over inflating what you are doing. That has stuck with me. So part of me does fear that I am wasting my time trying to solve these problems. I don't think what I am doing is novel. I do think that I am solving it differently because I make it accessible/Open Source. Most people solving these problems/building it just keep their code at work and don't try to help others use it. If you are up for it shoot me an email sean@pion.ly and https://www.linkedin.com/in/sean-dubois/ https://www.linkedin.com/in/sean-dubois/ I would love to have a friend that calls me out/is honest whats good work and what is just BS :) ----- [0] https://github.com/pion/webrtc https://github.com/pion/webrtc [1] https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c https://github.com/awslabs/amazon-kinesis-video-streams-webr... [2] https://github.com/Sean-Der/embedded-sdk https://github.com/Sean-Der/embedded-sdk

by a2128 1y ago

Probably 1,000 for the system prompt, 400 for the audio speech-to-text, 8 for the query, 180 for the thinking, 12 for the tool call, 33 for the response with a useless follow-up question

godelski 1y ago

All to achieve something that could be done with a Raspberry Pi. You could do all this locally too. https://gist.github.com/mgarratt/afb3b57a08e2eb2479eb6083a86d8a64 https://gist.github.com/mgarratt/afb3b57a08e2eb2479eb6083a86... https://www.xda-developers.com/ollama-ai-comparison-raspberry-pi-5/ https://www.xda-developers.com/ollama-ai-comparison-raspberr... https://www.xda-developers.com/raspberry-pi-voice-assistant-how/ https://www.xda-developers.com/raspberry-pi-voice-assistant-... https://www.youtube.com/watch?v=o1sN1lB76EA https://www.youtube.com/watch?v=o1sN1lB76EA

Sean-Der 1y ago
This project isn’t tightly coupled with anything. Any service that supports WebRTC should work! Also I was hoping to push people toward a RTOS. Better experience then a raspberry pi, I can cycle power and be back way quicker. Also cheaper/more power efficient. I also think I unfairly like ESP because it’s an excuse to write C :)
- godelski 1y ago
  Home Assistant integrates with WebRTC btw[0]. Also, why make the ESP32 the the hotspot? Why not just connect to the same network? Then you're not really range limited. > I also think I unfairly like ESP because it’s an excuse to write C :) Is the comment about Home Assistant being python? Yeah, I can get that. Feels weird to be using slow scripting languages on lean hardware. Though of course you can write whatever routines in C and just patch it in to the interface. The ESPs are cheaper (here's the non-dev kit which has WiFi[1]), but way less powerful. I don't think you could get away with doing things on device. Though I wouldn't call that dev kit cheaper and that price point was context of my comment. FWIW, I don't think there's really anything wrong with the project other than just that it comes off as doing things that have already been done before but presenting as if something novel was done. I'm all for reinventing the wheel. For fun, education, or even to improve. Just if I'm being honest, it came off with some weird vibes because of that. I imagine that's how some people are responding as well. [0] https://www.home-assistant.io/integrations/homeassistant/ https://www.home-assistant.io/integrations/homeassistant/ [1] https://shop.m5stack.com/products/m5stamp-esp32s3-module https://shop.m5stack.com/products/m5stamp-esp32s3-module
  - regularfry 1y ago
    > Also, why make the ESP32 the the hotspot? Why not just connect to the same network? Then you're not really range limited. Because then they don't have to include the ability to configure wifi, which (while not that hard) is one more thing to do and for a hackathon that's not really contributing to the end goal.
    - Sean-Der 1y ago
      I couldn't get on WiFi at the office at all. Corporate WiFi had a bunch of hoops to jump through that made ESP32 hard. Once I got it working it felt really cool though. As a user I don't want to configure WiFi on the microcontroller at all. I would be really cool if I could walk up to a 'smart device' and set my phone next to it and do zero configuration.
      - regularfry 1y ago
        Hah, that doesn't surprise me either. You're really hoping for guest WiFi in that situation.
      - godelski 1y ago
        That makes a lot more sense now. Thanks
  - Sean-Der 1y ago
    I thought about your comment a lot. I worry that most people just say nice things (but think the opposite) so I appreciate you being direct. ----- I don't expect you to know anything about me. It made me feel like you have written me off/dismissed me when you mention HomeAssistant + WebRTC. HomeAssistant uses Go2RTC and the WebRTC library it uses is Pion[0]. I created that and maintain it. Getting WebRTC on tiny devices is something I have been working on for years and always doing it Open Source/making it best for users. ----- > comes off as doing things that have already been done before but presenting as if something novel was done. I don't think 'Hardware AI Assistant' is a novel idea. What I hoped was a novel idea was putting it in an easy to use package for others to build upon and use. WebRTC + Hardware is something I have been trying to make easier for a long time https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c https://github.com/awslabs/amazon-kinesis-video-streams-webr... [1] I wrote this code, but things still felt too difficult. When ESP32s got powerful enough to do WebRTC I wrote [2]. Reflect inherits from that. So I am proud of that. I have been on this journey to make RTC + cheap hardware possible so that others can build upon that. ----- Again I really appreciate your comment, and sorry to be so defensive. Someone I really respected (and I thought they respected me) said the same thing about my work not being novel. They said people have been building security cameras for years that use WebRTC, you are over inflating what you are doing. That has stuck with me. So part of me does fear that I am wasting my time trying to solve these problems. I don't think what I am doing is novel. I do think that I am solving it differently because I make it accessible/Open Source. Most people solving these problems/building it just keep their code at work and don't try to help others use it. If you are up for it shoot me an email sean@pion.ly and https://www.linkedin.com/in/sean-dubois/ https://www.linkedin.com/in/sean-dubois/ I would love to have a friend that calls me out/is honest whats good work and what is just BS :) ----- [0] https://github.com/pion/webrtc https://github.com/pion/webrtc [1] https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c https://github.com/awslabs/amazon-kinesis-video-streams-webr... [2] https://github.com/Sean-Der/embedded-sdk https://github.com/Sean-Der/embedded-sdk