4 ms·

Take it to the next level, integrate the chatbot into a browser extension side panel. Let people navigate to websites that contain the information. This will w

Take it to the next level, integrate the chatbot into a browser extension side panel. Let people navigate to websites that contain the information. This will work. It will allow the chatbot to provide up to the minute data and information from the source. It will allow the user to maintain context -- like a popup dialog allows the user to maintain visual context. And, it will incentivize content creators to curate and provide information and data as people will be visiting their websites. If anyone thinks this might be a good idea also, I've already laid down the foundation approaching a browser extension side panel as a framework like Electron or Playwright and did the grunt work. [0] I put the VSCode IPC and other core libraries into this project. The IPC is important because a browser extension with this use case requires looking at a browser as a distributed system of javascript processes that communicate a a dozen different ways > Environments: Node.js main process, Node.js child process, Node.js worker thread, browser main thread (window), iframe, dedicated Web Worker, Shared Worker, Service Worker, AudioWorklet. > Communication: fetch/XMLHttpRequest, WebSocket, RTCDataChannel, EventSource, BroadcastChannel, SharedArrayBuffer + Atomics, localStorage storage events, MessageChannel/MessagePort, postMessage/onmessage, Worker.postMessage/worker.onmessage, parentPort.postMessage/parentPort.on('message'), ChildProcess.send/process.on('message'), stdin/stdout streams. and VSCode provides a protocol interface with only `onMessage` and `send` so I can define my own that are not provided creating a consistent API for communication. Regardless, I have it working but it needs to be completely rewritten. [0]https://github.com/adam-s/doomberg-terminal https://github.com/adam-s/doomberg-terminal

_thisdot 1y ago
This is already a thing with Gemini in Chrome[0]. The Browser Company’s new browser, Dia[1], is supposedly another similar product [0] = https://gemini.google/overview/gemini-in-chrome/?hl=en https://gemini.google/overview/gemini-in-chrome/?hl=en [1] = https://www.diabrowser.com/ https://www.diabrowser.com/
- dataviz1000 1y ago
  I wasn't aware of dia browser, thank you for sharing. I've been doing browser automation for a while and have become convinced the way to accomplish human in the middle is to fork Chromium and create a custom browser. However, there is one problem, there are 3.5B Chrome users. Getting someone to install a browser extension is hard enough, a whole new browser much, much more difficult. I went with experimenting with stock trading as a demo because people need huge incentive and value to make the critical jump to install an app. There is potential for niche curated business intelligence in trading or real estate, for example, where not only providing the chat bot but also time series data embeddings ect. I'm a little sad because I'm late to this party. Nevertheless, there needs to incentive also for people to continue to publish the data, ideas, and information so the chatbots are going to have help the content creators help them curate and provide the data by getting users to navigate to the webpages. I remember reading Google's Search Engine Optimization guide back in 2009 when I built a news publishing website for an industry newpaper. The tone was here is how to optimize your website for google crawlers to help us help you get traffic to your website. Google is nothing without people creating.
eikenberry 1y ago
This and many other applications of this sort will depend on AIs becoming ubiquitous, cheap and not metered. Metered access (like most current SAS AIs) will deter these sorts of heavy use cases. Running locally will be best, both for pricing and so you can have it build up context over time.
- dataviz1000 1y ago
  As an experiment, because p47's social media posts move the markets $60 in a day, and the last thing on Earth I want to be doing is reading them so the system makes an API request for any new ones, then checks for links, video, and images. It uses OpenAI whisper running with transformers.js on the local machine using webgpu for the inference to transcribe the video and audio and image to text for ocr. I tried to do the text generation locally but any decent model although will run caused my Macbook M3 to get so hot I could cook a steak on it while freezing the rendering for the whole computer. image-to-text and video, audio-to-text works fine, there are lot of uses for text generation that work but to get high quality analysis to see if a social media post might cause the stock market to crash requires sending the data out to an api. If the side panel requires searching for links to navigate to it requires a third party api. Working with it, I think the next hardware race will be getting these models to run on personal computers in next 2 - 5 years and I have a suspicion Microsoft is ahead of Apple.

_thisdot 1y ago

This is already a thing with Gemini in Chrome[0]. The Browser Company’s new browser, Dia[1], is supposedly another similar product [0] = https://gemini.google/overview/gemini-in-chrome/?hl=en https://gemini.google/overview/gemini-in-chrome/?hl=en [1] = https://www.diabrowser.com/ https://www.diabrowser.com/

dataviz1000 1y ago
I wasn't aware of dia browser, thank you for sharing. I've been doing browser automation for a while and have become convinced the way to accomplish human in the middle is to fork Chromium and create a custom browser. However, there is one problem, there are 3.5B Chrome users. Getting someone to install a browser extension is hard enough, a whole new browser much, much more difficult. I went with experimenting with stock trading as a demo because people need huge incentive and value to make the critical jump to install an app. There is potential for niche curated business intelligence in trading or real estate, for example, where not only providing the chat bot but also time series data embeddings ect. I'm a little sad because I'm late to this party. Nevertheless, there needs to incentive also for people to continue to publish the data, ideas, and information so the chatbots are going to have help the content creators help them curate and provide the data by getting users to navigate to the webpages. I remember reading Google's Search Engine Optimization guide back in 2009 when I built a news publishing website for an industry newpaper. The tone was here is how to optimize your website for google crawlers to help us help you get traffic to your website. Google is nothing without people creating.

eikenberry 1y ago

This and many other applications of this sort will depend on AIs becoming ubiquitous, cheap and not metered. Metered access (like most current SAS AIs) will deter these sorts of heavy use cases. Running locally will be best, both for pricing and so you can have it build up context over time.

dataviz1000 1y ago
As an experiment, because p47's social media posts move the markets $60 in a day, and the last thing on Earth I want to be doing is reading them so the system makes an API request for any new ones, then checks for links, video, and images. It uses OpenAI whisper running with transformers.js on the local machine using webgpu for the inference to transcribe the video and audio and image to text for ocr. I tried to do the text generation locally but any decent model although will run caused my Macbook M3 to get so hot I could cook a steak on it while freezing the rendering for the whole computer. image-to-text and video, audio-to-text works fine, there are lot of uses for text generation that work but to get high quality analysis to see if a social media post might cause the stock market to crash requires sending the data out to an api. If the side panel requires searching for links to navigate to it requires a third party api. Working with it, I think the next hardware race will be getting these models to run on personal computers in next 2 - 5 years and I have a suspicion Microsoft is ahead of Apple.