3 ms·

Using Claude to parse screen-scrapes of a server's boot status is certainly novel. I did not expect a mention of AI usage in an article like this. With that sa

Using Claude to parse screen-scrapes of a server's boot status is certainly novel. I did not expect a mention of AI usage in an article like this. With that said, I wonder why they used AI at all here. Could they not have keyed off certain keywords or other information present in a screen scrape, rather than rely on Claude to parse it?

ordersofmag 2y ago
The article says "...we can obtain a near real-time image of the server screen". Implying that what they have is an image file, not text. Getting keywords out of an image file would normally take an OCR step. A single API call to Claude does the trick without the extra tooling.
- chatmasta 2y ago
  I asked ChatGPT to make text from a photo of a whiteboard and it just executed some Pytesseract code… I’m reminded of the guy who setup an iPhone farm to use the iOS on-device OCR because he couldn’t find anything better.
Nextgrid 2y ago
I wonder how much of this is just to get a talking point about AI in the article (and I guess it works - we're talking about it). If you literally need to just detect whether it's at the firmware splash screen or not, simply checking if enough pixels on the image are white would detect that splash screen just fine.
- ca508 2y ago
  author here; Claude use here was pure laziness - and personally, I found it quite funny that it worked. We could sample pixels and try and build that detection, but $<1c per run to write a prompt and get some json was too hilarious not to ship to prod. Maybe it needs to have more complex logic/detection and we need something more complex down the road. But it's like easy and cheap OCR for now. what was kinda funnier was that I tried to get Claude to generate its own Go client code to upload the image and run the prompt; it totally totally hallucinated on that part :).

3 ms·

Using Claude to parse screen-scrapes of a server's boot status is certainly novel. I did not expect a mention of AI usage in an article like this. With that sa

by bovermyer 2y ago

ordersofmag 2y ago
The article says "...we can obtain a near real-time image of the server screen". Implying that what they have is an image file, not text. Getting keywords out of an image file would normally take an OCR step. A single API call to Claude does the trick without the extra tooling.
- chatmasta 2y ago
  I asked ChatGPT to make text from a photo of a whiteboard and it just executed some Pytesseract code… I’m reminded of the guy who setup an iPhone farm to use the iOS on-device OCR because he couldn’t find anything better.
Nextgrid 2y ago
I wonder how much of this is just to get a talking point about AI in the article (and I guess it works - we're talking about it). If you literally need to just detect whether it's at the firmware splash screen or not, simply checking if enough pixels on the image are white would detect that splash screen just fine.
- ca508 2y ago
  author here; Claude use here was pure laziness - and personally, I found it quite funny that it worked. We could sample pixels and try and build that detection, but $<1c per run to write a prompt and get some json was too hilarious not to ship to prod. Maybe it needs to have more complex logic/detection and we need something more complex down the road. But it's like easy and cheap OCR for now. what was kinda funnier was that I tried to get Claude to generate its own Go client code to upload the image and run the prompt; it totally totally hallucinated on that part :).