6 ms·

Show HN: Checking Pope's election results with smoke test script for chimney

26 points by vasusen 1y ago

This Playwright test script uses AI to test if there's smoke coming out of the Sistine Chapel chimney and whether that smoke is white. The test only passes if the smoke is white. Currently, set to use Google Gemini Flash 2.0, but you can switch it to use other LLM providers/models by setting the environment variable in the Github actions workflow: https://github.com/donobu-inc/donobu-papal-election-tests/blob/main/.github/workflows/tests.yaml https://github.com/donobu-inc/donobu-papal-election-tests/bl... I've set it to run every minute during the Papal Conclave election times - https://github.com/donobu-inc/donobu-papal-election-tests/actions https://github.com/donobu-inc/donobu-papal-election-tests/ac...

hcaz 1y ago
What was your motivation to use AI for this instead of simple image analysis?
- akmarinov 1y ago
  AI’s cool
  - codr7 1y ago
    AI is for cool fools
- ksajh 1y ago
  Writing prompts is much simpler than image analysis plus they make a AI test framework thingy
- vasusen 1y ago
  Image analysis would be better. I just wanted to test it quickly with different models (Gemini, GPT 4o, etc.) using an AI testing framework I am building
devonbleak 1y ago
Alternative version: check for dings on my phone from every news outlet sending a notification about it.
akmarinov 1y ago
Test should be passing now
- vasusen 1y ago
  It passed right when the smoke started coming out - https://github.com/donobu-inc/donobu-papal-election-tests/blob/main/demo/logs.png https://github.com/donobu-inc/donobu-papal-election-tests/bl...
alkh 1y ago
How much did you end up spending on API credits for Flash 2.0?
- vasusen 1y ago
  $0.29 over the past 2 days
sfcurryniggers 1y ago
[dead]
aitchnyu 1y ago
Instead of AI looking at your code and browser and writing Playwright scripts, AI is directly controlling browser and asserting over tests. Do we have to wait for on-prem multimodal low latency AI for this to be viable? And nice "smoke test" and making me curious about your product.
- vasusen 1y ago
  Yes, Google gemini flash and other models are reasonably fast but on-prem multimodal models will make these dramatically better. We are prepping for that future by being local-first including a desktop app.

26 points by vasusen 1y ago

hcaz 1y ago

What was your motivation to use AI for this instead of simple image analysis?

akmarinov 1y ago
AI’s cool
- codr7 1y ago
  AI is for cool fools
ksajh 1y ago
Writing prompts is much simpler than image analysis plus they make a AI test framework thingy
vasusen 1y ago
Image analysis would be better. I just wanted to test it quickly with different models (Gemini, GPT 4o, etc.) using an AI testing framework I am building

devonbleak 1y ago

Alternative version: check for dings on my phone from every news outlet sending a notification about it.

akmarinov 1y ago

Test should be passing now

vasusen 1y ago
It passed right when the smoke started coming out - https://github.com/donobu-inc/donobu-papal-election-tests/blob/main/demo/logs.png https://github.com/donobu-inc/donobu-papal-election-tests/bl...

alkh 1y ago

How much did you end up spending on API credits for Flash 2.0?

vasusen 1y ago
$0.29 over the past 2 days

sfcurryniggers 1y ago

[dead]

aitchnyu 1y ago

Instead of AI looking at your code and browser and writing Playwright scripts, AI is directly controlling browser and asserting over tests. Do we have to wait for on-prem multimodal low latency AI for this to be viable? And nice "smoke test" and making me curious about your product.

vasusen 1y ago
Yes, Google gemini flash and other models are reasonably fast but on-prem multimodal models will make these dramatically better. We are prepping for that future by being local-first including a desktop app.