4 ms·

I have been working on something for NYC. Its not ready yet but its basically an AI-native DOHMH. I took all of these https://www.nyc.gov/site/doh/health/heal

I have been working on something for NYC. Its not ready yet but its basically an AI-native DOHMH. I took all of these https://www.nyc.gov/site/doh/health/health-topics.page https://www.nyc.gov/site/doh/health/health-topics.page and turned them into guided protocols, staffed w/ the required agents and HITL. Here are the "Programs" - https://dohmh.codify.nyc/programs https://dohmh.codify.nyc/programs example protocol: https://dohmh.codify.nyc/protocol/71776 https://dohmh.codify.nyc/protocol/71776 I was hoping it would be enough to stand out and get an interview w/ the "PIT Crew" but no such luck, reach out if you want to chat or hack on some NYC/Civic stuff.

Wilduck 2mo ago
My man, you're posting this project all over the HN comments, but I think you need to step back and really think about what you're building and what the user experience is. Your link to an example protocol: https://dohmh.codify.nyc/protocol/71776 https://dohmh.codify.nyc/protocol/71776 makes basically zero sense to me. Similarly the program page. What do you expect someone clicking on these links to take away from them?
- deleted 2mo ago
  [deleted]
- Wilduck 2mo ago
  Your reply to this comment got flagged dead for some reason (not sure why, it seemed like a good faith comment) so I'm responding with a couple thoughts here. Looking at your set of steps, I'm definitely getting lost around step 3 "The program then guides you one step at a time. Each step is a small module: a short assessment (a few structured questions), a task, a referral, a verification, and so on. It adapts to your answers rather than dumping a wall of text." Am I wrong that this flow is not built yet? Or is completely inaccessible to me? Like, looking at the program page https://dohmh.codify.nyc/program/71776 https://dohmh.codify.nyc/program/71776 I don't see how I can follow any of the steps. You say this is drawn from DOHMH pages, so I found the corresponding one for this program: https://www.nyc.gov/site/doh/health/health-topics/stay-healthy.page https://www.nyc.gov/site/doh/health/health-topics/stay-healt.... I think this page is miles better than what you've built for a few reasons: 1. It comes with context in the form of paragraphs of explanation 2. It doesn't use undefined words like "program", "agent", "target". No where on that page is a string like "PersonalPrepTaskAgent". 3. It provides links to more resources, also written in nice human readable language I don't want to be too hard on your project though. I've had to interact with the Pennsylvania DMV website a number of times, and trying to understand how to meet the requirements for a residential parking permit when I moved into my house was crazy because I needed some non-standard documentation due to having out-of-state plates. I would have loved a someone to break down the steps to me in very clear, understandable language. But I'm not sure your site is doing that right now. As a last note, I wouldn't say you're "spamming" the comments, but I might recommend reaching out to potential users directly and asking them for harsh feedback. Posting into the void here clearly isn't getting you enough actionable feedback.
  - arionhardison 2mo ago
    I have no idea either LOL.
  - arionhardison 2mo ago
    Its just not accessible to you, I am removing the login requirement right now because I think if you could easily click "Start" it might become a little clearer. This is really great feedback man; and I am literally going to fix these things right now. 1. I am going to open up all the programs so users can click through all of the steps and really understand, like a preview. 2. I am going to just use my horrible writing skills and do the copy myself. I am very insecure about my writing so I leaned on AI for that probably way too much. 3. I am going to add a video NOT of program/protocols but of the actual process that its automating; e.g.: https://wiki.codify.nyc/agent/posts/nyc-311-apply-victim-compensation https://wiki.codify.nyc/agent/posts/nyc-311-apply-victim-com... and some of the user testimonials; for example - the core use case is care plan automation. 1k of the 2k users I was speaking about all were from crohns.ai I think I not only missed the mark here but did the exact opposite of what I wanted to. Codification is supposed to simplify complex workflows and it seems like I did the opposite here and I can totally see how nobody would ever be able to tell that I am automating a process here. The amount of embarrassment you just saved me I cannot thank you enough for. I was about to do my first "email/blast" style marketing campaign but I am going to dig into this first. If you don't mind, I would like to email you when I made the fixes. Edit: The thing I don't know how to express is that the page you like, that can take a lot of work. I want to automated all of the things possible so that instead of reading a page its essentially just a program and the agent guides you (for real) through everything required to complete whatever process or flow the page describes.
    - Wilduck 2mo ago
      Glad the feedback was helpful! Feel free to email me at either email in my profile. I think your next steps laid out here are good ideas. In particular, fully documenting (with video or text) a handful of actual successful uses of your site / the programs seems like a great next step.
- arionhardison 2mo ago
  Damn, thats stings but...totally fair so here is a basic breakdown - 1. A user would start at https://dohmh.codify.nyc https://dohmh.codify.nyc and say and say whatever their intent is, e.g.: "I think I got food poisoning" or "I think I have COVID" etc... 2. The system would then either find a program for that intent or help the user make a protocol to reach their desired outcome (for that intent). 3. The program is a set of modules like: Assessments, Followups, Services etc... and the user is guided through each of them in an automated fashon. HITL is there when needed but the agent is the primary guide. The whole point is that lots of processes just break down to a series of steps and deterministic logic that can be "codified". Thats why I call it "Codify". I have had about 2000 programs completed but thats only 15% of the programs that users have started; so I think you may be right. I really need to improve my messaging. One another note, I genuinely would like to thank you for the feedback. Critical feedback is hard to hear many time but its by far the most useful. I in no way meant to spam the comments but I don't otherwise know how to execute on the "tell people" part of "Do things, tell people".
  - pibaker 1mo ago
    > https://dohmh.codify.nyc https://dohmh.codify.nyc Why are you using a voice recognition box instead of just a plain old search bar? It took me a second to realize what's even going on and I backed out because I ain't giving you microphone access. If people want to use voice input all phone operating systems already have that. I'd really not want to talk about my symptoms if my throat is sore from covid too. Then I found some of the actual protocols on your website, like the "Dohmh Weight And Health." I don't understand a single thing going on on that page. Why does clicking something that looks like a video play button take me to a page with a chatbot? Why is there a list of "policy makers" on the side? I am being shown eight steps and they all look like "health Context HealthContextAgent blah blah." What am I even looking at??? I don't want to be rude but this feels a textbook example of having too many tokens on your account and too little understanding of the problems you are trying to solve.
    - arionhardison 1mo ago
      You are NOT at all being rude. I actually feel like I am hitting the jackpot today with this and the previous feedbacks so I would also like to thank you. I would like to provide some explanation - 0. In prison you cannot always type things on your tablet. Often, we preferred phone / voice interfaces. I am correcting this now; defaulting to text input. 1. My pops was a social service worker for 30 years; he would always explain to me all the repetitive stuff that people had to deal with and how it hurt them so this: "Codify" is meant to be automated workflows that mitigate that barrier. Codification seemed like the best way to do that; I still believe this and its not domain specific, speaking to lack of problem understanding. The agent leads people through the "codified" workflows (aka: Programs). 2. I have no tokens, budget or other; I setup my own local coding ENV so maybe change textbooks, LOL (kidding ofcorse; glut of tokens = AI slop i get it). 3. What am I even looking at??? - When you enter things like "I think I have COVID" into ChatGPT etc.. you get a wall of text that people don't want to read. When people go here https://www.nyc.gov/site/doh/covid/covid-19-main.page https://www.nyc.gov/site/doh/covid/covid-19-main.page - same thing. What you are looking at is my attempt to turn these walls of text into real automated workflows that have agents and HITL's guide them from an intent to an outcome. I obviously am not doing that good a job ATM but 2k+m users have completed their programs w/ verified outcomes but thats less than 15% of the programs initiated so your feedback is extremely welcomed and appreciated.
xerox13ster 1mo ago
Stop what you are doing and please for the love of god waste your time and tokens on something, anything else before you get yourself fined. Who is Alister Martin and why would I willingly verbally tell this page my medical or mental health symptoms? What kind of data protections does this LLM have against erroneously sharing HIPAA PHI, because that alone looks like a HIPAA violation waiting to happen. Am I signing a HIPAA release form to allow Alister to share my Protected Health Information with the Counselors, SMEs, clinics, and clinicians it finds? If you have not even conisdered HIPAA compliance, which it looks like you haven't, now that you've been made aware that this needs addressed it can be considered neglect and YOU ARE RISKING UP TO A $73,000 FINE PER HIPAA VIOLATION!!!! The protocol is a mess that seems to have nothing for the user to actually do, and is full of what are apparently prompts to an LLM? It's laid out like a flowchart, but it has absolutely no intuitive or indicated flow. When I click on things it's locked. How exactly is protocol population relevant to staying healthy? Why on earth would I ever hand my medical history over to an LLM? What is this site supposed to actually do? When I click start program I get brought to what looks like an LLM dev environment, not a step-by-step program to follow. Am I supposed to give it all my personal information and medical history then tell it I want to kill myself so it can recommend mental hygiene like getting better sleep and trying to avoid noise in NYC, or tell me it's ok, that I'm not ending it all, and that I will be joining it in the digital afterlife? You're really trusting these agents to make the determination that someone is in crisis? When I'm in crisis, I shut down. I'm not reaching for my phone or laptop. I don't even interact with my loved ones in that state; interacting with a chatbot who tells me to call 988 would be the very thing that makes me actually kill myself. It's already insulting enough when I'm googling my thoughts trying to find other real people who have dealt with them and gotten through and it takes up the whole page to tell me to call some person ignorant of my situation so they can say "damn that's crazy, don't kill yourself tho it gets better" for 40 minutes while I sob. Nobody wants or needs AI-native "Mental Hygiene". The product is giving vulnerable people AI psychosis. The very last thing that should be introduced is an "AI" "mental health resource" from a trusted government agency. I truly, sincerely hope the PIT crew saw AI-native and put you on a permanent blacklist. The site is not well built in any sense of the word. Have YOU even used the site, critically, as though you were seeing it for the first time? It is befuddling at best, and actively confusing on main. The site is not navigable, because there is no top bar, hamburger menu, or nav links anywhere on the Program page. When I visit the programs link (which is inaccessible from any page but itself or a protocol page), and then go to the second page of programs, the link turns into this monstrosity: https://dohmh.codify.nyc/programs/0-all/0-all/0-all/0-all/0-all/new-first/popular/0-all/5 https://dohmh.codify.nyc/programs/0-all/0-all/0-all/0-all/0-... What is the purpose of the 8 load bearing '0-all's? If I remove one of them, I get a page with no programs. They don't stack by what page of results you're on, that's the number at the end. Then you also handle sort and popularity sort in the URL (with popularity sort cryptically using one of the last two '0-all's for some reason), indicating that those are SSR pages, but upon inspection, the sorted data is provided as a JSON payload parsed by your page script. Why change the URL if you are getting the data via JSON payload from the server? Why not just hydrate the data within the structure of the page and show a UI indicator for the current sort (which you currently do NOT do at all for popularity). "Browse through thousands of programs" > There are 212. I dunno about you, but I'm pretty sure you're a significant digit short there. I learned that thousand had four digits, not three, when I was 4 years old. While inspecting I found the structure behind that URL above: "pages/programs/_category/_tag/_level/_signs/_time/_access/_order/_popular/_search/index", but that doesn't explain why the category,level, signs, time or access are '0-all', or what that indicates for the page.....OH MY GOD! is the /_signs/ section of that URL so that you can URL-encode what currency symbol (price sign, as you put it) to use, such as dollar sign, Euro, Pound???!! Please tell me this is not the case. I don't even want to see the source code for this. I could almost guarantee you have hardcoded secrets and credentials in the running production codebase. I'm sorry, I know that this strays pretty far from the HN guidelines and ethos but I just can't restrain myself here. This is so unbelievably, utterly dispiriting as someone who's cared about technology and it's development and good practices for much of my life. I can't approach this with anything but a sharp tongue and critical eye, because if you won't look at it yourself (and you didn't), someone else has to tell you what they see.

Wilduck 2mo ago

My man, you're posting this project all over the HN comments, but I think you need to step back and really think about what you're building and what the user experience is. Your link to an example protocol: https://dohmh.codify.nyc/protocol/71776 https://dohmh.codify.nyc/protocol/71776 makes basically zero sense to me. Similarly the program page. What do you expect someone clicking on these links to take away from them?

deleted 2mo ago
[deleted]
Wilduck 2mo ago
Your reply to this comment got flagged dead for some reason (not sure why, it seemed like a good faith comment) so I'm responding with a couple thoughts here. Looking at your set of steps, I'm definitely getting lost around step 3 "The program then guides you one step at a time. Each step is a small module: a short assessment (a few structured questions), a task, a referral, a verification, and so on. It adapts to your answers rather than dumping a wall of text." Am I wrong that this flow is not built yet? Or is completely inaccessible to me? Like, looking at the program page https://dohmh.codify.nyc/program/71776 https://dohmh.codify.nyc/program/71776 I don't see how I can follow any of the steps. You say this is drawn from DOHMH pages, so I found the corresponding one for this program: https://www.nyc.gov/site/doh/health/health-topics/stay-healthy.page https://www.nyc.gov/site/doh/health/health-topics/stay-healt.... I think this page is miles better than what you've built for a few reasons: 1. It comes with context in the form of paragraphs of explanation 2. It doesn't use undefined words like "program", "agent", "target". No where on that page is a string like "PersonalPrepTaskAgent". 3. It provides links to more resources, also written in nice human readable language I don't want to be too hard on your project though. I've had to interact with the Pennsylvania DMV website a number of times, and trying to understand how to meet the requirements for a residential parking permit when I moved into my house was crazy because I needed some non-standard documentation due to having out-of-state plates. I would have loved a someone to break down the steps to me in very clear, understandable language. But I'm not sure your site is doing that right now. As a last note, I wouldn't say you're "spamming" the comments, but I might recommend reaching out to potential users directly and asking them for harsh feedback. Posting into the void here clearly isn't getting you enough actionable feedback.
- arionhardison 2mo ago
  I have no idea either LOL.
- arionhardison 2mo ago
  Its just not accessible to you, I am removing the login requirement right now because I think if you could easily click "Start" it might become a little clearer. This is really great feedback man; and I am literally going to fix these things right now. 1. I am going to open up all the programs so users can click through all of the steps and really understand, like a preview. 2. I am going to just use my horrible writing skills and do the copy myself. I am very insecure about my writing so I leaned on AI for that probably way too much. 3. I am going to add a video NOT of program/protocols but of the actual process that its automating; e.g.: https://wiki.codify.nyc/agent/posts/nyc-311-apply-victim-compensation https://wiki.codify.nyc/agent/posts/nyc-311-apply-victim-com... and some of the user testimonials; for example - the core use case is care plan automation. 1k of the 2k users I was speaking about all were from crohns.ai I think I not only missed the mark here but did the exact opposite of what I wanted to. Codification is supposed to simplify complex workflows and it seems like I did the opposite here and I can totally see how nobody would ever be able to tell that I am automating a process here. The amount of embarrassment you just saved me I cannot thank you enough for. I was about to do my first "email/blast" style marketing campaign but I am going to dig into this first. If you don't mind, I would like to email you when I made the fixes. Edit: The thing I don't know how to express is that the page you like, that can take a lot of work. I want to automated all of the things possible so that instead of reading a page its essentially just a program and the agent guides you (for real) through everything required to complete whatever process or flow the page describes.
  - Wilduck 2mo ago
    Glad the feedback was helpful! Feel free to email me at either email in my profile. I think your next steps laid out here are good ideas. In particular, fully documenting (with video or text) a handful of actual successful uses of your site / the programs seems like a great next step.
arionhardison 2mo ago
Damn, thats stings but...totally fair so here is a basic breakdown - 1. A user would start at https://dohmh.codify.nyc https://dohmh.codify.nyc and say and say whatever their intent is, e.g.: "I think I got food poisoning" or "I think I have COVID" etc... 2. The system would then either find a program for that intent or help the user make a protocol to reach their desired outcome (for that intent). 3. The program is a set of modules like: Assessments, Followups, Services etc... and the user is guided through each of them in an automated fashon. HITL is there when needed but the agent is the primary guide. The whole point is that lots of processes just break down to a series of steps and deterministic logic that can be "codified". Thats why I call it "Codify". I have had about 2000 programs completed but thats only 15% of the programs that users have started; so I think you may be right. I really need to improve my messaging. One another note, I genuinely would like to thank you for the feedback. Critical feedback is hard to hear many time but its by far the most useful. I in no way meant to spam the comments but I don't otherwise know how to execute on the "tell people" part of "Do things, tell people".
- pibaker 1mo ago
  > https://dohmh.codify.nyc https://dohmh.codify.nyc Why are you using a voice recognition box instead of just a plain old search bar? It took me a second to realize what's even going on and I backed out because I ain't giving you microphone access. If people want to use voice input all phone operating systems already have that. I'd really not want to talk about my symptoms if my throat is sore from covid too. Then I found some of the actual protocols on your website, like the "Dohmh Weight And Health." I don't understand a single thing going on on that page. Why does clicking something that looks like a video play button take me to a page with a chatbot? Why is there a list of "policy makers" on the side? I am being shown eight steps and they all look like "health Context HealthContextAgent blah blah." What am I even looking at??? I don't want to be rude but this feels a textbook example of having too many tokens on your account and too little understanding of the problems you are trying to solve.
  - arionhardison 1mo ago
    You are NOT at all being rude. I actually feel like I am hitting the jackpot today with this and the previous feedbacks so I would also like to thank you. I would like to provide some explanation - 0. In prison you cannot always type things on your tablet. Often, we preferred phone / voice interfaces. I am correcting this now; defaulting to text input. 1. My pops was a social service worker for 30 years; he would always explain to me all the repetitive stuff that people had to deal with and how it hurt them so this: "Codify" is meant to be automated workflows that mitigate that barrier. Codification seemed like the best way to do that; I still believe this and its not domain specific, speaking to lack of problem understanding. The agent leads people through the "codified" workflows (aka: Programs). 2. I have no tokens, budget or other; I setup my own local coding ENV so maybe change textbooks, LOL (kidding ofcorse; glut of tokens = AI slop i get it). 3. What am I even looking at??? - When you enter things like "I think I have COVID" into ChatGPT etc.. you get a wall of text that people don't want to read. When people go here https://www.nyc.gov/site/doh/covid/covid-19-main.page https://www.nyc.gov/site/doh/covid/covid-19-main.page - same thing. What you are looking at is my attempt to turn these walls of text into real automated workflows that have agents and HITL's guide them from an intent to an outcome. I obviously am not doing that good a job ATM but 2k+m users have completed their programs w/ verified outcomes but thats less than 15% of the programs initiated so your feedback is extremely welcomed and appreciated.

xerox13ster 1mo ago

Stop what you are doing and please for the love of god waste your time and tokens on something, anything else before you get yourself fined. Who is Alister Martin and why would I willingly verbally tell this page my medical or mental health symptoms? What kind of data protections does this LLM have against erroneously sharing HIPAA PHI, because that alone looks like a HIPAA violation waiting to happen. Am I signing a HIPAA release form to allow Alister to share my Protected Health Information with the Counselors, SMEs, clinics, and clinicians it finds? If you have not even conisdered HIPAA compliance, which it looks like you haven't, now that you've been made aware that this needs addressed it can be considered neglect and YOU ARE RISKING UP TO A $73,000 FINE PER HIPAA VIOLATION!!!! The protocol is a mess that seems to have nothing for the user to actually do, and is full of what are apparently prompts to an LLM? It's laid out like a flowchart, but it has absolutely no intuitive or indicated flow. When I click on things it's locked. How exactly is protocol population relevant to staying healthy? Why on earth would I ever hand my medical history over to an LLM? What is this site supposed to actually do? When I click start program I get brought to what looks like an LLM dev environment, not a step-by-step program to follow. Am I supposed to give it all my personal information and medical history then tell it I want to kill myself so it can recommend mental hygiene like getting better sleep and trying to avoid noise in NYC, or tell me it's ok, that I'm not ending it all, and that I will be joining it in the digital afterlife? You're really trusting these agents to make the determination that someone is in crisis? When I'm in crisis, I shut down. I'm not reaching for my phone or laptop. I don't even interact with my loved ones in that state; interacting with a chatbot who tells me to call 988 would be the very thing that makes me actually kill myself. It's already insulting enough when I'm googling my thoughts trying to find other real people who have dealt with them and gotten through and it takes up the whole page to tell me to call some person ignorant of my situation so they can say "damn that's crazy, don't kill yourself tho it gets better" for 40 minutes while I sob. Nobody wants or needs AI-native "Mental Hygiene". The product is giving vulnerable people AI psychosis. The very last thing that should be introduced is an "AI" "mental health resource" from a trusted government agency. I truly, sincerely hope the PIT crew saw AI-native and put you on a permanent blacklist. The site is not well built in any sense of the word. Have YOU even used the site, critically, as though you were seeing it for the first time? It is befuddling at best, and actively confusing on main. The site is not navigable, because there is no top bar, hamburger menu, or nav links anywhere on the Program page. When I visit the programs link (which is inaccessible from any page but itself or a protocol page), and then go to the second page of programs, the link turns into this monstrosity: https://dohmh.codify.nyc/programs/0-all/0-all/0-all/0-all/0-all/new-first/popular/0-all/5 https://dohmh.codify.nyc/programs/0-all/0-all/0-all/0-all/0-... What is the purpose of the 8 load bearing '0-all's? If I remove one of them, I get a page with no programs. They don't stack by what page of results you're on, that's the number at the end. Then you also handle sort and popularity sort in the URL (with popularity sort cryptically using one of the last two '0-all's for some reason), indicating that those are SSR pages, but upon inspection, the sorted data is provided as a JSON payload parsed by your page script. Why change the URL if you are getting the data via JSON payload from the server? Why not just hydrate the data within the structure of the page and show a UI indicator for the current sort (which you currently do NOT do at all for popularity). "Browse through thousands of programs" > There are 212. I dunno about you, but I'm pretty sure you're a significant digit short there. I learned that thousand had four digits, not three, when I was 4 years old. While inspecting I found the structure behind that URL above: "pages/programs/_category/_tag/_level/_signs/_time/_access/_order/_popular/_search/index", but that doesn't explain why the category,level, signs, time or access are '0-all', or what that indicates for the page.....OH MY GOD! is the /_signs/ section of that URL so that you can URL-encode what currency symbol (price sign, as you put it) to use, such as dollar sign, Euro, Pound???!! Please tell me this is not the case. I don't even want to see the source code for this. I could almost guarantee you have hardcoded secrets and credentials in the running production codebase. I'm sorry, I know that this strays pretty far from the HN guidelines and ethos but I just can't restrain myself here. This is so unbelievably, utterly dispiriting as someone who's cared about technology and it's development and good practices for much of my life. I can't approach this with anything but a sharp tongue and critical eye, because if you won't look at it yourself (and you didn't), someone else has to tell you what they see.