3 ms·

I agree with most of the stuff in this article but I'm a bit puzzled by some of the attitudes of the author. They seem to care more about the LLM delivering cod

I agree with most of the stuff in this article but I'm a bit puzzled by some of the attitudes of the author. They seem to care more about the LLM delivering code with poor accessibility than they care about the LLM delivering completely wrong answers. The "any good developer would realize this is bad code" rings strongly as "no true developer would think the LLM's bad answer was correct". Seems like a short sighted opinion to me. I also think you can replace "accessibility" with any number of programming meta concepts and find problems too. How about "internationalization"? Are LLMs any good at producing code that is nicely internationalized? Or more importantly "security". Are LLMs going to produce millions of lines of poorly secured code that people never double check? Almost assuredly. The fact is that LLMs are prediction engines. They run off of probabilities based on the prompt and the training model. Thus, unless the training model is weighted towards cherry picked examples of excellent code, it's going to follow the masses. And the masses write bad-to-average code mostly.

wlesieutre 3y ago
> They seem to care more about the LLM delivering code with poor accessibility than they care about the LLM delivering completely wrong answers. I think the idea with this is that if it gives you completely wrong answers and the code doesn't work, it will obviously not work and have to figure out how to fix it. Meanwhile when it gives you code that appears to do what you wanted except the accessibility is broken, you'll ship it because you don't realize there's anything wrong with it.
- bluefirebrand 3y ago
  The problem is that often it will give answers that are only subtly wrong, and those will get shipped too. I think my puzzlement is with the focus on accessibility as though it was a high priority item. In my experience it's usually an afterthought, if it's a thought at all. Personally I've never worked on a codebase where accessibility was in the top 5 priorities. No one would ever block a prod release for an accessibility mistake. But like I said, you could take this whole argument, find+replace "accessibility" with "security" and you would have a much more compelling argument imo. Given time constraints, code should prioritize security over accessibility basically always.
  - runarberg 3y ago
    I don’t think you are a minority in web development, but if you are shipping to a wide user base on the open web, you are definitely on the wrong, and you may be doing something illegal. As far as web development goes, accessibility is actually something you must screw up, rather then something you have to build up. In most cases what you do is accessible by default (as is talked about in this article) and you have to do something weird to brake it. What you build by not thinking about accessibility might not have the best usability for assistive technology, but it should at least work. That said, throughout my 10+ years as a web developer, I have consistently been reminded about accessibility. It is all over the literature, if you go to a random page on MDN there is probably a bullet point about accessibility implications. As a student, accessibility was at the forefront. In fact, as an expert front end developer, it is my responsibility to make sure what I build is accessible. In fact, project managers often don’t know this, and I have to explain to them. A good project manager would know to take an expert advice. > you could take this whole argument, find+replace "accessibility" with "security" and you would have a much more compelling argument imo. Given time constraints, code should prioritize security over accessibility basically always. I’m sorry, but this mentality demonstrates a massive disrespect for a portion of your user base (given you are targeting a general audience; as opposed to internal tools). If your work can’t be used by somebody with a disability (or because their touchpad stopped working), you are not only being rather rude, but you may be braking the law. Everybody deserves the possibility to use your work equally.
  - EscargotCult 3y ago
    I'll just quote from Stevey's Google Platforms Rant > Like anything else big and important in life, Accessibility has an evil twin who, jilted by the unbalanced affection displayed by their parents in their youth, has grown into an equally powerful Arch-Nemesis (yes, there's more than one nemesis to accessibility) named Security. And boy howdy are the two ever at odds. > But I'll argue that Accessibility is actually more important than Security because dialing Accessibility to zero means you have no product at all, whereas dialing Security to zero can still get you a reasonably successful product such as the Playstation Network. I'm fully aware that I'm commenting with a drive-by facetious block quote, but it is a reality that "insecure but accessible" has more users than "secure but inaccessible".
- im3w1l 3y ago
  To me this sounds just like human written code that mostly works but has a couple of issues. I don't see why we couldn't apply the same techniques to deal with it - unit testing, review, qa.

wlesieutre 3y ago

> They seem to care more about the LLM delivering code with poor accessibility than they care about the LLM delivering completely wrong answers. I think the idea with this is that if it gives you completely wrong answers and the code doesn't work, it will obviously not work and have to figure out how to fix it. Meanwhile when it gives you code that appears to do what you wanted except the accessibility is broken, you'll ship it because you don't realize there's anything wrong with it.

bluefirebrand 3y ago
The problem is that often it will give answers that are only subtly wrong, and those will get shipped too. I think my puzzlement is with the focus on accessibility as though it was a high priority item. In my experience it's usually an afterthought, if it's a thought at all. Personally I've never worked on a codebase where accessibility was in the top 5 priorities. No one would ever block a prod release for an accessibility mistake. But like I said, you could take this whole argument, find+replace "accessibility" with "security" and you would have a much more compelling argument imo. Given time constraints, code should prioritize security over accessibility basically always.
- runarberg 3y ago
  I don’t think you are a minority in web development, but if you are shipping to a wide user base on the open web, you are definitely on the wrong, and you may be doing something illegal. As far as web development goes, accessibility is actually something you must screw up, rather then something you have to build up. In most cases what you do is accessible by default (as is talked about in this article) and you have to do something weird to brake it. What you build by not thinking about accessibility might not have the best usability for assistive technology, but it should at least work. That said, throughout my 10+ years as a web developer, I have consistently been reminded about accessibility. It is all over the literature, if you go to a random page on MDN there is probably a bullet point about accessibility implications. As a student, accessibility was at the forefront. In fact, as an expert front end developer, it is my responsibility to make sure what I build is accessible. In fact, project managers often don’t know this, and I have to explain to them. A good project manager would know to take an expert advice. > you could take this whole argument, find+replace "accessibility" with "security" and you would have a much more compelling argument imo. Given time constraints, code should prioritize security over accessibility basically always. I’m sorry, but this mentality demonstrates a massive disrespect for a portion of your user base (given you are targeting a general audience; as opposed to internal tools). If your work can’t be used by somebody with a disability (or because their touchpad stopped working), you are not only being rather rude, but you may be braking the law. Everybody deserves the possibility to use your work equally.
- EscargotCult 3y ago
  I'll just quote from Stevey's Google Platforms Rant > Like anything else big and important in life, Accessibility has an evil twin who, jilted by the unbalanced affection displayed by their parents in their youth, has grown into an equally powerful Arch-Nemesis (yes, there's more than one nemesis to accessibility) named Security. And boy howdy are the two ever at odds. > But I'll argue that Accessibility is actually more important than Security because dialing Accessibility to zero means you have no product at all, whereas dialing Security to zero can still get you a reasonably successful product such as the Playstation Network. I'm fully aware that I'm commenting with a drive-by facetious block quote, but it is a reality that "insecure but accessible" has more users than "secure but inaccessible".
im3w1l 3y ago
To me this sounds just like human written code that mostly works but has a couple of issues. I don't see why we couldn't apply the same techniques to deal with it - unit testing, review, qa.