7 ms·

A heck of a wild bug chase

66 points by togakangaroo 2y ago

jof 2y ago
It seems to me like the underlying issue was ignoring HTTP semantics and making a state-changing link like a logout link a plain <a> (HTTP GET) and not something like a form submission (HTTP POST). Having intuition for the foundational layers of our tools saves so much time and future headaches.
- graypegg 2y ago
  To be fair, <a> tags can't send out non-GET requests. Which yes, can be interpreted as "logout controls should be buttons in forms, not links", but I would really like native htmx-like attributes for intractable html elements.
  - zbentley 2y ago
    What’s stopping you from using htmx with a form? Hx-post works as a form element attribute just fine.
    - graypegg 2y ago
      Nothing. I just think it logically makes sense as a native feature.
- Ethee 2y ago
  Genuine question: How do you believe one should learn these semantics? This is more something I've been pondering myself recently, because I agree with you that the foundational knowledge for our work in any tech stack is usually the most important for understanding higher abstractions. But with so much to know it feels impossible to 'know it all' so to speak especially if you wear more than one specialized hat. Then beyond that even if you're only trying to learn just the foundations how do you know what those foundations are if you're not already inundated in that stack? This is mostly just my personal ramblings, but I'd be curious other peoples viewpoints on this.
  - ori_b 2y ago
    The RFCs are often fairly well written and not so hard to digest.
  - pixl97 2y ago
    While it may not be quite the same answer you're looking for, I'd suggest the OWASP, and at least their top 10 for sure. Learning about SSRF may not have stopped this behavior (it's coming from the authenticated browser), but if you're doing CSRF checks you won't get logged out by random links on other peoples sites, and that whatever logged you out was a legitimate action.
  - codetrotter 2y ago
    I remember many years ago when I used to read print magazines about programming and web development. One of those magazines told a story about a web site that had lost a lot of data. What had happened? Well, somehow they had this page that 1. Required no authentication at all, and 2. Was using links like <a href="/path/to/file?action=delete>Delete file</a> And so the Google web crawler had come across this page and happily visited each and every one of those links. That’s when I learned about the importance of using forms with POST requests for certain actions instead of using links that send GET requests. And then some years later someone told me about this thing called HATEOAS and about RESTful APIs and that actually there are different HTTP verbs you can use other than just GET and POST. Like for example DELETE /path/to/file As for your question about how someone is supposed to learn that these days? Ideally whatever web development tutorials or courses or books they are using would at some point tell them about the different HTTP verbs that exists, and of how and when to use each of them, and crucially to tell them about bad consequences of using GET for anything that has side-effects like logging out a session or deleting a file.
  - 2OEH8eoCRo0 2y ago
    It requires slowing down. Unheard of.
    - morning-coffee 2y ago
      Exactly. And ditching the "move fast and break things" mindset. Learn your craft and embrace the learning process. Always be curious about how the stuff below your layer works, fundamentally. Recurse on searching for the seminal works that defined those layers. This seems appropriately relevant today: https://news.ycombinator.com/item?id=41208627 https://news.ycombinator.com/item?id=41208627 We (the industry) have built up so many layers upon layers and frameworks designed to make things easier that it just seems to attract newcomers to software engineering with this mindset that all it takes is to start with the sample-app for a high level framework, hack on it with trial and error until it does something they want, and then take to social media with proclamations of "Look! I built a thing! You can hire me to build your thing now!"
- recursivedoubts 2y ago
  This is a very good example where the HTML extensions that alex proposed here: https://www.youtube.com/watch?v=inRB6ull5WQ https://www.youtube.com/watch?v=inRB6ull5WQ (TLDW: allow buttons to make HTTP requests; allow buttons & forms to issue PUT, PATCH & DELETE; allow buttons, forms & links to target elements in the DOM by id instead of only iframes) would improve the web platform. You could have a stand-alone logout button that issues a DELETE to /session or whatever. Nice and clean.
  - williamdclt 2y ago
    I mean, it should just be a submit for a form with a /logout POST action. It’s standard and what web devs have been doing for decades
    - recursivedoubts 2y ago
      Yeah, the problem is that it requires a form, which has layout implications w/o styling and POST is not idempotent, whereas a logout operation typically is idempotent. Being able to issue a DELETE to a URL like /session from an element that doesn't have layout implications would be ideal.
      - de46le 2y ago
        A button doesn't have to be inside a form, though. You could have an empty form as a neighbour to the button (or anywhere else inside the page body), and associate the button with it. <button form="logout-form" ...>logout</button> <form name="logout-form"></form> No layout implications that way, barring any nth-child css (solvable by putting the form somewhere else). Doesn't solve the form being limited to GET/POST, but styling concerns are atleast handled.
        recursivedoubts 2y ago
        doable but rarely used, inconvenient and awkward, alex proposes allowing buttons to be stand-alone hypermedia controls which also allows multiple buttons located within a form to perform different actions (e.g. save v. cancel)
- togakangaroo 2y ago
  Author of the post here, There was no form submission, I'm not sure where you got that. There was also no POST. Though yes, I agree that in the core HTTP semantic, you wouldn't want to change state on a GET and that should include not calling `Set-Cookie`. And yet the reality is that that nearly every application - and many popular libraries like auth0 - do in fact set and clear cookies on `GET`. The issue here was that the `Link` component in NextJs - does preloading by default (which is a bad idea exactly for the above reason of reality being different from theory) - doesn't do preloading by default when running on the dev server (so you don't see the error until its deployed) - because it does preloading directly in javascript, it can't possibly follow the HTTP semantic of not actually applying cookies until later when the cached route is used Everything else was the wild goose chase bits. Also I asked claude to criticize the article as a web forum might before publishing, and this is definitely the tone it gave :D Oh, also, I'm pretty sure I got the part wrong where i was talking about the preload attribute in HTML, but so far no one's noticed. I should correct that.
  - thedanbob 2y ago
    > There was no form submission, I'm not sure where you got that. There was also no POST. OP was saying the logout function should have been behind a form submission / POST.
    - togakangaroo 2y ago
      Ah, yes, I mean, agree that would have been technically correct, but like I said, its just not how a lot of the web works. auth0-nextjs seems to react to `GET` by default (though it might also work with `POST` and you certainly can override things)
      - soneca 2y ago
        So OP was correct that a proper use of the foundational layer of HTTP would have saved time, yours in particular, right? Also, I didn’t get your ”Claude predicted your tone smiley” thing. OP tone seemed polite and clear. Your tone, on the other hand, seemed defensive and dismissive. Even after you realizing that you initially misunderstood what OP said, adding a “I mean” and a “but I like I said” to reinforce you were right even while misreading what OP said (rather than just acknowledging you got it wrong in the first reading). I would go even further and speculate that you were predisposed to get a dismissive tone from a web forum (your previous Claude test suggests that) so much that you got a perfectly fine comment and misread in a way that it felt in the “wrong tone” to you. Even misunderstanding what the post said. All of that to confirm your predisposition.
andrewfromx 2y ago
I always do <a href="#" id="logout">logout</a> so there is no url to accidentally GET call anyways.
- Etheryte 2y ago
  This is the worst of both worlds, it doesn't conform to the spec and it has incorrect semantics. See HTMHell [0] for a brief discussion on the topic, albeit with a slightly different example. [0] https://www.htmhell.dev/8-anchor-tag-used-as-button/ https://www.htmhell.dev/8-anchor-tag-used-as-button/
  - andrewfromx 2y ago
    ok fine <a href="/" id="logout">logout</a> but at least / is safe to GET
- whbrown 2y ago
  Why make it a link if you aren't linking to anything?
  - andrewfromx 2y ago
    so you don't have to set "cursor: pointer" in css
    - dzaima 2y ago
      And it gives some baseline accessibility functionality. Sucks that there isn't any gerneral "I am clickable" element that neither has bad default click behavior, nor has its own likely-unfitting styling.
      - danparsonson 2y ago
        What's wrong with a button? You can make it look like an <a> can't you?
danjl 2y ago
Authentication is a major hurdle. Back in the days of desktop software, there was no authentication. Mobile apps can often avoid authentication too. Despite decades of web coding, and lots of "this authentication system will make life easy" claims, it is still hard and easy to mess up.
- theamk 2y ago
  it's impossible to avoid authentication once you start sharing data between systems though. This includes basic stuff like "I only want to share the list with our graduates", as well things like "those preferences should be the same on every device which user X owns".
  - danjl 2y ago
    As my former boss at Sun, Scott McNeely said "the network is the computer"
deleted 2y ago
[deleted]
langsoul-com 2y ago
So would this simply work if the Link component from nextjs wasn't used?
- kaoD 2y ago
  Yes but the underlying problem would still be there, lurking for the future. If e.g. I had a browser extension to prefetch links (for faster navigation) the same issue would be present. The problem is logging out on a GET request, as discussed in current top comment. It's just semantically incorrect and many tools will (correctly) assume GET requests don't have side-effects (in HTTP's terms, it's a safe method). E.g. it's very easy to have a GET request cached by mistake (and I've seen some faulty proxies do that, completely ignoring the upstream cache-control). This is not a problem on Next or its Link component. It's on op's code (and maybe auth0-nextjs allowing logout on GET requests).
realxrobau 2y ago
So, in summary, because you added a whole bunch of stuff that didn't need to be there, it broke. Colour me surprised. It sounds like this could be implemented almost completely without any of that, especially as you were using JavaScript for the data. All I can hope is that you've learnt a lesson about unnecessary overcomplication.
throw156754228 2y ago
OP learnt about idempotency in http the hard way. However the post reminded me that I don't particularly like this hydration step of NextJS apps where there is javascript executing that is difficult if not impossible to step debug.
hschne 2y ago
> I didn’t want to deal with databases. So instead I used a third party authentication service, store some data in JSON files, and also threw up a lamda gateway to store some more data in Google Sheets? It's not relevant to the bug hunt, but I'm genuinely intrigued. Is this approach considered easier to work with than using a regular ol' DB?
- jboogie77 2y ago
  My first thought as well. This is the most complicated stack I’ve seen for something so simple. Just convinced me more to avoid using JavaScript as a backend
  - kaoD 2y ago
    Honest question: how is that JS's fault?
williamdclt 2y ago
> It is wild how decisions made in one part of a technical stack can manifest at another point in time, in another place in the stack, and in such a convoluted manner. While this isn't something that _only_ happens in modern javascript, it certainly is a pattern. These bugs are convoluted and difficult to debug because the technologies are convoluted, stacked on top of each other compounding the bugs, and devs do not understand the underlying platform. Honestly, it's once-again a good case study for why I'd stay from NextJS or "modern JS" devs (despite being in this ecosystem myself, Node/React/RN): - NextJS, one of the most modern techs in wide use, makes debugging _harder_ than vanilla JS?? This is crazy - Reimplementing browser behaviour in JS is _exactly_ what I would expect to be the root cause of various difficult-to-debug-and-to-fix bugs down the line - Using GET for a logout is a misunderstanding of HTTP semantics. This could have broken in other ways (eg integrating turbolinks in the app). Well done for debugging and fixing this, but honestly... this doesn't speak to the strength of the technology choices or the author's understanding of the platform

66 points by togakangaroo 2y ago

jof 2y ago

It seems to me like the underlying issue was ignoring HTTP semantics and making a state-changing link like a logout link a plain <a> (HTTP GET) and not something like a form submission (HTTP POST). Having intuition for the foundational layers of our tools saves so much time and future headaches.

graypegg 2y ago
To be fair, <a> tags can't send out non-GET requests. Which yes, can be interpreted as "logout controls should be buttons in forms, not links", but I would really like native htmx-like attributes for intractable html elements.
- zbentley 2y ago
  What’s stopping you from using htmx with a form? Hx-post works as a form element attribute just fine.
  - graypegg 2y ago
    Nothing. I just think it logically makes sense as a native feature.
Ethee 2y ago
Genuine question: How do you believe one should learn these semantics? This is more something I've been pondering myself recently, because I agree with you that the foundational knowledge for our work in any tech stack is usually the most important for understanding higher abstractions. But with so much to know it feels impossible to 'know it all' so to speak especially if you wear more than one specialized hat. Then beyond that even if you're only trying to learn just the foundations how do you know what those foundations are if you're not already inundated in that stack? This is mostly just my personal ramblings, but I'd be curious other peoples viewpoints on this.
- ori_b 2y ago
  The RFCs are often fairly well written and not so hard to digest.
- pixl97 2y ago
  While it may not be quite the same answer you're looking for, I'd suggest the OWASP, and at least their top 10 for sure. Learning about SSRF may not have stopped this behavior (it's coming from the authenticated browser), but if you're doing CSRF checks you won't get logged out by random links on other peoples sites, and that whatever logged you out was a legitimate action.
- codetrotter 2y ago
  I remember many years ago when I used to read print magazines about programming and web development. One of those magazines told a story about a web site that had lost a lot of data. What had happened? Well, somehow they had this page that 1. Required no authentication at all, and 2. Was using links like <a href="/path/to/file?action=delete>Delete file</a> And so the Google web crawler had come across this page and happily visited each and every one of those links. That’s when I learned about the importance of using forms with POST requests for certain actions instead of using links that send GET requests. And then some years later someone told me about this thing called HATEOAS and about RESTful APIs and that actually there are different HTTP verbs you can use other than just GET and POST. Like for example DELETE /path/to/file As for your question about how someone is supposed to learn that these days? Ideally whatever web development tutorials or courses or books they are using would at some point tell them about the different HTTP verbs that exists, and of how and when to use each of them, and crucially to tell them about bad consequences of using GET for anything that has side-effects like logging out a session or deleting a file.
- 2OEH8eoCRo0 2y ago
  It requires slowing down. Unheard of.
  - morning-coffee 2y ago
    Exactly. And ditching the "move fast and break things" mindset. Learn your craft and embrace the learning process. Always be curious about how the stuff below your layer works, fundamentally. Recurse on searching for the seminal works that defined those layers. This seems appropriately relevant today: https://news.ycombinator.com/item?id=41208627 https://news.ycombinator.com/item?id=41208627 We (the industry) have built up so many layers upon layers and frameworks designed to make things easier that it just seems to attract newcomers to software engineering with this mindset that all it takes is to start with the sample-app for a high level framework, hack on it with trial and error until it does something they want, and then take to social media with proclamations of "Look! I built a thing! You can hire me to build your thing now!"
recursivedoubts 2y ago
This is a very good example where the HTML extensions that alex proposed here: https://www.youtube.com/watch?v=inRB6ull5WQ https://www.youtube.com/watch?v=inRB6ull5WQ (TLDW: allow buttons to make HTTP requests; allow buttons & forms to issue PUT, PATCH & DELETE; allow buttons, forms & links to target elements in the DOM by id instead of only iframes) would improve the web platform. You could have a stand-alone logout button that issues a DELETE to /session or whatever. Nice and clean.
- williamdclt 2y ago
  I mean, it should just be a submit for a form with a /logout POST action. It’s standard and what web devs have been doing for decades
  - recursivedoubts 2y ago
    Yeah, the problem is that it requires a form, which has layout implications w/o styling and POST is not idempotent, whereas a logout operation typically is idempotent. Being able to issue a DELETE to a URL like /session from an element that doesn't have layout implications would be ideal.
    - de46le 2y ago
      A button doesn't have to be inside a form, though. You could have an empty form as a neighbour to the button (or anywhere else inside the page body), and associate the button with it. <button form="logout-form" ...>logout</button> <form name="logout-form"></form> No layout implications that way, barring any nth-child css (solvable by putting the form somewhere else). Doesn't solve the form being limited to GET/POST, but styling concerns are atleast handled.
      - recursivedoubts 2y ago
        doable but rarely used, inconvenient and awkward, alex proposes allowing buttons to be stand-alone hypermedia controls which also allows multiple buttons located within a form to perform different actions (e.g. save v. cancel)
togakangaroo 2y ago
Author of the post here, There was no form submission, I'm not sure where you got that. There was also no POST. Though yes, I agree that in the core HTTP semantic, you wouldn't want to change state on a GET and that should include not calling `Set-Cookie`. And yet the reality is that that nearly every application - and many popular libraries like auth0 - do in fact set and clear cookies on `GET`. The issue here was that the `Link` component in NextJs - does preloading by default (which is a bad idea exactly for the above reason of reality being different from theory) - doesn't do preloading by default when running on the dev server (so you don't see the error until its deployed) - because it does preloading directly in javascript, it can't possibly follow the HTTP semantic of not actually applying cookies until later when the cached route is used Everything else was the wild goose chase bits. Also I asked claude to criticize the article as a web forum might before publishing, and this is definitely the tone it gave :D Oh, also, I'm pretty sure I got the part wrong where i was talking about the preload attribute in HTML, but so far no one's noticed. I should correct that.
- thedanbob 2y ago
  > There was no form submission, I'm not sure where you got that. There was also no POST. OP was saying the logout function should have been behind a form submission / POST.
  - togakangaroo 2y ago
    Ah, yes, I mean, agree that would have been technically correct, but like I said, its just not how a lot of the web works. auth0-nextjs seems to react to `GET` by default (though it might also work with `POST` and you certainly can override things)
    - soneca 2y ago
      So OP was correct that a proper use of the foundational layer of HTTP would have saved time, yours in particular, right? Also, I didn’t get your ”Claude predicted your tone smiley” thing. OP tone seemed polite and clear. Your tone, on the other hand, seemed defensive and dismissive. Even after you realizing that you initially misunderstood what OP said, adding a “I mean” and a “but I like I said” to reinforce you were right even while misreading what OP said (rather than just acknowledging you got it wrong in the first reading). I would go even further and speculate that you were predisposed to get a dismissive tone from a web forum (your previous Claude test suggests that) so much that you got a perfectly fine comment and misread in a way that it felt in the “wrong tone” to you. Even misunderstanding what the post said. All of that to confirm your predisposition.

andrewfromx 2y ago

I always do <a href="#" id="logout">logout</a> so there is no url to accidentally GET call anyways.

Etheryte 2y ago
This is the worst of both worlds, it doesn't conform to the spec and it has incorrect semantics. See HTMHell [0] for a brief discussion on the topic, albeit with a slightly different example. [0] https://www.htmhell.dev/8-anchor-tag-used-as-button/ https://www.htmhell.dev/8-anchor-tag-used-as-button/
- andrewfromx 2y ago
  ok fine <a href="/" id="logout">logout</a> but at least / is safe to GET
whbrown 2y ago
Why make it a link if you aren't linking to anything?
- andrewfromx 2y ago
  so you don't have to set "cursor: pointer" in css
  - dzaima 2y ago
    And it gives some baseline accessibility functionality. Sucks that there isn't any gerneral "I am clickable" element that neither has bad default click behavior, nor has its own likely-unfitting styling.
    - danparsonson 2y ago
      What's wrong with a button? You can make it look like an <a> can't you?

danjl 2y ago

Authentication is a major hurdle. Back in the days of desktop software, there was no authentication. Mobile apps can often avoid authentication too. Despite decades of web coding, and lots of "this authentication system will make life easy" claims, it is still hard and easy to mess up.

theamk 2y ago
it's impossible to avoid authentication once you start sharing data between systems though. This includes basic stuff like "I only want to share the list with our graduates", as well things like "those preferences should be the same on every device which user X owns".
- danjl 2y ago
  As my former boss at Sun, Scott McNeely said "the network is the computer"

deleted 2y ago

[deleted]

langsoul-com 2y ago

So would this simply work if the Link component from nextjs wasn't used?

kaoD 2y ago
Yes but the underlying problem would still be there, lurking for the future. If e.g. I had a browser extension to prefetch links (for faster navigation) the same issue would be present. The problem is logging out on a GET request, as discussed in current top comment. It's just semantically incorrect and many tools will (correctly) assume GET requests don't have side-effects (in HTTP's terms, it's a safe method). E.g. it's very easy to have a GET request cached by mistake (and I've seen some faulty proxies do that, completely ignoring the upstream cache-control). This is not a problem on Next or its Link component. It's on op's code (and maybe auth0-nextjs allowing logout on GET requests).

realxrobau 2y ago

So, in summary, because you added a whole bunch of stuff that didn't need to be there, it broke. Colour me surprised. It sounds like this could be implemented almost completely without any of that, especially as you were using JavaScript for the data. All I can hope is that you've learnt a lesson about unnecessary overcomplication.

throw156754228 2y ago

OP learnt about idempotency in http the hard way. However the post reminded me that I don't particularly like this hydration step of NextJS apps where there is javascript executing that is difficult if not impossible to step debug.

hschne 2y ago

> I didn’t want to deal with databases. So instead I used a third party authentication service, store some data in JSON files, and also threw up a lamda gateway to store some more data in Google Sheets? It's not relevant to the bug hunt, but I'm genuinely intrigued. Is this approach considered easier to work with than using a regular ol' DB?

jboogie77 2y ago
My first thought as well. This is the most complicated stack I’ve seen for something so simple. Just convinced me more to avoid using JavaScript as a backend
- kaoD 2y ago
  Honest question: how is that JS's fault?

williamdclt 2y ago

> It is wild how decisions made in one part of a technical stack can manifest at another point in time, in another place in the stack, and in such a convoluted manner. While this isn't something that _only_ happens in modern javascript, it certainly is a pattern. These bugs are convoluted and difficult to debug because the technologies are convoluted, stacked on top of each other compounding the bugs, and devs do not understand the underlying platform. Honestly, it's once-again a good case study for why I'd stay from NextJS or "modern JS" devs (despite being in this ecosystem myself, Node/React/RN): - NextJS, one of the most modern techs in wide use, makes debugging _harder_ than vanilla JS?? This is crazy - Reimplementing browser behaviour in JS is _exactly_ what I would expect to be the root cause of various difficult-to-debug-and-to-fix bugs down the line - Using GET for a logout is a misunderstanding of HTTP semantics. This could have broken in other ways (eg integrating turbolinks in the app). Well done for debugging and fixing this, but honestly... this doesn't speak to the strength of the technology choices or the author's understanding of the platform