5 ms·

> I’ve needed to send off an HTTP request with some data to log when a user does something like navigate to a different page or submit a form. No need for that

> I’ve needed to send off an HTTP request with some data to log when a user does something like navigate to a different page or submit a form. No need for that. Stop hoarding data!

capableweb 5y ago
What's wrong with doing something like this? If it contains PII, then I agree, the additional hassle of dealing with everything that comes with handling PII (like GDPR) becomes too much, easier to just don't do it. But if it doesn't contain PII, it can be useful to see how many people drop off a form VS submitting it for example.
- Moru 5y ago
  Record how many opens the form and compare to how many actually submit the form?
  - capableweb 5y ago
    Yeah, I'm not saying this is a silver-bullet that works in all cases, but some use cases do work better with it. For example, if said form shows one question at a time without any backend requests being done between questions, you could do one last try to log which question the user dropped off at. But again, no silver-bullet.
  - vntok 5y ago
    How many users submitted the form vs how many users reached the form's page won't tell you anything about why they left or how long they struggled before submitting. A beacon sent for each click on links on the form's page will tell you how many users left for this or that other page, which is very useful to know and optimize by removing the link and/or bringing its contents to the page itself. A beacon sent for each form submission attempt will tell you how many users make mistakes while filling in the form, how good the error messages are, and whether the users tend to fix their input or leave. Bonus points for incrementing an "attempt" hidden field on each submit button click so that you can see how many attempts it takes for all fields to be filled correctly. Tracking what works and what doesn't can drive a lot of UX improvements, for the immediate and measurable benefit of the user.
- fivea 5y ago
  > If it contains PII (...) How can you tell?
  - capableweb 5y ago
    How you can tell if the data you're collecting is PII? I guess there are many definitions of PII, but I generally base my own understanding on the GDPR definition: > ‘Personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person.
    - fivea 5y ago
      > How you can tell if the data you're collecting is PII? No. How can you tell, as a user of any random website, that the script your browser is running as part of that page you've opened isn't shipping PII collected from your usage.
      - mattlondon 5y ago
        You can tell by not entering any PII into the site in the first place. IP is technically PII in some places. Personally I am not worried if a criminal gets "my" IP in the same way that I am not worried if they have my phone number. I would be worried if they had my name address age bank account info etc, but then I don't give that out freely
        fivea 5y ago
        > You can tell by not entering any PII into the site in the first place. That's not the definition of PII. At best, that covers a small subset of PII. PII means any information that can be used to identify you, either directly or indirectly. When you access a website combined with which link you opened or which search keywords you used can be used to infer who you are. How do you tell if a script is not shipping that info to an undisclosed third-party?
- tantalor 5y ago
  IP addr & cookies are PII
  - capableweb 5y ago
    IP addresses, yes. Cookies not always, depends on what you store in them. So my point remains, nothing wrong with hoarding data if there is no PII.
soylentgraham 5y ago
Maybe it was to log a player off from a game, rather than have a dangling player. (Though websocket would work better)
- deleted 5y ago
  [deleted]
- inglor 5y ago
  This is true and in general websockets (well, and webtransport/webrtc etc) are the only reliable way to detect navigation from the backend since none of the methods mentioned in the article work if the user simply closed the tab.
  - ncann 5y ago
    onunload/onbeforeunload doesn't work?
    - jen20 5y ago
      What if they kill -9 the browser process?
    - inglor 5y ago
      Not always, not necessarily if the tab was in the background when closed (switch then close), not if they force quit (like the other comment I upvoted says), not if chrome updates etc. The other alternatives are opening another type of connection like an RTCPeerConnection to your server and seeing when that closes.
- deleted 5y ago
  [deleted]
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
Ekaros 5y ago
Just thinking here. If you did not have SPA, couldn't you just I don't know log when they request those pages? Like, aren't those done with HTTP request anyway... So why the hell you need to do the logging on client side... Then again that would be fun to send enough nice bogus log information... As clearly that is what they want.
- xg15 5y ago
  This works as long as the user is navigating around inside your site - but it won't work if the user e.g. navigates to a different domain or closes the browser window. If you specifically want to track when a user leaves a page, you will miss a lot of events this way.
  - maccard 5y ago
    What's the difference between leaving a page and leaving the tab hanging open in the background for a week?
    - xg15 5y ago
      One is done more often than the other. Of course there is no way to perfectly, reliably capture 100% of the events. Even if browsers had a dedicated API for that, a user could simply kill the process or pull the plug. However, you can make sure you capture the vast majority of events. Clicking on a link or closing a tab are much more likely events than killing the browser or leaving a tab open for weeks. (On desktop at least, might be different for mobile. But even there, you might want to track how many pages just stay open forever - which would also require you to track close events.)
dylan604 5y ago
From an honest analytics parsing question, how do you decipher 4 different iOS users on the same device model from a Starbucks or the VPN exit node sharing the same IP from access_log?
- hnarn 5y ago
  From an honest privacy advocate: what makes you feel you have the right to be able to? Why do you think end-users install things like "uBlock Origin", or why do smartphone manufacturers like Apple introduce features like random MAC address generation when connecting to wireless networks? Why do people use VPNs in the first place? Because data points are being collected without our consent to profile us, and most of us do not want that. The type of differentiation you are talking about by definition lacks consent, because if you had consent it would be trivial, you could do it by cookies or other forms of identification that the user consented to.
  - giantrobot 5y ago
    Look I'm just trying to find the best way to stealthily install cameras in every bedroom of someone's house. It's just so I can track their every movement and sell their intimate details to completely unrelated third parties. They're only going to compile all that data to psychologically abuse those people until they spend money! Is that so wrong‽
    - lsaferite 5y ago
      Hyperbole does nothing to genuinely further a discussion here.
  - dylan604 5y ago
    I'm a huge privacy advocate, so I'm not trying to do anything nefarious. However, I would like to know if my per visitor type numbers are accurate. As a small business, these numbers are useful even without having any desire of removing anonymity. There's a huge difference from wanting to tracking users and just know the correct number of people visiting. How can you honestly not think that knowing an accurate count of visitors is a useful metric?

> I’ve needed to send off an HTTP request with some data to log when a user does something like navigate to a different page or submit a form. No need for that. Stop hoarding data!

capableweb 5y ago

What's wrong with doing something like this? If it contains PII, then I agree, the additional hassle of dealing with everything that comes with handling PII (like GDPR) becomes too much, easier to just don't do it. But if it doesn't contain PII, it can be useful to see how many people drop off a form VS submitting it for example.

Moru 5y ago
Record how many opens the form and compare to how many actually submit the form?
- capableweb 5y ago
  Yeah, I'm not saying this is a silver-bullet that works in all cases, but some use cases do work better with it. For example, if said form shows one question at a time without any backend requests being done between questions, you could do one last try to log which question the user dropped off at. But again, no silver-bullet.
- vntok 5y ago
  How many users submitted the form vs how many users reached the form's page won't tell you anything about why they left or how long they struggled before submitting. A beacon sent for each click on links on the form's page will tell you how many users left for this or that other page, which is very useful to know and optimize by removing the link and/or bringing its contents to the page itself. A beacon sent for each form submission attempt will tell you how many users make mistakes while filling in the form, how good the error messages are, and whether the users tend to fix their input or leave. Bonus points for incrementing an "attempt" hidden field on each submit button click so that you can see how many attempts it takes for all fields to be filled correctly. Tracking what works and what doesn't can drive a lot of UX improvements, for the immediate and measurable benefit of the user.
fivea 5y ago
> If it contains PII (...) How can you tell?
- capableweb 5y ago
  How you can tell if the data you're collecting is PII? I guess there are many definitions of PII, but I generally base my own understanding on the GDPR definition: > ‘Personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person.
  - fivea 5y ago
    > How you can tell if the data you're collecting is PII? No. How can you tell, as a user of any random website, that the script your browser is running as part of that page you've opened isn't shipping PII collected from your usage.
    - mattlondon 5y ago
      You can tell by not entering any PII into the site in the first place. IP is technically PII in some places. Personally I am not worried if a criminal gets "my" IP in the same way that I am not worried if they have my phone number. I would be worried if they had my name address age bank account info etc, but then I don't give that out freely
      - fivea 5y ago
        > You can tell by not entering any PII into the site in the first place. That's not the definition of PII. At best, that covers a small subset of PII. PII means any information that can be used to identify you, either directly or indirectly. When you access a website combined with which link you opened or which search keywords you used can be used to infer who you are. How do you tell if a script is not shipping that info to an undisclosed third-party?
tantalor 5y ago
IP addr & cookies are PII
- capableweb 5y ago
  IP addresses, yes. Cookies not always, depends on what you store in them. So my point remains, nothing wrong with hoarding data if there is no PII.

soylentgraham 5y ago

Maybe it was to log a player off from a game, rather than have a dangling player. (Though websocket would work better)

deleted 5y ago
[deleted]
inglor 5y ago
This is true and in general websockets (well, and webtransport/webrtc etc) are the only reliable way to detect navigation from the backend since none of the methods mentioned in the article work if the user simply closed the tab.
- ncann 5y ago
  onunload/onbeforeunload doesn't work?
  - jen20 5y ago
    What if they kill -9 the browser process?
  - inglor 5y ago
    Not always, not necessarily if the tab was in the background when closed (switch then close), not if they force quit (like the other comment I upvoted says), not if chrome updates etc. The other alternatives are opening another type of connection like an RTCPeerConnection to your server and seeing when that closes.
deleted 5y ago
[deleted]

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

Ekaros 5y ago

Just thinking here. If you did not have SPA, couldn't you just I don't know log when they request those pages? Like, aren't those done with HTTP request anyway... So why the hell you need to do the logging on client side... Then again that would be fun to send enough nice bogus log information... As clearly that is what they want.

xg15 5y ago
This works as long as the user is navigating around inside your site - but it won't work if the user e.g. navigates to a different domain or closes the browser window. If you specifically want to track when a user leaves a page, you will miss a lot of events this way.
- maccard 5y ago
  What's the difference between leaving a page and leaving the tab hanging open in the background for a week?
  - xg15 5y ago
    One is done more often than the other. Of course there is no way to perfectly, reliably capture 100% of the events. Even if browsers had a dedicated API for that, a user could simply kill the process or pull the plug. However, you can make sure you capture the vast majority of events. Clicking on a link or closing a tab are much more likely events than killing the browser or leaving a tab open for weeks. (On desktop at least, might be different for mobile. But even there, you might want to track how many pages just stay open forever - which would also require you to track close events.)

dylan604 5y ago

From an honest analytics parsing question, how do you decipher 4 different iOS users on the same device model from a Starbucks or the VPN exit node sharing the same IP from access_log?

hnarn 5y ago
From an honest privacy advocate: what makes you feel you have the right to be able to? Why do you think end-users install things like "uBlock Origin", or why do smartphone manufacturers like Apple introduce features like random MAC address generation when connecting to wireless networks? Why do people use VPNs in the first place? Because data points are being collected without our consent to profile us, and most of us do not want that. The type of differentiation you are talking about by definition lacks consent, because if you had consent it would be trivial, you could do it by cookies or other forms of identification that the user consented to.
- giantrobot 5y ago
  Look I'm just trying to find the best way to stealthily install cameras in every bedroom of someone's house. It's just so I can track their every movement and sell their intimate details to completely unrelated third parties. They're only going to compile all that data to psychologically abuse those people until they spend money! Is that so wrong‽
  - lsaferite 5y ago
    Hyperbole does nothing to genuinely further a discussion here.
- dylan604 5y ago
  I'm a huge privacy advocate, so I'm not trying to do anything nefarious. However, I would like to know if my per visitor type numbers are accurate. As a small business, these numbers are useful even without having any desire of removing anonymity. There's a huge difference from wanting to tracking users and just know the correct number of people visiting. How can you honestly not think that knowing an accurate count of visitors is a useful metric?