11 ms·

NZ’s biggest data breach shows retention is the sleeping giant of data security

314 points by EdwardDiego 4y ago

steve_gh 4y ago
Define "need" In my job (infrastructure management) I am constantly fighting against people wanting to throw data away. "We know the condition of the bridge" they say - "so we don't need the past 20 years of maintenance records. It's too expensive to keep". What they lose is the ability to understand how things change over the long term - how bridge supports and bearings degrade, so they can take proactive steps in the future and replace things in a planned way, rather than waiting for them to break.* *Actually, what we do is a bit more complicated than this, but hopefully you get the point.
- clouddrover 4y ago
  How is the condition of the bridge a privacy concern?
  - segfaultbuserr 4y ago
    +1, I don't think this is relevant in this context. When we speak of treating data as toxic assets to minimize its collection and retention, it's about personal information with privacy concerns, not something like maintenance records.
- largepeepee 4y ago
  Maybe they don't want record because the data also holds incriminating evidence of how they understaffed the maintenance crew and skipped upkeep to fill their pockets in the short term.
- throwbadubadu 4y ago
  Please, doesn't help to pull that (not personally related data) in here, this is just not comparable. Such data can be kept forever, maybe should even be public because of public interest, and likely even publicly owned. Personal data / PII, history and also current cases again and again show should be minimized. Germany had it pretty right with its https://de.m.wikipedia.org/wiki/Datenvermeidung_und_Datensparsamkeit https://de.m.wikipedia.org/wiki/Datenvermeidung_und_Datenspa... though it gets attacked and holed out recently everyday again.. for the public interest they say, but for the profit interest of some corps they mean.. if not even for spying on citizens...
totetsu 4y ago
> However, if after people have worked with Latitude their privacy harms have not been resolved to their satisfaction, we encourage them to make a complaint to our Office. How does an organization that has leaked PI even resolve a 'privacy harm'. Once that information is out it's out. there's no taking it back.
- sacrosancty 4y ago
  [dead]
anenefan 4y ago
The biggest problem is how data is stored and used - it seems it got common place to have just one for all the customer dealings, rather than a number that can synch easily - but to combat data loss, keeping small database facing the web or other attack points exposing a limited amount of information removes much of the reward a lucky hacker might gain. For most uses of online and customers, it shouldn't need date of birth and their driver's licence number, just the necessities, like customer number and secure access ... maybe a hashed credit card number to confirm. Anything more than that generally requires people to contact support ... why then have a singular database that with the right exploit ... oops sorry about that.
- guidedlight 4y ago
  My understanding is that attack was through a third party that had access to Latitude’s network for legitimate business purposes. Counter terrorism legislation requires than financial services companies store customer identification. The issue was not adequately restricting the third party communications into Latitude’s network.
  - EdwardDiego 4y ago
    My PII was compromised. I've never had a Gem Visa card, but I have had hire purchases 10 - 15 years ago with Harvey Norman, who some time ago shifted from in-house credit to using Latitude, and obviously uploaded their entire customer DB. > Counter terrorism legislation requires than financial services companies store customer identification. I was very much not their customer. The third party was https://dxc.com https://dxc.com
  - anenefan 4y ago
    If there's a chink in the security, it's just a matter of time ... as such, there's a question that's not really being asked, does x or any third party need all the details. Sometimes they do because which ever company had found a cheap service to do all the account processing or some other task. But more often I don't think third parties don't need full access to the master database via web access ... if they do then surely the customer needs to be informed who the company's partner is, and what that company's policy is to guarding any personal data loss/ misuse / retention. From what I see lately (and there has been some massive data loss here in Australia in just the last year) there's a very care free lax attitude with a few shrugs after data is lost - with the hope naughty hackers can be blamed.
illiarian 4y ago
And then people complain that things like GDPR are not needed...
olliej 4y ago
This seems to be unforced data retention, which is leaked/stolen, as is pretty much a given. However the privacy commissioner is saying "data retention is a problem", and at the same time we have: Spy agencies: we need everyone to collect data to prevent terrorism! Politicians: We need to do that to protect children! Politicians: Also if you ever leak any of the data we forced you to collect and store data whether you want it or not, you will be liable.
- throwbadubadu 4y ago
  These, but also politicians: Its the new oil, we need it for the economy, stay competitive and throw all principles learned by blood away. If we would just start with all their data first for their in office part, as that is of real public interest again...
- rstuart4133 4y ago
  > This seems to be unforced data retention, I didn't get that impression. Drivers licences and passports aren't things banks generally care about. It sounds more like the "know you customer" and "anti money laundering" hoops all financial institutions are forced to jump through now. The rest of your comment rings true, although "spy agencies" is too narrow. It's spy agencies, police, the tax office, the social welfare agencies (like Centre Link in Australia), the financial agencies like Australia's ASIC. I don't think it's politically possible to stop the data collection. A few real terrorist plots in Australia have been caught with this data, banks have been fined billions for allowing what is effectively money washing. Regardless of what I personally think, the Law Enforcement Agencies will say not collecting this information is the equivalent of "let the terrorists/paedophiles win" making any argument to stop it a near impossible sell. But that doesn't matter. You can insist this data is collected without creating a wide open barn door like we have currently using cryptographic protocols. As an example, the government could you an app on your phone, lets say a car rental agency requires proof you have a valid drivers licence. You present the phone to an NFC reader, the phone says "XYZ Business has an authority to verify you have a current licence and it's expiry date", you click "hand it over", the phone provides the required data to the rental company. It's signed to prove to the rental company the the government has certified you have a licence expiring on some date. It also contains a lot of details about you the police can use to chase you down if you are in a road accident - but that's all encrypted, so no one can see it, including the car rental company. So it doesn't matter if it leaks. Barn door closed. Well sort of. The car rental is probably going to collect a lot of information about you anyway, like a name, contact phone and address. But that's because they want it, not because the government demands it. Hopefully those details won't be sufficient to pull off identity fraud. The government ID's they are collecting now definitely are ID fraud material. This way of handling sensitive data isn't novel. We already do something like this for credit card details. When a web site offers to remember your credit card, it's likely they've never seen it, let alone remember it. The only people who have seen it is the payment processor - Stripe maybe. What they are remembering is a token Stripe gives to them, which they can use to charge your card again. But the token reveals nothing about you. What's more it is useless to everyone bar them - Stripe won't accept it from anyone else.
jacquesm 4y ago
Companies were told for years that their data is an asset. Instead they should treat it like a toxic substance and a liability, to get rid of it as soon as they can. But it will be a long time before this view will become more prevalent.
- userbinator 4y ago
  I very much hope not, for the sake of future historians. Orwell put it best: "He who controls the past controls the future. He who controls the present controls the past."
  - mtrycz2 4y ago
    Did you really just compare private companies data retention to totalitarian censorship, and in a good way?
  - olliej 4y ago
    Indeed, for future historians it is critical that every company store the address, id, and private info of all their customers, for all time, regardless of risk. Without which we may forget that I lived on Paeroa St for two years while at uni, information that will be crucial in a discussion of the habits of early 2000s university students in New Zealand, and the impact that had on the fall of western civilization and the rise of our new fish-centric ethno-religion in the year 1200 AT (After Tilapia).
    - rickdeckard 4y ago
      Let's not forget that all this personal data should also enter public domain for historians of the world to process. How else will future generations be able to put your residence on Paeroa St in connection with the consumption of fish in that area during that time, to reveal the political stance of you and your offspring against the ruling party of the Tilapia...
    - pizzaknife 4y ago
      Hail and Bless Our Lord ./makes gill flapping gestures
deleted 4y ago
[deleted]
silisili 4y ago
On the topic of data retention, though subtly different, can anyone succinctly explain to me when 'DELETE FROM' became a 4 letter word? Everyone wants soft delete deleted_at instead. Everywhere. Many ORMs even force this behavior. Why? Why can't we just delete records anymore? I'm sure some have specific reasons for this, but I'm trying to find the broader reason that everyone has jumped on this bandwagon. Maybe I'm too YAGNI, but I prefer delete by default, unless justified. Not the inverse...
- zimpenfish 4y ago
  > Why? Why can't we just delete records anymore? I think there's a certain amount of "you must keep records pertaining to financial transactions for N years" going on combined with (more recently, I suppose) abuse/spam protection.
  - bell-cot 4y ago
    > "you must keep records pertaining to financial transactions..." Old-school solution - keep the records, in off-line hard drives in a fire-proof safe, with each one labeled "physically destroy on $Date".
    - NoZebra120vClip 4y ago
      Simple, easy! Amazing! Now, what do you do with standard backups such as the ones which mingle data which is 1 day, 1 year, and 5, 10, and 20 years old?
- mnsc 4y ago
  One broader reason could be "event sourcing" at a buzzword level. The allure of having a full history of the system/data at every point in time. Combine this with ai at the buzzword level. You could then use the full history of the world as input data to large ai models. But why?
  - onlypositive 4y ago
sivers 4y ago
New Zealanders, here's the link to freeze your credit: https://www.centrix.co.nz/my-credit-score/suppress-your-credit-file/ https://www.centrix.co.nz/my-credit-score/suppress-your-cred... Two years ago they made you submit a manual request then prove with a police file or something that you have real proof that you're in real danger of identity theft. Now it seems it's more automated.
- lostlogin 4y ago
  Are credit checks a thing in NZ? I’ve lived here all my life and have never knowingly had one run on me, and have no idea of my rating.
  - ghewgill 4y ago
    It's a thing. You can check your score for free at https://www.creditsimple.co.nz/ https://www.creditsimple.co.nz/
    - xupybd 4y ago
      It's worth seeing how much data is stored against you.
    - robocat 4y ago
      Very scary: I signed up and they know a lot of personal information such as my bank account numbers (presented completely without obfuscation), and my exact mortgage amounts, plus other information that shouldn’t be so easily available. No actual authentication to sign up - presumably you can do so if you know a few personal details and somebodies driver’s license number or passport details (neither of which are secure information). Thanks G
      - boredumb 4y ago
        I believe it was nerd wallet, but I signed in a few years back and was surprised to see they had almost a decade of financial data on me. I don't recall them printing out my unobsfuscated account numbers but it was a bit unnerving regardless.
Matsta 4y ago
Just to give some context, I got the email letting me know my data was part of the breach. I bought an iMac on interest free finance direct from Apple in 2014 via Gem (which is now lattitude). So I imagine everyone who has bought a Apple product on finance is going to be caught up in this. Also looked at the Aussie Apple site and they still use Lattitude for finance payments.
Prunkton 4y ago
Data retention is one of the pillars of the GDPR. You can be charged for storing more data than is actually needed. For regulated entities like banks, it is important to have processes in place and demonstrate them during your annual audit. In case of non-compliance, you may face fines or even lose your license temporarily until the issue is fixed.
lastdong 4y ago
And that’s why GDPR is so important in Europe. There’s education needed, both for companies and users. Well informed people will better demand their rights, and hopefully change perception at companies.
WheatMillington 4y ago
This is an enraging response from our sleepy regulator. Maybe if the Privacy commissioner did their damn job...
aunty_helen 4y ago
> which isn’t okay. This is about the bigget teeth the privacy commissioner has. Suprised to see such a heavy handed response from them given it was only 20% of the population. Really, the NZ privacy commissioner's office is a joke. You can see this when they say things like: > The Office of the Privacy Commissioner also encourages individuals to challenge hard why an agency needs to collect and retain their personal information. I've talked to people who work in govt that have had full cctv surveillance at city wide (Hamilton) level Ok'd by them. Like no worries mate, they're in public so you can use casino face detection technology on them to track wherever anyone within the inner city is detected for 40 days of collection capacity.
- mr_toad 4y ago
  > Really, the NZ privacy commissioner's office is a joke. Well, the privacy act has no teeth, so all they can do is put their hand in their pocket and pretend they have a pistol. The act is about 30 years old now, it’s totally unsuited to the modern age and it desperately needs an overhaul.
  - jcz_nz 4y ago
    No, it's barely two years old now: https://www.legislation.govt.nz/act/public/2020/0031/latest/LMS23223.html https://www.legislation.govt.nz/act/public/2020/0031/latest/...
- heresie-dabord 4y ago
  > drivers’ licences [...] as well as people’s passports. Some of the 14 million New Zealand and Australia records taken are up to 18 years old, which isn’t okay. A question about NZ English... How is that last clause (the final five syllables) used/interpreted? To my ear, it sounds comical. "Our office wishes to formally advise millions of citizens that your trust has once again been violated, and in our official role we declare this to be a not okay event."
  - TheReveller 4y ago
    It means 'which isn't acceptable'. I am also confused as to why they used this turn of phrase, perhaps to sound more colloquial. It sounds weird to me too, and comical, coming from a govt department, but perhaps it is indicating that the privacy commission finds that to be something the population would not approve of, but which it has no actual enforcement mechanism.
cudgy 4y ago
“A key finding from the NZ Institute of Directors’ Director Sentiment Survey report, released late last year, was that a significant proportion of boards were not sufficiently prepared for a digital future and had an “it won’t happen to us” approach. The message from the Office of the Privacy Commissioner is “wake up to yourselves”. We talk to organisations almost every week who are counting the cost of a cyber data breach. Can you risk the impact to your customers and your reputation?” Agencies should not be collecting or retaining personal information unless it is necessary for a lawful purpose connected with their function or activity. All agencies should have a personal information retention schedule that they review regularly. The simple discipline of deciding how long information will be retained as you collect it and acting on these decisions will save you and your customers a lot of pain.” So many panels, surveys, commissions, and watchdog groups state the obvious and yet little action on those recommendations is ever taken. In fact, many times the actions that are taken exacerbate the problem. The recently proposed bills to try and deal with TikTok and the social media data collection practices is a great example of this: propose sweeping new laws that violate privacy and free speech in order to combat privacy issues.
boredumb 4y ago
It's almost amusing to watch how companies and consultants approach personally identifiable information in 2023 compared to just a few years earlier. I've been paid a lot of money to be able to get peoples data to essentially be able to self destruct and make sure before it does that it's transported and stored securely. If I was to walk into an ad tech space in 2016 and tell people about my new found talent of scheduling data scrambling jobs against PII and rambling on about migrating all user data to be encrypted at rest they would have called the police.
- fuzzfactor 4y ago
  This is about what I was predicting when AOL got popular and Amazon.com got launched.
- jonathankoren 4y ago
  I always find it amusing that before PII got taken halfway seriously, basic protections were assumed to cause massive problems for the business, and yet life went on just fine. See also: GDPR transition
washywashy 4y ago
Think about every little tiny mom-pop medical office that either runs their own IT or pays a medical-industry-centered consultancy that usually has an IT branch. I think you’d be surprised to see what is out there and what is being used.
photochemsyn 4y ago
Snowden's "Permanent Record" is an interesting read, but at the end he seemed to view a transition from the government collecting and storing data on individuals to the private sector collecting and holding that data as some kind of positive event. It's really not any different, and neither entity is 100% secure against data theft and exposure. Corporations and governments have an obsession with tracking individuals, although probably for different reasons (profits and tax collection being the main goals, I imagine, with 'social credit score' as a newer more dystopian concept). This is well-documented in the US and no doubt applies to NZ as well: > "As we have documented at The Privacy Issue, pacts with U.S. intelligence are a mix of cooperation with, and infiltration by, three-letter agencies. Though nearly every major Big Tech company has cooperated with the NSA's surveillance programs in some way, the Google cloud was notably backdoored via clandestine means that were likely unknown to the company, with NSA agents gleefully bragging about the infiltration in an internal presentation." https://theprivacyissue.com/government-surveillance/battle-compelled-data-sharing https://theprivacyissue.com/government-surveillance/battle-c... New Zealand is a member of the so-called Five Eyes Collective so the NSA likely has full access to all personal data of New Zealanders, let alone everyone else in the region: https://www.theguardian.com/us-news/2015/mar/05/new-zealand-spying-on-pacific-allies-for-five-eyes-and-nsa-snowden-files-show https://www.theguardian.com/us-news/2015/mar/05/new-zealand-...
- logicalmonster 4y ago
  > he seemed to view a transition from the government collecting and storing data on individuals to the private sector collecting and holding that data as some kind of positive event. It's really not any different, Isn't it different? At worst, companies like Walmart, Google, and Amazon are creepy in using any information they can collect to try to sell you useless products. At worst, the government can use any data they collect about you to throw you in a metal cage for life, take your kids away from you, drone strike you, steal your money, inject you with drugs without your consent, execute you, etc. One is a mere annoyance. The other is a direct threat to your life, liberty, and property.
  - nabnob 4y ago
killjoywashere 4y ago
I've worked on DoD HPCMP systems and they would handle ITAR up to and including nuclear weapons simulations but wouldn't touch PII. But the real risk, I think, is the collection of every packet going over the wire. There are entities storing every packet in the hopes of eventually using quantum decryption to decrypt it all. The question has moved from "should it be secret" to "how long must it be secret?" because the longer you need to keep it secret, the more computationally expensive the encryption is going to be.
- jjav 4y ago
  > The question has moved from "should it be secret" to "how long must it be secret?" because the longer you need to keep it secret, the more computationally expensive the encryption is going to be. To be fair, encryption strenght has always been rated on the basis of how long it should stay secret. For example a site like https://www.keylength.com/en/compare/ https://www.keylength.com/en/compare/ (which has been around a very long time) guides you to pick key strenght based on how many years you'd want the data to be protected.
xp84 4y ago
Interesting to see how this plays out in a different jurisdiction to mine. Here in the US, it seems like as long as penalties remain very light, PII retention and security is not taken very seriously. Equifax's breach in 2017 was essentially every valuable piece of PII they had, on more than half the households in America, and they settled for $300 million in a civil suit. Their net income (after all expenses) every year is in the $500-700 million range. They should have been fined directly, an amount intentionally set to force complete liquidation, shareholders should have been completely wiped out, and all proceeds from the liquidation after court costs should have been distributed to every victim of the breach. If that was the case, 100% of US companies would now treat PII with the respect it deserves. As it stands, nah, it's nbd here.
shadowgovt 4y ago
I received a letter from a movie theater I worked for as a college student for a summer job, some fifteen years after I graduated college. They let me know a data breach had occurred exposing my social security number and (out-of-date, but still relevant for answering security questions) PII including address and telephone number. The breach vector? Legally-mandated tax records. Someone cracked the padlock on the warehouse they were stored in and made off with the hard copies. It is entirely possible that this sort of data exposure is, in the limit, impossible to guard against. Perhaps the better approach is to ask whether there's any way to make that data less valuable.
tastysandwich 4y ago
I'm getting really annoyed by this one-sided trust with companies. We're just expected to hand over all our sensitive documents, with no idea how it's stored or who can access. I was sent a shipment recently and the sender didn't put my last name. So the courier asked me to email them a copy of my passport or driver's licence. I rang them directly and was told that this is the only way of providing my identity. I said hell no and the package was sent back (they ended up resending it with my full name). I find the worst culprit to be healthcare providers. I've had two recently ask me to send my credit card information in the regular mail. I drove down to one of them to do it in person, and they put my details in a filing cabinet. I sure hope that lock is good! (Pro tip: if you ever need to send CC details via mail and you cannot arrange an alternative, use registered post with signature so you can confirm it made it to the recipient)
avastmick 4y ago
I led the technical design team for RealMe, New Zealand’s digital identity service. Sadly, the service is woefully undervalued and poorly used. The technology is now out dated. It was built to directly mitigate events like the Latitude breach whereby the service gives an answer to an identity question rather than spraying PII across the economy. Answers were formed by pulling data from disparate authoritative sources in real time, a set of tokens were created and an audit record created and shared with PII owner consent. No personal information was ever intended to be shared or stored. It was an elegant solution for New Zealand, though we were mindful of a potential scaling issues in larger jurisdictions. The financial sector was the initial target to help with AML/KYC flows. The banks in particular lobbied for access to the PII rather than an answer to the question so the service was devalued from the get go. If we’d won that answer I believe that digital identity and personal information sharing would be very different today.
- karateka 4y ago
  I must say I always liked being able to use RealMe as kind of a single sign on for multiple sites, it’s a shame it never got more widespread
  - jcz_nz 4y ago
    It was never approved for broader use outside of govt, and the DIA insists on stupid-high "cost recovery", making it uneconomical. Forest and trees, and lack of actual political leadership.
- kiwidrew 4y ago
  I miss the days when NZ was truly innovative. RealMe was very promising when first introduced. Of course it was never going to be adopted en masse by the private sector since part of the "get approval to use RealMe on your website" was "get Parliament to pass an Order in Council adding you to the authorized users schedule".

314 points by EdwardDiego 4y ago

Define "need" In my job (infrastructure management) I am constantly fighting against people wanting to throw data away. "We know the condition of the bridge" they say - "so we don't need the past 20 years of maintenance records. It's too expensive to keep". What they lose is the ability to understand how things change over the long term - how bridge supports and bearings degrade, so they can take proactive steps in the future and replace things in a planned way, rather than waiting for them to break.* *Actually, what we do is a bit more complicated than this, but hopefully you get the point.

clouddrover 4y ago
How is the condition of the bridge a privacy concern?
- segfaultbuserr 4y ago
  +1, I don't think this is relevant in this context. When we speak of treating data as toxic assets to minimize its collection and retention, it's about personal information with privacy concerns, not something like maintenance records.
largepeepee 4y ago
Maybe they don't want record because the data also holds incriminating evidence of how they understaffed the maintenance crew and skipped upkeep to fill their pockets in the short term.
throwbadubadu 4y ago
Please, doesn't help to pull that (not personally related data) in here, this is just not comparable. Such data can be kept forever, maybe should even be public because of public interest, and likely even publicly owned. Personal data / PII, history and also current cases again and again show should be minimized. Germany had it pretty right with its https://de.m.wikipedia.org/wiki/Datenvermeidung_und_Datensparsamkeit https://de.m.wikipedia.org/wiki/Datenvermeidung_und_Datenspa... though it gets attacked and holed out recently everyday again.. for the public interest they say, but for the profit interest of some corps they mean.. if not even for spying on citizens...

totetsu 4y ago

> However, if after people have worked with Latitude their privacy harms have not been resolved to their satisfaction, we encourage them to make a complaint to our Office. How does an organization that has leaked PI even resolve a 'privacy harm'. Once that information is out it's out. there's no taking it back.

sacrosancty 4y ago
[dead]

anenefan 4y ago

The biggest problem is how data is stored and used - it seems it got common place to have just one for all the customer dealings, rather than a number that can synch easily - but to combat data loss, keeping small database facing the web or other attack points exposing a limited amount of information removes much of the reward a lucky hacker might gain. For most uses of online and customers, it shouldn't need date of birth and their driver's licence number, just the necessities, like customer number and secure access ... maybe a hashed credit card number to confirm. Anything more than that generally requires people to contact support ... why then have a singular database that with the right exploit ... oops sorry about that.

guidedlight 4y ago
My understanding is that attack was through a third party that had access to Latitude’s network for legitimate business purposes. Counter terrorism legislation requires than financial services companies store customer identification. The issue was not adequately restricting the third party communications into Latitude’s network.
- EdwardDiego 4y ago
  My PII was compromised. I've never had a Gem Visa card, but I have had hire purchases 10 - 15 years ago with Harvey Norman, who some time ago shifted from in-house credit to using Latitude, and obviously uploaded their entire customer DB. > Counter terrorism legislation requires than financial services companies store customer identification. I was very much not their customer. The third party was https://dxc.com https://dxc.com
- anenefan 4y ago
  If there's a chink in the security, it's just a matter of time ... as such, there's a question that's not really being asked, does x or any third party need all the details. Sometimes they do because which ever company had found a cheap service to do all the account processing or some other task. But more often I don't think third parties don't need full access to the master database via web access ... if they do then surely the customer needs to be informed who the company's partner is, and what that company's policy is to guarding any personal data loss/ misuse / retention. From what I see lately (and there has been some massive data loss here in Australia in just the last year) there's a very care free lax attitude with a few shrugs after data is lost - with the hope naughty hackers can be blamed.

illiarian 4y ago

And then people complain that things like GDPR are not needed...

olliej 4y ago

This seems to be unforced data retention, which is leaked/stolen, as is pretty much a given. However the privacy commissioner is saying "data retention is a problem", and at the same time we have: Spy agencies: we need everyone to collect data to prevent terrorism! Politicians: We need to do that to protect children! Politicians: Also if you ever leak any of the data we forced you to collect and store data whether you want it or not, you will be liable.

throwbadubadu 4y ago
These, but also politicians: Its the new oil, we need it for the economy, stay competitive and throw all principles learned by blood away. If we would just start with all their data first for their in office part, as that is of real public interest again...
rstuart4133 4y ago
> This seems to be unforced data retention, I didn't get that impression. Drivers licences and passports aren't things banks generally care about. It sounds more like the "know you customer" and "anti money laundering" hoops all financial institutions are forced to jump through now. The rest of your comment rings true, although "spy agencies" is too narrow. It's spy agencies, police, the tax office, the social welfare agencies (like Centre Link in Australia), the financial agencies like Australia's ASIC. I don't think it's politically possible to stop the data collection. A few real terrorist plots in Australia have been caught with this data, banks have been fined billions for allowing what is effectively money washing. Regardless of what I personally think, the Law Enforcement Agencies will say not collecting this information is the equivalent of "let the terrorists/paedophiles win" making any argument to stop it a near impossible sell. But that doesn't matter. You can insist this data is collected without creating a wide open barn door like we have currently using cryptographic protocols. As an example, the government could you an app on your phone, lets say a car rental agency requires proof you have a valid drivers licence. You present the phone to an NFC reader, the phone says "XYZ Business has an authority to verify you have a current licence and it's expiry date", you click "hand it over", the phone provides the required data to the rental company. It's signed to prove to the rental company the the government has certified you have a licence expiring on some date. It also contains a lot of details about you the police can use to chase you down if you are in a road accident - but that's all encrypted, so no one can see it, including the car rental company. So it doesn't matter if it leaks. Barn door closed. Well sort of. The car rental is probably going to collect a lot of information about you anyway, like a name, contact phone and address. But that's because they want it, not because the government demands it. Hopefully those details won't be sufficient to pull off identity fraud. The government ID's they are collecting now definitely are ID fraud material. This way of handling sensitive data isn't novel. We already do something like this for credit card details. When a web site offers to remember your credit card, it's likely they've never seen it, let alone remember it. The only people who have seen it is the payment processor - Stripe maybe. What they are remembering is a token Stripe gives to them, which they can use to charge your card again. But the token reveals nothing about you. What's more it is useless to everyone bar them - Stripe won't accept it from anyone else.

jacquesm 4y ago

Companies were told for years that their data is an asset. Instead they should treat it like a toxic substance and a liability, to get rid of it as soon as they can. But it will be a long time before this view will become more prevalent.

userbinator 4y ago
I very much hope not, for the sake of future historians. Orwell put it best: "He who controls the past controls the future. He who controls the present controls the past."
- mtrycz2 4y ago
  Did you really just compare private companies data retention to totalitarian censorship, and in a good way?
- olliej 4y ago
  Indeed, for future historians it is critical that every company store the address, id, and private info of all their customers, for all time, regardless of risk. Without which we may forget that I lived on Paeroa St for two years while at uni, information that will be crucial in a discussion of the habits of early 2000s university students in New Zealand, and the impact that had on the fall of western civilization and the rise of our new fish-centric ethno-religion in the year 1200 AT (After Tilapia).
  - rickdeckard 4y ago
    Let's not forget that all this personal data should also enter public domain for historians of the world to process. How else will future generations be able to put your residence on Paeroa St in connection with the consumption of fish in that area during that time, to reveal the political stance of you and your offspring against the ruling party of the Tilapia...
  - pizzaknife 4y ago
    Hail and Bless Our Lord ./makes gill flapping gestures

deleted 4y ago

[deleted]

silisili 4y ago

On the topic of data retention, though subtly different, can anyone succinctly explain to me when 'DELETE FROM' became a 4 letter word? Everyone wants soft delete deleted_at instead. Everywhere. Many ORMs even force this behavior. Why? Why can't we just delete records anymore? I'm sure some have specific reasons for this, but I'm trying to find the broader reason that everyone has jumped on this bandwagon. Maybe I'm too YAGNI, but I prefer delete by default, unless justified. Not the inverse...

zimpenfish 4y ago
> Why? Why can't we just delete records anymore? I think there's a certain amount of "you must keep records pertaining to financial transactions for N years" going on combined with (more recently, I suppose) abuse/spam protection.
- bell-cot 4y ago
  > "you must keep records pertaining to financial transactions..." Old-school solution - keep the records, in off-line hard drives in a fire-proof safe, with each one labeled "physically destroy on $Date".
  - NoZebra120vClip 4y ago
    Simple, easy! Amazing! Now, what do you do with standard backups such as the ones which mingle data which is 1 day, 1 year, and 5, 10, and 20 years old?
mnsc 4y ago
One broader reason could be "event sourcing" at a buzzword level. The allure of having a full history of the system/data at every point in time. Combine this with ai at the buzzword level. You could then use the full history of the world as input data to large ai models. But why?
- onlypositive 4y ago

sivers 4y ago

New Zealanders, here's the link to freeze your credit: https://www.centrix.co.nz/my-credit-score/suppress-your-credit-file/ https://www.centrix.co.nz/my-credit-score/suppress-your-cred... Two years ago they made you submit a manual request then prove with a police file or something that you have real proof that you're in real danger of identity theft. Now it seems it's more automated.

lostlogin 4y ago
Are credit checks a thing in NZ? I’ve lived here all my life and have never knowingly had one run on me, and have no idea of my rating.
- ghewgill 4y ago
  It's a thing. You can check your score for free at https://www.creditsimple.co.nz/ https://www.creditsimple.co.nz/
  - xupybd 4y ago
    It's worth seeing how much data is stored against you.
  - robocat 4y ago
    Very scary: I signed up and they know a lot of personal information such as my bank account numbers (presented completely without obfuscation), and my exact mortgage amounts, plus other information that shouldn’t be so easily available. No actual authentication to sign up - presumably you can do so if you know a few personal details and somebodies driver’s license number or passport details (neither of which are secure information). Thanks G
    - boredumb 4y ago
      I believe it was nerd wallet, but I signed in a few years back and was surprised to see they had almost a decade of financial data on me. I don't recall them printing out my unobsfuscated account numbers but it was a bit unnerving regardless.

Matsta 4y ago

Just to give some context, I got the email letting me know my data was part of the breach. I bought an iMac on interest free finance direct from Apple in 2014 via Gem (which is now lattitude). So I imagine everyone who has bought a Apple product on finance is going to be caught up in this. Also looked at the Aussie Apple site and they still use Lattitude for finance payments.

Prunkton 4y ago

Data retention is one of the pillars of the GDPR. You can be charged for storing more data than is actually needed. For regulated entities like banks, it is important to have processes in place and demonstrate them during your annual audit. In case of non-compliance, you may face fines or even lose your license temporarily until the issue is fixed.

lastdong 4y ago

And that’s why GDPR is so important in Europe. There’s education needed, both for companies and users. Well informed people will better demand their rights, and hopefully change perception at companies.

WheatMillington 4y ago

This is an enraging response from our sleepy regulator. Maybe if the Privacy commissioner did their damn job...

aunty_helen 4y ago

> which isn’t okay. This is about the bigget teeth the privacy commissioner has. Suprised to see such a heavy handed response from them given it was only 20% of the population. Really, the NZ privacy commissioner's office is a joke. You can see this when they say things like: > The Office of the Privacy Commissioner also encourages individuals to challenge hard why an agency needs to collect and retain their personal information. I've talked to people who work in govt that have had full cctv surveillance at city wide (Hamilton) level Ok'd by them. Like no worries mate, they're in public so you can use casino face detection technology on them to track wherever anyone within the inner city is detected for 40 days of collection capacity.

mr_toad 4y ago
> Really, the NZ privacy commissioner's office is a joke. Well, the privacy act has no teeth, so all they can do is put their hand in their pocket and pretend they have a pistol. The act is about 30 years old now, it’s totally unsuited to the modern age and it desperately needs an overhaul.
- jcz_nz 4y ago
  No, it's barely two years old now: https://www.legislation.govt.nz/act/public/2020/0031/latest/LMS23223.html https://www.legislation.govt.nz/act/public/2020/0031/latest/...
heresie-dabord 4y ago
> drivers’ licences [...] as well as people’s passports. Some of the 14 million New Zealand and Australia records taken are up to 18 years old, which isn’t okay. A question about NZ English... How is that last clause (the final five syllables) used/interpreted? To my ear, it sounds comical. "Our office wishes to formally advise millions of citizens that your trust has once again been violated, and in our official role we declare this to be a not okay event."
- TheReveller 4y ago
  It means 'which isn't acceptable'. I am also confused as to why they used this turn of phrase, perhaps to sound more colloquial. It sounds weird to me too, and comical, coming from a govt department, but perhaps it is indicating that the privacy commission finds that to be something the population would not approve of, but which it has no actual enforcement mechanism.

cudgy 4y ago

“A key finding from the NZ Institute of Directors’ Director Sentiment Survey report, released late last year, was that a significant proportion of boards were not sufficiently prepared for a digital future and had an “it won’t happen to us” approach. The message from the Office of the Privacy Commissioner is “wake up to yourselves”. We talk to organisations almost every week who are counting the cost of a cyber data breach. Can you risk the impact to your customers and your reputation?” Agencies should not be collecting or retaining personal information unless it is necessary for a lawful purpose connected with their function or activity. All agencies should have a personal information retention schedule that they review regularly. The simple discipline of deciding how long information will be retained as you collect it and acting on these decisions will save you and your customers a lot of pain.” So many panels, surveys, commissions, and watchdog groups state the obvious and yet little action on those recommendations is ever taken. In fact, many times the actions that are taken exacerbate the problem. The recently proposed bills to try and deal with TikTok and the social media data collection practices is a great example of this: propose sweeping new laws that violate privacy and free speech in order to combat privacy issues.

boredumb 4y ago

It's almost amusing to watch how companies and consultants approach personally identifiable information in 2023 compared to just a few years earlier. I've been paid a lot of money to be able to get peoples data to essentially be able to self destruct and make sure before it does that it's transported and stored securely. If I was to walk into an ad tech space in 2016 and tell people about my new found talent of scheduling data scrambling jobs against PII and rambling on about migrating all user data to be encrypted at rest they would have called the police.

fuzzfactor 4y ago
This is about what I was predicting when AOL got popular and Amazon.com got launched.
jonathankoren 4y ago
I always find it amusing that before PII got taken halfway seriously, basic protections were assumed to cause massive problems for the business, and yet life went on just fine. See also: GDPR transition

washywashy 4y ago

Think about every little tiny mom-pop medical office that either runs their own IT or pays a medical-industry-centered consultancy that usually has an IT branch. I think you’d be surprised to see what is out there and what is being used.

photochemsyn 4y ago

Snowden's "Permanent Record" is an interesting read, but at the end he seemed to view a transition from the government collecting and storing data on individuals to the private sector collecting and holding that data as some kind of positive event. It's really not any different, and neither entity is 100% secure against data theft and exposure. Corporations and governments have an obsession with tracking individuals, although probably for different reasons (profits and tax collection being the main goals, I imagine, with 'social credit score' as a newer more dystopian concept). This is well-documented in the US and no doubt applies to NZ as well: > "As we have documented at The Privacy Issue, pacts with U.S. intelligence are a mix of cooperation with, and infiltration by, three-letter agencies. Though nearly every major Big Tech company has cooperated with the NSA's surveillance programs in some way, the Google cloud was notably backdoored via clandestine means that were likely unknown to the company, with NSA agents gleefully bragging about the infiltration in an internal presentation." https://theprivacyissue.com/government-surveillance/battle-compelled-data-sharing https://theprivacyissue.com/government-surveillance/battle-c... New Zealand is a member of the so-called Five Eyes Collective so the NSA likely has full access to all personal data of New Zealanders, let alone everyone else in the region: https://www.theguardian.com/us-news/2015/mar/05/new-zealand-spying-on-pacific-allies-for-five-eyes-and-nsa-snowden-files-show https://www.theguardian.com/us-news/2015/mar/05/new-zealand-...

logicalmonster 4y ago
> he seemed to view a transition from the government collecting and storing data on individuals to the private sector collecting and holding that data as some kind of positive event. It's really not any different, Isn't it different? At worst, companies like Walmart, Google, and Amazon are creepy in using any information they can collect to try to sell you useless products. At worst, the government can use any data they collect about you to throw you in a metal cage for life, take your kids away from you, drone strike you, steal your money, inject you with drugs without your consent, execute you, etc. One is a mere annoyance. The other is a direct threat to your life, liberty, and property.
- nabnob 4y ago

killjoywashere 4y ago

I've worked on DoD HPCMP systems and they would handle ITAR up to and including nuclear weapons simulations but wouldn't touch PII. But the real risk, I think, is the collection of every packet going over the wire. There are entities storing every packet in the hopes of eventually using quantum decryption to decrypt it all. The question has moved from "should it be secret" to "how long must it be secret?" because the longer you need to keep it secret, the more computationally expensive the encryption is going to be.

jjav 4y ago
> The question has moved from "should it be secret" to "how long must it be secret?" because the longer you need to keep it secret, the more computationally expensive the encryption is going to be. To be fair, encryption strenght has always been rated on the basis of how long it should stay secret. For example a site like https://www.keylength.com/en/compare/ https://www.keylength.com/en/compare/ (which has been around a very long time) guides you to pick key strenght based on how many years you'd want the data to be protected.

xp84 4y ago

Interesting to see how this plays out in a different jurisdiction to mine. Here in the US, it seems like as long as penalties remain very light, PII retention and security is not taken very seriously. Equifax's breach in 2017 was essentially every valuable piece of PII they had, on more than half the households in America, and they settled for $300 million in a civil suit. Their net income (after all expenses) every year is in the $500-700 million range. They should have been fined directly, an amount intentionally set to force complete liquidation, shareholders should have been completely wiped out, and all proceeds from the liquidation after court costs should have been distributed to every victim of the breach. If that was the case, 100% of US companies would now treat PII with the respect it deserves. As it stands, nah, it's nbd here.

shadowgovt 4y ago

I received a letter from a movie theater I worked for as a college student for a summer job, some fifteen years after I graduated college. They let me know a data breach had occurred exposing my social security number and (out-of-date, but still relevant for answering security questions) PII including address and telephone number. The breach vector? Legally-mandated tax records. Someone cracked the padlock on the warehouse they were stored in and made off with the hard copies. It is entirely possible that this sort of data exposure is, in the limit, impossible to guard against. Perhaps the better approach is to ask whether there's any way to make that data less valuable.

tastysandwich 4y ago

I'm getting really annoyed by this one-sided trust with companies. We're just expected to hand over all our sensitive documents, with no idea how it's stored or who can access. I was sent a shipment recently and the sender didn't put my last name. So the courier asked me to email them a copy of my passport or driver's licence. I rang them directly and was told that this is the only way of providing my identity. I said hell no and the package was sent back (they ended up resending it with my full name). I find the worst culprit to be healthcare providers. I've had two recently ask me to send my credit card information in the regular mail. I drove down to one of them to do it in person, and they put my details in a filing cabinet. I sure hope that lock is good! (Pro tip: if you ever need to send CC details via mail and you cannot arrange an alternative, use registered post with signature so you can confirm it made it to the recipient)

avastmick 4y ago

I led the technical design team for RealMe, New Zealand’s digital identity service. Sadly, the service is woefully undervalued and poorly used. The technology is now out dated. It was built to directly mitigate events like the Latitude breach whereby the service gives an answer to an identity question rather than spraying PII across the economy. Answers were formed by pulling data from disparate authoritative sources in real time, a set of tokens were created and an audit record created and shared with PII owner consent. No personal information was ever intended to be shared or stored. It was an elegant solution for New Zealand, though we were mindful of a potential scaling issues in larger jurisdictions. The financial sector was the initial target to help with AML/KYC flows. The banks in particular lobbied for access to the PII rather than an answer to the question so the service was devalued from the get go. If we’d won that answer I believe that digital identity and personal information sharing would be very different today.

karateka 4y ago
I must say I always liked being able to use RealMe as kind of a single sign on for multiple sites, it’s a shame it never got more widespread
- jcz_nz 4y ago
  It was never approved for broader use outside of govt, and the DIA insists on stupid-high "cost recovery", making it uneconomical. Forest and trees, and lack of actual political leadership.
kiwidrew 4y ago
I miss the days when NZ was truly innovative. RealMe was very promising when first introduced. Of course it was never going to be adopted en masse by the private sector since part of the "get approval to use RealMe on your website" was "get Parliament to pass an Order in Council adding you to the authorized users schedule".