8 ms·

Technical issue resolved after ‘150k police records lost’

83 points by edward 6y ago

silasdavis 6y ago
Or more accurately, accidentally deleted. 'losing' these records would actually be a lot worse. I wouldn't underestimate the police's ability suffer a breach and lose the records either. It's worth remembering many of these records are are kept on the basis of non conviction (though there is a time limit).
codeulike 6y ago
It comes after around 40,000 alerts relating to European criminals were removed from the same database, the PNC, following Britain's post-Brexit deal with the EU. Right so reading between the lines they needed to do some post-Brexit data cleaning and someone messed up the WHERE clause. But where are the backups? Surely accidentally deleted data should be fairly easy to recover from historical backups which they surely surely must have?
- randunel 6y ago
  Wouldn't such international data removal agreements also include backups? What's the point in removing PII from one of the databases, if it remains in their backups?
  - codeulike 6y ago
    You have a point but I have doubts that any organisation on the planet has the capability/willingness/resources to surgically remove data from historical backups. I think its more likely that no backups exist, or that, say, no backups exist past X days in the past. Or worse, that backups exist, but they have been found to be unrestorable.
  - codeulike 6y ago
    Under GDPR right-to-erasure you dont necessarily have to erase data from backups but must be clear about how long it will hang around. The key issue is to put the backup data ‘beyond use’, even if it cannot be immediately overwritten. You must ensure that you do not use the data within the backup for any other purpose, ie that the backup is simply held on your systems until it is replaced in line with an established schedule. https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/individual-rights/right-to-erasure/#ib5 https://ico.org.uk/for-organisations/guide-to-data-protectio...
  - tremon 6y ago
    You could handle such requests by tombstoning the user ID (in addition to deleting the data). By keeping the tombstone records in separate storage (it's usually only kB, just a list of hard-removed user IDs), you can guarantee (re-do) the erasure even after having to do a full database restore.
  - vidarh 6y ago
    Backup. Encrypt the backup. Run the deletion. Verify you haven't lost anything. Verify it again. Put the backup encryption key beyond use other than in the case of a suitably agreed emergency (e.g. in the case of an agreement with another party, hand them the key). Then after a suitable grace period, once you're sure everything is ok, you destroy the backup.
    - nightcracker 6y ago
      > Put the backup encryption key beyond use other than in the case of a suitably agreed emergency (e.g. in the case of an agreement with another party, hand them the key). Like search warrants? We know those never get abused. > Then after a suitable grace period, once you're sure everything is ok, you destroy the backup. But we need a search warrant to access the backup anyway so we don't really have to delete it. Best keep it, just in case.
      - vidarh 6y ago
        > Like search warrants? We know those never get abused. No, like catastrophic loss of data, hence why I outlined a process of a doing this temporarily in the case of agreement with a another party to put the data entirely beyond your own use unless the party you've agreed the deletion with agrees the situation is serious enough. > But we need a search warrant to access the backup anyway so we don't really have to delete it. Best keep it, just in case. If you don't trust the party to delete the data, then adding in a temporary period of retention makes no difference, because in that case they could just as well have ignored the demand for the initial deletion entirely.
trestenhortz 6y ago
You can make pretty much any sort of tech fuckup and be forgiven. The only two unforgivable sins are “no working backups” and being hacked. Even being hacked is kinda forgivable cause it’s so hard to avoid. No working backups is totally unforgivable. Any tech person who walks into any tech job at any level up from junior to CTO has a duty as one of their first tasks to find out if working backups are being done of their work.
- franze 6y ago
  yeah, long time ago I synced the CRM database of the austria press agency to my sony clie https://en.wikipedia.org/wiki/CLI%C3%89 https://en.wikipedia.org/wiki/CLI%C3%89 and then my sony clie synced it back. completely deleting all data other then the email and name on the CRM server. gladly there was a backup, but the backup could not get used to get the database back-up. the discussion with IT were fabolous: "yes, we have it all backed up, so the backup is there, but we cant use it to bring the CRM back-up. but its backed up and we even have backups from the backup. they can also not get used to bringt it back up." since then it is my standard procedure to make sure that backups can actually be used to bring projects back up. usually the first test "use backup to bring stuff back up" fails.
  - KMag 6y ago
    If you haven't tested your backups, you don't really have backups. If you aren't allowed to verify the backups yourself, find a safe "mistake" to make to force the responsible parties to run through restoring from backups. Bonus points for using the backup testing exercise as an excuse to convince your manager to let you do a bit of refactoring to remove some particularly ugly module you've been itching to get rid of. Factor out the ugly module, then "accidentally" delete it. My fist job out of school was writing software models of pre-prototype communications equipment for a defense contractor. There was lots of government red tape, and crazy prices being charged everywhere. We couldn't use CVS or SVN (this predated git) to store our code, because they weren't government-approved tools. We could use PVCS to store source code, but only for the build artefacts and corresponding source code of official production versions delivered to the government. (And only people that had taken a multi-hour course to be certified on PVCS could touch it. Every release involved a couple government employees coming out to the defense contractor and physically watching the one guy on the team check in the release to PVCS and burn a pair of DVDs for the government. So, we were forced to do source control by "last one out on Friday has to zip up the entire shared drive and name it with the date". But, at least we were required to pay some crazy price to have the sysadmins backup the shared network drive to tape every week (or was it every day?). Once a month, the admins would take the current tape and send it via courier to a bomb-proof, fire-proof bunker out in the mountains in Pennsylvania or some such. They'd then put the spare tape in the tape drive to start making backups. As soon as the bunker got the latest tape, they'd send back the oldest tape via courier to become the spare tape. So, we always had at least one tape physically present in the bunker. It required 3 tapes to always have at least one tape in the bunker and one tape in the drive. I was less than a year out of school, but based on my interactions with the system admins, I figured there was at most a 50/50 chance they got the backups correct. I asked them once how the backups were going, and got back a very low key, "yea, yea, don't worry about it, we got it" answer. I convinced my boss to let us fake a source code loss: zip up two copies of the network drive, and store the zip files on two different systems. Check the integrity of both zip files, and then "accidentally" delete a directory containing some source we were in the midst of removing anyway. The sysadmins were unable to get our "accidentally" deleted source code back for us. They had neglected to erase the spare tape when putting it into the drive, so they were always only appending to our backups. They just ignored the errors when the tape got full.
deleted 6y ago
[deleted]
bb123 6y ago
A more accurate headline would be '150k police records accidentally destroyed'. There is a huge difference between accidental deletion and leaving a hard drive full of data on the tube. I'd give the BBC the benefit of the doubt that the unclear wording is accidental and not for click bait purposes, but I feel like I'm doing that more and more for them at the moment.
justinclift 6y ago
Hang on. This sounds like the fingerprint and DNA data for people arrested and never charged. But which the police insisted on hanging on to anyway "just in case". Maybe it's to public benefit for it having been lost?
- J-dawg 6y ago
  The UK took a wrong turn when it gave the police their own extra-judicial justice system. The system of police cautions, warnings and fixed penalties allow the police to administer punishment with none of the due process that a court would provide. The idea of a "police caution" is a misnomer that implies it's just harmless warning, when in fact it can affect the course of someone's life many years later. Furthermore, people are blackmailed into accepting these cautions under the threat of going to trial and potentially receiving a more severe punishment after conviction. Even more outrageous is the retention of records from the PREVENT programme and "non-crime hate incidents". These are people who have not even committed a crime or been arrested. The whole system stinks, and given that most people manage to avoid interaction with the police it is sadly unlikely to get the scrutiny it deserves. The outrageous behaviour of (some of) the police during recent lockdowns actually gives me a little hope, as it as drawn widespread attention to the way they routinely abuse their power.
  - iso1210 6y ago
    The general view in the population is the police do not deal with normal people other than as victims. If you're arrested, or even questioned, you're seen as guilty. Besides if you've nothing to hide you've nothing to fear. This view doesn't hold when the police get you for speeding (go find real criminals) Many police officers are quite reasonable, but then you have Derbyshire, whose actions over covid have likely killed people (by reducing compliance with lockdown orders) The difference with the police in covid is the public are suddenly seeing that the police are dealing with normal people, not just criminal scum who should be lucky they aren't shot. As I understood it, the Protection of Freedoms Act 2012 really clamped down on DNA and fingerprint storage. Not enough, but at least it was a move in the right direction. Particularly bad was that if you were charged and found innocent, they would still keep your records for 3 years.
ashergill 6y ago
I disagree with describing these events as 'technical issues' or 'IT errors'. As other commenters have noted it appears that there just wasn't a working backup. If that's the case, it's more of an organizational failing in my view: no-one cared enough to make a working backup. It's similar to the 'IT error' in our test and trace system recently, where thousands of positive covid test results were uncounted because Excel truncated the file after a certain number of rows. In my view, it's not really the tech at fault, it's that the organization was using Excel because they didn't care enough to do it properly. Calling it a 'technical issue' puts the blame on a malfunction, making it seem unfortunate but unavoidable, and absolving the organization that actually caused the issue.
- rwmj 6y ago
  I guess it's tricky when you are legally required to delete something - are you able to keep a backup of which contains that thing?
  - drdeadringer 6y ago
    I wonder how feasible, realistic, &c it would be to have included in the deletion/destruction policy to have a step/section in there to say "after verification of deletion, make a new backup to replace the old backup" which would be a backup refresh "outside" of the usual periodic backup.
    - thunfischbrot 6y ago
      That would work if you kept only backup version. That would be bad-what do you do in case of accidental record deletion or encryption malware detected after a backup? Check out e.g. https://www.backblaze.com/blog/the-3-2-1-backup-strategy/ https://www.backblaze.com/blog/the-3-2-1-backup-strategy/
  - woliveirajr 6y ago
    Wow. As a former "IAAL" (but not in the US), this gave me shivering ideas.Time to write some article/book about it.
d1sxeyes 6y ago
As I understand, '[f]ingerprint, DNA and arrest history records were deleted', but it's fine because 'the wiped records were those of people arrested and released when no further action was taken'. If the police cannot be trusted to put basic data protection measures in place (such as backups), perhaps they should not keep biometric data about people that have done nothing wrong[0]? [0]: more accurately, have not been charged with wrongdoing, meaning either there was insufficient evidence, or prosecution was not deemed to be in the public interest.
- vaduz 6y ago
  Or they have been processed, not charged with any wrongdoing, but they have not applied to have their records deleted under the post-2006 "Exceptional Case" procedure or post-2015 "Record Deletion" procedure. [0] Or maybe they just did not yet turn 100 years of age, when the expunge is supposed to happen automatically. [0] https://www.acro.police.uk/Services/Record-deletion https://www.acro.police.uk/Services/Record-deletion
  - Silhouette 6y ago
    The idea that someone who has been wronged by the criminal justice system should then be required to understand the consequences and make a special application just to have the record set straight is invidious, to say the least. No normal organisation processing personal data gets to keep whatever they feel like, however unjustified, and only remove it under sufferance and on specific request. That wasn't even appropriate before the GDPR came into effect, and it's clearly prohibited by law now if there was any doubt before. So why should governments get special privileges in this respect? There could hardly be a more serious case of processing personal data than a database used by police and security services that have exceptional legal powers with the potential to destroy someone's life. The argument that it's necessary for any sort of legitimate policing or security purpose when we're talking specifically about individuals who were not charged or who were acquitted at trial is tenuous at best. Our governments and courts are on the wrong side of this issue. They're so blinded by some combination of fear, paranoia and the power of modern technologies that they seem to have forgotten the moral imperatives that everyone should be treated fairly and equally before the law and that we are supposed to be living in a free country.
jack_riminton 6y ago
Why do we in the UK have politicians and public servants who have such a scant disregard for “IT”?
gerardnll 6y ago
Not even the government uses backups... crazy. Imagine other issues like privacy, I do not expect them to be compliant. They do not look to be that different than an average citizen at least in IT terms.
- vaduz 6y ago
  Infrastructure and operation of PNC has been outsourced to Logica (now CGI, since 2012). It's not operated by the government itself. I consider it par the course, though in this case I cannot be objective as my $CurrentCorpo had to kick off CGI/Logica early by 2014 from multiple projects - for, appropriately, gross incompetence in managing their own data centre (a redundant data centre in Wales that goes down for a week due to border routers being out of commission and failover problems tends to ruffle some feathers at high levels, when you asked and paid for for 99.95%+ SLA).
- ocdtrekkie 6y ago
  The big thing is that government organizations are very complicated, and due to bidding processes, have a lot of different vendors with different solutions involved. Vendors respond to proposals with complete bespoke solutions that may or may not integrate well with existing infrastructure design, and then some of those vendors are actively resistant to changes that integrate them in. It's really fun explaining to a vendor that 1. A system is going to be domain-joined so it can be managed and monitored. 2. The users will not have administrative rights to the machine. 3. The vendor will not have the ability to remote access the machine whenever they want. 4. Security software and policy settings will be applied to the machine. It continues to amaze me how often these four items absolutely blow the minds of the vendors I've dealt with.
markstos 6y ago
I recently audited how equitable some funds spend on new sidewalks had been in Bloomington, Indiana. In the process I found that one column in the project ranking spreadsheet had been sorted in the wrong order for the last five years. This effectively scrambled the final rank order. Excel was not fault, there was a lack of peer review and clear documentation about how the formula was supposed to work. https://mark.stosberg.com/sidewalk-equity-audit https://mark.stosberg.com/sidewalk-equity-audit
- ajcp 6y ago
  Thank you for sharing! This is very interesting. Also, unsolicited but maybe helpful: if I have a link to a PDF on a site I try and call it out as a download in the description. "A two page executive summary is also available for download" I say this because even though browser settings can be set to NOT download PDF files instead of automatically opening them in the browser itself, for some reason that can vary from site to site. I click a link and all of a sudden I'm downloading something? Panic!
Traster 6y ago
For those of you outside of the UK, the home office has a reputation for a horrible beauracracy where nothing works and everything gets screwed up. Imagine that instead of accidentally destroying police records, you destroyed the paperwork showing people had a legal right to live in the UK. And imagine that you then instituted a policy of deporting the people who couldn't prove they had a legal right to live in the UK. And imagine that all happened under the person who would go on to become prime minister, and that it would all come to light whilst she was prime minister.... And that wasn't the thing to cause her to resign. This is just a minor blip for the home office, and your ideas of back ups are quaint, but that's not he problem here.

83 points by edward 6y ago

Or more accurately, accidentally deleted. 'losing' these records would actually be a lot worse. I wouldn't underestimate the police's ability suffer a breach and lose the records either. It's worth remembering many of these records are are kept on the basis of non conviction (though there is a time limit).

codeulike 6y ago

It comes after around 40,000 alerts relating to European criminals were removed from the same database, the PNC, following Britain's post-Brexit deal with the EU. Right so reading between the lines they needed to do some post-Brexit data cleaning and someone messed up the WHERE clause. But where are the backups? Surely accidentally deleted data should be fairly easy to recover from historical backups which they surely surely must have?

randunel 6y ago
Wouldn't such international data removal agreements also include backups? What's the point in removing PII from one of the databases, if it remains in their backups?
- codeulike 6y ago
  You have a point but I have doubts that any organisation on the planet has the capability/willingness/resources to surgically remove data from historical backups. I think its more likely that no backups exist, or that, say, no backups exist past X days in the past. Or worse, that backups exist, but they have been found to be unrestorable.
- codeulike 6y ago
  Under GDPR right-to-erasure you dont necessarily have to erase data from backups but must be clear about how long it will hang around. The key issue is to put the backup data ‘beyond use’, even if it cannot be immediately overwritten. You must ensure that you do not use the data within the backup for any other purpose, ie that the backup is simply held on your systems until it is replaced in line with an established schedule. https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/individual-rights/right-to-erasure/#ib5 https://ico.org.uk/for-organisations/guide-to-data-protectio...
- tremon 6y ago
  You could handle such requests by tombstoning the user ID (in addition to deleting the data). By keeping the tombstone records in separate storage (it's usually only kB, just a list of hard-removed user IDs), you can guarantee (re-do) the erasure even after having to do a full database restore.
- vidarh 6y ago
  Backup. Encrypt the backup. Run the deletion. Verify you haven't lost anything. Verify it again. Put the backup encryption key beyond use other than in the case of a suitably agreed emergency (e.g. in the case of an agreement with another party, hand them the key). Then after a suitable grace period, once you're sure everything is ok, you destroy the backup.
  - nightcracker 6y ago
    > Put the backup encryption key beyond use other than in the case of a suitably agreed emergency (e.g. in the case of an agreement with another party, hand them the key). Like search warrants? We know those never get abused. > Then after a suitable grace period, once you're sure everything is ok, you destroy the backup. But we need a search warrant to access the backup anyway so we don't really have to delete it. Best keep it, just in case.
    - vidarh 6y ago
      > Like search warrants? We know those never get abused. No, like catastrophic loss of data, hence why I outlined a process of a doing this temporarily in the case of agreement with a another party to put the data entirely beyond your own use unless the party you've agreed the deletion with agrees the situation is serious enough. > But we need a search warrant to access the backup anyway so we don't really have to delete it. Best keep it, just in case. If you don't trust the party to delete the data, then adding in a temporary period of retention makes no difference, because in that case they could just as well have ignored the demand for the initial deletion entirely.

trestenhortz 6y ago

You can make pretty much any sort of tech fuckup and be forgiven. The only two unforgivable sins are “no working backups” and being hacked. Even being hacked is kinda forgivable cause it’s so hard to avoid. No working backups is totally unforgivable. Any tech person who walks into any tech job at any level up from junior to CTO has a duty as one of their first tasks to find out if working backups are being done of their work.

franze 6y ago
yeah, long time ago I synced the CRM database of the austria press agency to my sony clie https://en.wikipedia.org/wiki/CLI%C3%89 https://en.wikipedia.org/wiki/CLI%C3%89 and then my sony clie synced it back. completely deleting all data other then the email and name on the CRM server. gladly there was a backup, but the backup could not get used to get the database back-up. the discussion with IT were fabolous: "yes, we have it all backed up, so the backup is there, but we cant use it to bring the CRM back-up. but its backed up and we even have backups from the backup. they can also not get used to bringt it back up." since then it is my standard procedure to make sure that backups can actually be used to bring projects back up. usually the first test "use backup to bring stuff back up" fails.
- KMag 6y ago
  If you haven't tested your backups, you don't really have backups. If you aren't allowed to verify the backups yourself, find a safe "mistake" to make to force the responsible parties to run through restoring from backups. Bonus points for using the backup testing exercise as an excuse to convince your manager to let you do a bit of refactoring to remove some particularly ugly module you've been itching to get rid of. Factor out the ugly module, then "accidentally" delete it. My fist job out of school was writing software models of pre-prototype communications equipment for a defense contractor. There was lots of government red tape, and crazy prices being charged everywhere. We couldn't use CVS or SVN (this predated git) to store our code, because they weren't government-approved tools. We could use PVCS to store source code, but only for the build artefacts and corresponding source code of official production versions delivered to the government. (And only people that had taken a multi-hour course to be certified on PVCS could touch it. Every release involved a couple government employees coming out to the defense contractor and physically watching the one guy on the team check in the release to PVCS and burn a pair of DVDs for the government. So, we were forced to do source control by "last one out on Friday has to zip up the entire shared drive and name it with the date". But, at least we were required to pay some crazy price to have the sysadmins backup the shared network drive to tape every week (or was it every day?). Once a month, the admins would take the current tape and send it via courier to a bomb-proof, fire-proof bunker out in the mountains in Pennsylvania or some such. They'd then put the spare tape in the tape drive to start making backups. As soon as the bunker got the latest tape, they'd send back the oldest tape via courier to become the spare tape. So, we always had at least one tape physically present in the bunker. It required 3 tapes to always have at least one tape in the bunker and one tape in the drive. I was less than a year out of school, but based on my interactions with the system admins, I figured there was at most a 50/50 chance they got the backups correct. I asked them once how the backups were going, and got back a very low key, "yea, yea, don't worry about it, we got it" answer. I convinced my boss to let us fake a source code loss: zip up two copies of the network drive, and store the zip files on two different systems. Check the integrity of both zip files, and then "accidentally" delete a directory containing some source we were in the midst of removing anyway. The sysadmins were unable to get our "accidentally" deleted source code back for us. They had neglected to erase the spare tape when putting it into the drive, so they were always only appending to our backups. They just ignored the errors when the tape got full.

deleted 6y ago

[deleted]

bb123 6y ago

A more accurate headline would be '150k police records accidentally destroyed'. There is a huge difference between accidental deletion and leaving a hard drive full of data on the tube. I'd give the BBC the benefit of the doubt that the unclear wording is accidental and not for click bait purposes, but I feel like I'm doing that more and more for them at the moment.

justinclift 6y ago

Hang on. This sounds like the fingerprint and DNA data for people arrested and never charged. But which the police insisted on hanging on to anyway "just in case". Maybe it's to public benefit for it having been lost?

J-dawg 6y ago
The UK took a wrong turn when it gave the police their own extra-judicial justice system. The system of police cautions, warnings and fixed penalties allow the police to administer punishment with none of the due process that a court would provide. The idea of a "police caution" is a misnomer that implies it's just harmless warning, when in fact it can affect the course of someone's life many years later. Furthermore, people are blackmailed into accepting these cautions under the threat of going to trial and potentially receiving a more severe punishment after conviction. Even more outrageous is the retention of records from the PREVENT programme and "non-crime hate incidents". These are people who have not even committed a crime or been arrested. The whole system stinks, and given that most people manage to avoid interaction with the police it is sadly unlikely to get the scrutiny it deserves. The outrageous behaviour of (some of) the police during recent lockdowns actually gives me a little hope, as it as drawn widespread attention to the way they routinely abuse their power.
- iso1210 6y ago
  The general view in the population is the police do not deal with normal people other than as victims. If you're arrested, or even questioned, you're seen as guilty. Besides if you've nothing to hide you've nothing to fear. This view doesn't hold when the police get you for speeding (go find real criminals) Many police officers are quite reasonable, but then you have Derbyshire, whose actions over covid have likely killed people (by reducing compliance with lockdown orders) The difference with the police in covid is the public are suddenly seeing that the police are dealing with normal people, not just criminal scum who should be lucky they aren't shot. As I understood it, the Protection of Freedoms Act 2012 really clamped down on DNA and fingerprint storage. Not enough, but at least it was a move in the right direction. Particularly bad was that if you were charged and found innocent, they would still keep your records for 3 years.

ashergill 6y ago

I disagree with describing these events as 'technical issues' or 'IT errors'. As other commenters have noted it appears that there just wasn't a working backup. If that's the case, it's more of an organizational failing in my view: no-one cared enough to make a working backup. It's similar to the 'IT error' in our test and trace system recently, where thousands of positive covid test results were uncounted because Excel truncated the file after a certain number of rows. In my view, it's not really the tech at fault, it's that the organization was using Excel because they didn't care enough to do it properly. Calling it a 'technical issue' puts the blame on a malfunction, making it seem unfortunate but unavoidable, and absolving the organization that actually caused the issue.

rwmj 6y ago
I guess it's tricky when you are legally required to delete something - are you able to keep a backup of which contains that thing?
- drdeadringer 6y ago
  I wonder how feasible, realistic, &c it would be to have included in the deletion/destruction policy to have a step/section in there to say "after verification of deletion, make a new backup to replace the old backup" which would be a backup refresh "outside" of the usual periodic backup.
  - thunfischbrot 6y ago
    That would work if you kept only backup version. That would be bad-what do you do in case of accidental record deletion or encryption malware detected after a backup? Check out e.g. https://www.backblaze.com/blog/the-3-2-1-backup-strategy/ https://www.backblaze.com/blog/the-3-2-1-backup-strategy/
- woliveirajr 6y ago
  Wow. As a former "IAAL" (but not in the US), this gave me shivering ideas.Time to write some article/book about it.

d1sxeyes 6y ago

As I understand, '[f]ingerprint, DNA and arrest history records were deleted', but it's fine because 'the wiped records were those of people arrested and released when no further action was taken'. If the police cannot be trusted to put basic data protection measures in place (such as backups), perhaps they should not keep biometric data about people that have done nothing wrong[0]? [0]: more accurately, have not been charged with wrongdoing, meaning either there was insufficient evidence, or prosecution was not deemed to be in the public interest.

vaduz 6y ago
Or they have been processed, not charged with any wrongdoing, but they have not applied to have their records deleted under the post-2006 "Exceptional Case" procedure or post-2015 "Record Deletion" procedure. [0] Or maybe they just did not yet turn 100 years of age, when the expunge is supposed to happen automatically. [0] https://www.acro.police.uk/Services/Record-deletion https://www.acro.police.uk/Services/Record-deletion
- Silhouette 6y ago
  The idea that someone who has been wronged by the criminal justice system should then be required to understand the consequences and make a special application just to have the record set straight is invidious, to say the least. No normal organisation processing personal data gets to keep whatever they feel like, however unjustified, and only remove it under sufferance and on specific request. That wasn't even appropriate before the GDPR came into effect, and it's clearly prohibited by law now if there was any doubt before. So why should governments get special privileges in this respect? There could hardly be a more serious case of processing personal data than a database used by police and security services that have exceptional legal powers with the potential to destroy someone's life. The argument that it's necessary for any sort of legitimate policing or security purpose when we're talking specifically about individuals who were not charged or who were acquitted at trial is tenuous at best. Our governments and courts are on the wrong side of this issue. They're so blinded by some combination of fear, paranoia and the power of modern technologies that they seem to have forgotten the moral imperatives that everyone should be treated fairly and equally before the law and that we are supposed to be living in a free country.

jack_riminton 6y ago

Why do we in the UK have politicians and public servants who have such a scant disregard for “IT”?

gerardnll 6y ago

Not even the government uses backups... crazy. Imagine other issues like privacy, I do not expect them to be compliant. They do not look to be that different than an average citizen at least in IT terms.

vaduz 6y ago
Infrastructure and operation of PNC has been outsourced to Logica (now CGI, since 2012). It's not operated by the government itself. I consider it par the course, though in this case I cannot be objective as my $CurrentCorpo had to kick off CGI/Logica early by 2014 from multiple projects - for, appropriately, gross incompetence in managing their own data centre (a redundant data centre in Wales that goes down for a week due to border routers being out of commission and failover problems tends to ruffle some feathers at high levels, when you asked and paid for for 99.95%+ SLA).
ocdtrekkie 6y ago
The big thing is that government organizations are very complicated, and due to bidding processes, have a lot of different vendors with different solutions involved. Vendors respond to proposals with complete bespoke solutions that may or may not integrate well with existing infrastructure design, and then some of those vendors are actively resistant to changes that integrate them in. It's really fun explaining to a vendor that 1. A system is going to be domain-joined so it can be managed and monitored. 2. The users will not have administrative rights to the machine. 3. The vendor will not have the ability to remote access the machine whenever they want. 4. Security software and policy settings will be applied to the machine. It continues to amaze me how often these four items absolutely blow the minds of the vendors I've dealt with.

markstos 6y ago

I recently audited how equitable some funds spend on new sidewalks had been in Bloomington, Indiana. In the process I found that one column in the project ranking spreadsheet had been sorted in the wrong order for the last five years. This effectively scrambled the final rank order. Excel was not fault, there was a lack of peer review and clear documentation about how the formula was supposed to work. https://mark.stosberg.com/sidewalk-equity-audit https://mark.stosberg.com/sidewalk-equity-audit

ajcp 6y ago
Thank you for sharing! This is very interesting. Also, unsolicited but maybe helpful: if I have a link to a PDF on a site I try and call it out as a download in the description. "A two page executive summary is also available for download" I say this because even though browser settings can be set to NOT download PDF files instead of automatically opening them in the browser itself, for some reason that can vary from site to site. I click a link and all of a sudden I'm downloading something? Panic!

Traster 6y ago

For those of you outside of the UK, the home office has a reputation for a horrible beauracracy where nothing works and everything gets screwed up. Imagine that instead of accidentally destroying police records, you destroyed the paperwork showing people had a legal right to live in the UK. And imagine that you then instituted a policy of deporting the people who couldn't prove they had a legal right to live in the UK. And imagine that all happened under the person who would go on to become prime minister, and that it would all come to light whilst she was prime minister.... And that wasn't the thing to cause her to resign. This is just a minor blip for the home office, and your ideas of back ups are quaint, but that's not he problem here.