4 ms·

Yawn, insulting others experience to make your point is trite. Perhaps you could justify your point rather than attack other people and appeal to authority?

scarface_74 3y ago
There is so much involved with any migration at scale - data migration, regression testing, network changes even where a company many have a physical connection to their cloud provider, security, compliance , training, and the list goes on.
- zmgsabst 3y ago
  Okay — and? How is any of that relevant to what I said? You’re ignoring that maintaining your code without modifying your data access (changing only the wrapper) significantly simplifies virtually everything you listed - with the exception of network topology changes.
  - scarface_74 3y ago
    Because in the grand scheme of things “your code” and even changing your code is just a minuscule part of what it takes if a migration did ever happen. So the question remains, have you ever been a part of a large scale migration? Have you ever been in the room when the planning was happening?
    - zmgsabst 3y ago
      No — changing your code makes every one of those aspects harder, more time consuming, and thus a worse option. (Again, except network topology.) And yes — I’ve helped migrate global systems on the peta-scale, for finance where compliance is legally mandated. The idea that changing your code at the same time is irrelevant is laughable: Changing your software significantly while migrating has caused major errors in projects I’ve seen attempt it — you’re essentially doing a rewrite while trying to migrate, which introduces bugs in compliance, security, etc. - - - - Edit to include a specific example: Let’s imagine I’m migrating a dataset, A. And my copy is A’. In the scenario where I have some wrappers, W and W’, I can write my validation test on the data, T, to work against that interface. So then T(W(A)) = T(W’(A’)) implies that W(A) = W’(A’), since T is a function (and the same code, run both places). My problem in demonstrating my data integrity reduces to ensuring that my wrappers faithfully marshal data — something I’ll get by writing unit tests. In the scenario where I don’t have those data wrappers, I have to write two tests — T and T’. But then what does T(A) = T’(A’) prove? Nothing directly — I need to prove that my two integrity tests are actually measuring the same thing. In this way, isolating the complexity of your data marshaling directly impacts the complexity of downstream tasks, such as verifying the integrity of your data.
      - scarface_74 3y ago
        Again, you’re looking at your little piece of work that is a speck in an entire organization. Your CTO is working with your CIO, compliance department, finance department, product owners, the PMO organization, etc. They are talking to the board about strategic decisions and what will give them a competitive advantage. He’s not losing sleep over whether you put a facade over your data access layer because in some distant future long after he is gone, AWS may raise prices. If the spend is large enough, he’s going to talk to his dedicated sales rep to lower prices long before he comes to zmgsabt and asks him did he make his data access class “cloud agnostic”. A dependency always slips in somewhere unless you are constantly testing and preparing for portability like Uber does. Hell, I released code that was part of a major official open source “AWS Solution” and got complaints a few weeks later that it was dependent on a region and that’s not the first time I’ve seen that happen. Let alone a dependency on all of the arns always having “aws” in them not thinking they wouldn’t work in China or gov-cloud. I hardcoded the partition. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... Another case is that “we are using MySQL to avoid lock in with AWS”. Years later. “Oh shit. The data team is loading data into the database using the AWS extension that lets you load data from S3 using a sql query”
        asah 3y ago
        OP here: aws_s3 is a nice example of a proprietary extension that makes it hard to leave AWS.
        zmgsabst 3y ago
        > Again, you’re looking at your little piece of work that is a speck in an entire organization. > He’s not losing sleep over whether you put a facade over your data access layer because in some distant future long after he is gone, AWS may raise prices. Of course not — he’s paying experts to avoid that situation, aligning with his strategic priorities. That’s my job. You also resorted back to personal attacks because you can’t make your point directly. > If the spend is large enough, he’s going to talk to his dedicated sales rep to lower prices long before he comes to zmgsabt and asks him did he make his data access class “cloud agnostic”. Like this ridiculous comment to someone whose job is being hired by CTOs looking to migrate. They literally come to me, for help doing exactly that. > Hell, I released code that was part of a major official open source “AWS Solution” and got complaints a few weeks later that it was dependent on a region and that’s not the first time I’ve seen that happen. > I hardcoded the partition. > “Oh shit. The data team is loading data into the database using the AWS extension that lets you load data from S3 using a sql query” Yes — that’s why companies pay me: I’m used to doing things that can’t depend on a region, service, etc while AWS engineering frequently makes that mistake (eg, all the US-East-1 magic). That’s precisely my point: Quality coding avoids and bounds mistakes like that, which slow your migration. - - - - So you’ve resorted to personal attacks, but didn’t actually respond to my specific example, and were insistent on my credentials but still haven’t shared the largest system you personally have migrated. But please keep it up: I make my money from people like you causing CTOs to hire people like me to clean up the mess — “whoops, accidentally wrote regionalized code!” or “we need over a year to migrate 100 VMs due to self-induced complexity!”
    - lallysingh 3y ago
      Yikes, no. Your code is the one thing you control. You can make it an asset for migration or a liability.
      - scarface_74 3y ago
        And “your code” is probably a tiny part of the entire enterprises architecture. The average enterprise has 250+ SaaS dependencies. Do you really think the CTO of a large organization is staying up late at night because a developer chose to use DynamoDB and didn’t put a facade over it to “avoid lock-in”?
        lallysingh 3y ago
        If they're getting screwed in pricing or availability vs a competitor, yes. They're asking "how's long do we have to stay with this bad option before we can switch?"

scarface_74 3y ago

There is so much involved with any migration at scale - data migration, regression testing, network changes even where a company many have a physical connection to their cloud provider, security, compliance , training, and the list goes on.

zmgsabst 3y ago
Okay — and? How is any of that relevant to what I said? You’re ignoring that maintaining your code without modifying your data access (changing only the wrapper) significantly simplifies virtually everything you listed - with the exception of network topology changes.
- scarface_74 3y ago
  Because in the grand scheme of things “your code” and even changing your code is just a minuscule part of what it takes if a migration did ever happen. So the question remains, have you ever been a part of a large scale migration? Have you ever been in the room when the planning was happening?
  - zmgsabst 3y ago
    No — changing your code makes every one of those aspects harder, more time consuming, and thus a worse option. (Again, except network topology.) And yes — I’ve helped migrate global systems on the peta-scale, for finance where compliance is legally mandated. The idea that changing your code at the same time is irrelevant is laughable: Changing your software significantly while migrating has caused major errors in projects I’ve seen attempt it — you’re essentially doing a rewrite while trying to migrate, which introduces bugs in compliance, security, etc. - - - - Edit to include a specific example: Let’s imagine I’m migrating a dataset, A. And my copy is A’. In the scenario where I have some wrappers, W and W’, I can write my validation test on the data, T, to work against that interface. So then T(W(A)) = T(W’(A’)) implies that W(A) = W’(A’), since T is a function (and the same code, run both places). My problem in demonstrating my data integrity reduces to ensuring that my wrappers faithfully marshal data — something I’ll get by writing unit tests. In the scenario where I don’t have those data wrappers, I have to write two tests — T and T’. But then what does T(A) = T’(A’) prove? Nothing directly — I need to prove that my two integrity tests are actually measuring the same thing. In this way, isolating the complexity of your data marshaling directly impacts the complexity of downstream tasks, such as verifying the integrity of your data.
    - scarface_74 3y ago
      Again, you’re looking at your little piece of work that is a speck in an entire organization. Your CTO is working with your CIO, compliance department, finance department, product owners, the PMO organization, etc. They are talking to the board about strategic decisions and what will give them a competitive advantage. He’s not losing sleep over whether you put a facade over your data access layer because in some distant future long after he is gone, AWS may raise prices. If the spend is large enough, he’s going to talk to his dedicated sales rep to lower prices long before he comes to zmgsabt and asks him did he make his data access class “cloud agnostic”. A dependency always slips in somewhere unless you are constantly testing and preparing for portability like Uber does. Hell, I released code that was part of a major official open source “AWS Solution” and got complaints a few weeks later that it was dependent on a region and that’s not the first time I’ve seen that happen. Let alone a dependency on all of the arns always having “aws” in them not thinking they wouldn’t work in China or gov-cloud. I hardcoded the partition. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... Another case is that “we are using MySQL to avoid lock in with AWS”. Years later. “Oh shit. The data team is loading data into the database using the AWS extension that lets you load data from S3 using a sql query”
      - asah 3y ago
        OP here: aws_s3 is a nice example of a proprietary extension that makes it hard to leave AWS.
      - zmgsabst 3y ago
        > Again, you’re looking at your little piece of work that is a speck in an entire organization. > He’s not losing sleep over whether you put a facade over your data access layer because in some distant future long after he is gone, AWS may raise prices. Of course not — he’s paying experts to avoid that situation, aligning with his strategic priorities. That’s my job. You also resorted back to personal attacks because you can’t make your point directly. > If the spend is large enough, he’s going to talk to his dedicated sales rep to lower prices long before he comes to zmgsabt and asks him did he make his data access class “cloud agnostic”. Like this ridiculous comment to someone whose job is being hired by CTOs looking to migrate. They literally come to me, for help doing exactly that. > Hell, I released code that was part of a major official open source “AWS Solution” and got complaints a few weeks later that it was dependent on a region and that’s not the first time I’ve seen that happen. > I hardcoded the partition. > “Oh shit. The data team is loading data into the database using the AWS extension that lets you load data from S3 using a sql query” Yes — that’s why companies pay me: I’m used to doing things that can’t depend on a region, service, etc while AWS engineering frequently makes that mistake (eg, all the US-East-1 magic). That’s precisely my point: Quality coding avoids and bounds mistakes like that, which slow your migration. - - - - So you’ve resorted to personal attacks, but didn’t actually respond to my specific example, and were insistent on my credentials but still haven’t shared the largest system you personally have migrated. But please keep it up: I make my money from people like you causing CTOs to hire people like me to clean up the mess — “whoops, accidentally wrote regionalized code!” or “we need over a year to migrate 100 VMs due to self-induced complexity!”
  - lallysingh 3y ago
    Yikes, no. Your code is the one thing you control. You can make it an asset for migration or a liability.
    - scarface_74 3y ago
      And “your code” is probably a tiny part of the entire enterprises architecture. The average enterprise has 250+ SaaS dependencies. Do you really think the CTO of a large organization is staying up late at night because a developer chose to use DynamoDB and didn’t put a facade over it to “avoid lock-in”?
      - lallysingh 3y ago
        If they're getting screwed in pricing or availability vs a competitor, yes. They're asking "how's long do we have to stay with this bad option before we can switch?"