4 ms·

No — changing your code makes every one of those aspects harder, more time consuming, and thus a worse option. (Again, except network topology.) And yes — I’ve

No — changing your code makes every one of those aspects harder, more time consuming, and thus a worse option. (Again, except network topology.) And yes — I’ve helped migrate global systems on the peta-scale, for finance where compliance is legally mandated. The idea that changing your code at the same time is irrelevant is laughable: Changing your software significantly while migrating has caused major errors in projects I’ve seen attempt it — you’re essentially doing a rewrite while trying to migrate, which introduces bugs in compliance, security, etc. - - - - Edit to include a specific example: Let’s imagine I’m migrating a dataset, A. And my copy is A’. In the scenario where I have some wrappers, W and W’, I can write my validation test on the data, T, to work against that interface. So then T(W(A)) = T(W’(A’)) implies that W(A) = W’(A’), since T is a function (and the same code, run both places). My problem in demonstrating my data integrity reduces to ensuring that my wrappers faithfully marshal data — something I’ll get by writing unit tests. In the scenario where I don’t have those data wrappers, I have to write two tests — T and T’. But then what does T(A) = T’(A’) prove? Nothing directly — I need to prove that my two integrity tests are actually measuring the same thing. In this way, isolating the complexity of your data marshaling directly impacts the complexity of downstream tasks, such as verifying the integrity of your data.

scarface_74 3y ago
Again, you’re looking at your little piece of work that is a speck in an entire organization. Your CTO is working with your CIO, compliance department, finance department, product owners, the PMO organization, etc. They are talking to the board about strategic decisions and what will give them a competitive advantage. He’s not losing sleep over whether you put a facade over your data access layer because in some distant future long after he is gone, AWS may raise prices. If the spend is large enough, he’s going to talk to his dedicated sales rep to lower prices long before he comes to zmgsabt and asks him did he make his data access class “cloud agnostic”. A dependency always slips in somewhere unless you are constantly testing and preparing for portability like Uber does. Hell, I released code that was part of a major official open source “AWS Solution” and got complaints a few weeks later that it was dependent on a region and that’s not the first time I’ve seen that happen. Let alone a dependency on all of the arns always having “aws” in them not thinking they wouldn’t work in China or gov-cloud. I hardcoded the partition. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... Another case is that “we are using MySQL to avoid lock in with AWS”. Years later. “Oh shit. The data team is loading data into the database using the AWS extension that lets you load data from S3 using a sql query”
- asah 3y ago
  OP here: aws_s3 is a nice example of a proprietary extension that makes it hard to leave AWS.
- zmgsabst 3y ago
  > Again, you’re looking at your little piece of work that is a speck in an entire organization. > He’s not losing sleep over whether you put a facade over your data access layer because in some distant future long after he is gone, AWS may raise prices. Of course not — he’s paying experts to avoid that situation, aligning with his strategic priorities. That’s my job. You also resorted back to personal attacks because you can’t make your point directly. > If the spend is large enough, he’s going to talk to his dedicated sales rep to lower prices long before he comes to zmgsabt and asks him did he make his data access class “cloud agnostic”. Like this ridiculous comment to someone whose job is being hired by CTOs looking to migrate. They literally come to me, for help doing exactly that. > Hell, I released code that was part of a major official open source “AWS Solution” and got complaints a few weeks later that it was dependent on a region and that’s not the first time I’ve seen that happen. > I hardcoded the partition. > “Oh shit. The data team is loading data into the database using the AWS extension that lets you load data from S3 using a sql query” Yes — that’s why companies pay me: I’m used to doing things that can’t depend on a region, service, etc while AWS engineering frequently makes that mistake (eg, all the US-East-1 magic). That’s precisely my point: Quality coding avoids and bounds mistakes like that, which slow your migration. - - - - So you’ve resorted to personal attacks, but didn’t actually respond to my specific example, and were insistent on my credentials but still haven’t shared the largest system you personally have migrated. But please keep it up: I make my money from people like you causing CTOs to hire people like me to clean up the mess — “whoops, accidentally wrote regionalized code!” or “we need over a year to migrate 100 VMs due to self-induced complexity!”
  - scarface_74 3y ago
    Scenario: I have $aLotOfData in S3 and I need to move it to a MySql database. Are you going to: A) write an ETL job that is going to be slower and more complex to avoid the “lock-inz” Or B) just call run “load data from S3…” As a sql extension? Are you going to tell them that they can’t use Glue? Athena? Redshift? Quicksite? When your company needs a call center are you going to tell them not to use Amazon Connect? Now they need to integrate the call center with Salesforce - are you going to tell them they can’t do that because they have to use Lambda? They want to have speech to text - now they need Amazon Lex. Then they want a reporting dashboard that comes from their contact trace records. Now they need Kinesis which is going to stream to S3. The typical company uses over 250 SaaS products that they have some form of integration with. Are you going to tell them not to use any of those either? Are you going to tell them not to use Salesforce? ServiceNow? Microsoft Office? If you work in the health care industry are you going to tell them not to use Epic?

scarface_74 3y ago

Again, you’re looking at your little piece of work that is a speck in an entire organization. Your CTO is working with your CIO, compliance department, finance department, product owners, the PMO organization, etc. They are talking to the board about strategic decisions and what will give them a competitive advantage. He’s not losing sleep over whether you put a facade over your data access layer because in some distant future long after he is gone, AWS may raise prices. If the spend is large enough, he’s going to talk to his dedicated sales rep to lower prices long before he comes to zmgsabt and asks him did he make his data access class “cloud agnostic”. A dependency always slips in somewhere unless you are constantly testing and preparing for portability like Uber does. Hell, I released code that was part of a major official open source “AWS Solution” and got complaints a few weeks later that it was dependent on a region and that’s not the first time I’ve seen that happen. Let alone a dependency on all of the arns always having “aws” in them not thinking they wouldn’t work in China or gov-cloud. I hardcoded the partition. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui... Another case is that “we are using MySQL to avoid lock in with AWS”. Years later. “Oh shit. The data team is loading data into the database using the AWS extension that lets you load data from S3 using a sql query”

asah 3y ago
OP here: aws_s3 is a nice example of a proprietary extension that makes it hard to leave AWS.
zmgsabst 3y ago
> Again, you’re looking at your little piece of work that is a speck in an entire organization. > He’s not losing sleep over whether you put a facade over your data access layer because in some distant future long after he is gone, AWS may raise prices. Of course not — he’s paying experts to avoid that situation, aligning with his strategic priorities. That’s my job. You also resorted back to personal attacks because you can’t make your point directly. > If the spend is large enough, he’s going to talk to his dedicated sales rep to lower prices long before he comes to zmgsabt and asks him did he make his data access class “cloud agnostic”. Like this ridiculous comment to someone whose job is being hired by CTOs looking to migrate. They literally come to me, for help doing exactly that. > Hell, I released code that was part of a major official open source “AWS Solution” and got complaints a few weeks later that it was dependent on a region and that’s not the first time I’ve seen that happen. > I hardcoded the partition. > “Oh shit. The data team is loading data into the database using the AWS extension that lets you load data from S3 using a sql query” Yes — that’s why companies pay me: I’m used to doing things that can’t depend on a region, service, etc while AWS engineering frequently makes that mistake (eg, all the US-East-1 magic). That’s precisely my point: Quality coding avoids and bounds mistakes like that, which slow your migration. - - - - So you’ve resorted to personal attacks, but didn’t actually respond to my specific example, and were insistent on my credentials but still haven’t shared the largest system you personally have migrated. But please keep it up: I make my money from people like you causing CTOs to hire people like me to clean up the mess — “whoops, accidentally wrote regionalized code!” or “we need over a year to migrate 100 VMs due to self-induced complexity!”
- scarface_74 3y ago
  Scenario: I have $aLotOfData in S3 and I need to move it to a MySql database. Are you going to: A) write an ETL job that is going to be slower and more complex to avoid the “lock-inz” Or B) just call run “load data from S3…” As a sql extension? Are you going to tell them that they can’t use Glue? Athena? Redshift? Quicksite? When your company needs a call center are you going to tell them not to use Amazon Connect? Now they need to integrate the call center with Salesforce - are you going to tell them they can’t do that because they have to use Lambda? They want to have speech to text - now they need Amazon Lex. Then they want a reporting dashboard that comes from their contact trace records. Now they need Kinesis which is going to stream to S3. The typical company uses over 250 SaaS products that they have some form of integration with. Are you going to tell them not to use any of those either? Are you going to tell them not to use Salesforce? ServiceNow? Microsoft Office? If you work in the health care industry are you going to tell them not to use Epic?