iamsomewalrus

7 ms·

31.
by iamsomewalrus 8y ago
I have personally only used them for debugging one off questions. That was my original intent. We do have teams that are considering using the snapshots for ML problems.
32.
by iamsomewalrus 8y ago
We use carrier pigeons from an S3 data center high in the Himalayas to a CloudFront distribution center in Atlantic City (don't ask me, ask the pigeons) to serve all requests. Just BI.
33.
by iamsomewalrus 8y ago
Absolutely! We have internships in San Francisco and Seattle. Usually, they coincide with the summer, but we on occasion have Fall interns. Reach out to me feeneyj @ amazon if you'd like I can forward you to the right people.
34.
by iamsomewalrus 8y ago
Having been back in SQL land for a bit (vanilla MySQL on RDS) I have to say that I _love_ a well designed SQL database. I forgot how much I had given up in NoSQL land. Goodreads hit scaling issues a while ago with Active Record and a single
35.
by iamsomewalrus 8y ago
I did! I'm a big fan of the Last Week in AWS newsletter. https://twitter.com/iamsomewalrus/status/1016368717880389632
36.
by iamsomewalrus 8y ago
:this: A concept that's underlying the move to a datalake architecture (read: keeping your data in its rawest form, and its transforms in S3 or HDFS) is decoupling your compute from storage. Motivating example: you have huge tables in
37.
by iamsomewalrus 8y ago
That’s good to hear! The underlying architecture project has been in motion for a few years. We’re excited that it’s out.
38.
by iamsomewalrus 8y ago
No, but we have a really responsive customer service team that watches the Goodreads feedback group very closely.
39.
by iamsomewalrus 8y ago
Come join my team and I’ll be happy to spill all the beans!
40.
by iamsomewalrus 8y ago
We can, and will do better in the areas you’ve mentioned. I’ve been bitten by that same search bug :-(. I’ll cut a ticket for that.
41.
by iamsomewalrus 8y ago
That’s “the question”. I will do my best to bluff through it. Engineering team size: you have teams large enough (~3-4) to own and iterate on a subset of related functionality for the long term. Tooling: you have a builder tools type team t
42.
by iamsomewalrus 8y ago
Data will eventually end up there. Our product and business teams are heavy users of Redshift. As mentioned in another comment I’ve found having Dynamo snapshots in Athena really useful as an oncall to sanity check snapshots (what was the s
43.
by iamsomewalrus 8y ago
We have another GR team that does that and it works well. It’s complicated by the fact that at Amazon every team uses their own AWS account. Concretely, a service’s DynamoDB tables don’t exist in the same AWS account let alone the same VPC
44.
by iamsomewalrus 8y ago
Thanks all. Long time lurker, first time commenter.
45.
by iamsomewalrus 8y ago
There’s no real policy that I’m aware of internally for teams to use microservices. Amazon has a lot of tooling to make it easy to spin up services, however. The first major project after being acquired was to make a pared down Goodreads ex
46.
by iamsomewalrus 8y ago
Athena is just a front end for the data that a typical user can understand (SQL!). the real value is that the data is: in S3, in a more efficient format (parquet), and available in the Glue catalog. The other replies got it right w.r.t. oth
47.
by iamsomewalrus 8y ago
Did we just become best friends with our “I am the Walrus” references? The DB scrape uses a template from Data Pipeline that under the hood uses the Dynamo DB Scan API. Not really surprising, but that API uses JSON. I wanted to use as much
48.
by iamsomewalrus 8y ago
Hey! I’m the author of this post. I’m pretty chuffed to see this here. Happy to answer any questions.
49.
by iamsomewalrus 9y ago
My team has run into the same problem in a large Rails app. We decided on a combination of service objects and data access objects (DAO) where appropriate. In a lot of cases this has really helped with the grep-ability of our codebase. Serv
50.
by iamsomewalrus 13y ago
I have two invites. Email in profile.