ryanworl

33 ms·

301.
by ryanworl 9y ago
“If you use Amazon EC2 for running your Amazon Kinesis Applications” is the key there. You don’t have to use Amazon EC2. Amazon Kinesis Application does not imply EC2. They use on-prem examples in a few Re:Invent presentations. It is just a
302.
by ryanworl 9y ago
For anyone who wants to move this much data out of AWS and into another cloud provider, Kinesis Streams does not charge for bandwidth out. If you do the math, it works out to a very large savings over doing public internet data transfers if
303.
by ryanworl 9y ago
Ironically, the popular media has historically been the one of the largest sources of consumer data for ages through magazine subscriptions. They sell their data to brokers like Axciom (and many others) who package it and literally sell it
304.
by ryanworl 9y ago
That makes sense. If you do ever want to check out Clickhouse and want someone to run it for you, Percona or Altinity [1] can probably help. Not affiliated with either, I just read their Clickhouse-related content. [1]( https://ww
305.
by ryanworl 9y ago
I think the choice to not go with Clickhouse deserves a bit more explanation than what was given in the article. Instead of writing all this code to do roll ups they could’ve used an AggregatingMergeTree table over their raw events table an
306.
by ryanworl 9y ago
They made money through discounts on retail shipping as well as fees. Concentrating pick-ups to a small number of warehouses while doing the specific packaging and labeling and palletizing requirements of the shipper increases the discount.
307.
by ryanworl 9y ago
I run a very large number of web performance tests and GA and the Facebook pixel do not add 400ms either alone or together of user-perceptible speed decreases. They will take some small amount of main thread time for parse and compile, but
308.
by ryanworl 9y ago
For 5, check out Clickhouse. It isn’t identical, but scanning a trillion rows a second is just a matter of sharding the data into enough nodes. https://clickhouse.yandex
309.
by ryanworl 9y ago
I don’t have a problem with taxation, nor a problem with revenue taxes which is essentially what a sales tax is. All I’m saying is the targeting here is obviously political and designed to hurt the American companies. The result will be tha
310.
by ryanworl 9y ago
Revenue taxes on businesses that are effectively monopolies in their markets are... not smart. They will raise prices by exactly the amount of the tax and the only ones suffering are their customers who now pay 3% more for everything. I bet
311.
by ryanworl 9y ago
In Lambda and Cloud Functions you can open a connection with a database (or anything else) and it will stay alive across invocations of the same underlying container. Not sure how Cloudflare workers behaves here, but from their docs they re
312.
by ryanworl 9y ago
Ok thanks. Do workers play nicely with managed CNAMEs?
313.
by ryanworl 9y ago
Can a Cloudflare app install a worker into a Cloudflare customer’s zone?
314.
by ryanworl 9y ago
I know I mentioned this in another comment, but you really should check out Clickhouse. They have a table engine for exactly this purpose. You create a table with the raw logs, then a materialized view (or another actual table) which declar
315.
by ryanworl 9y ago
Did you evaluate Clickhouse?
316.
by ryanworl 9y ago
Yes, this is called “managerial accounting” in business school.
317.
by ryanworl 9y ago
It seems like when using legitimate interest as a basis for processing that _what you do_ with the data is much more important than what it is you’re collecting in the first place. When registering an account with an online service, you wil
318.
by ryanworl 9y ago
No. In AWS Lambda at least, the container your application runs in lives through multiple requests until some period of inactivity passes and it is shut down. It isn’t “running” in the sense that your code is in control, but long-lived thin
319.
by ryanworl 9y ago
Yes, it definitely wouldn’t be performance of a contract! I have no insight into their reasoning here, I just wanted to bring this fact up given I see people talking about consent all the time when that is not the only basis allowed under t
320.
by ryanworl 9y ago
Consent is just one way to legally process personal data. If their basis for processing it is something other than consent, an opt-out is them going above their responsibility.
321.
by ryanworl 9y ago
Breaking down your JavaScript bundles into smaller files that contain the code necessary for the route you’re actually going to be viewing once the page is loaded. For other routes, you fetch the necessary code on-demand. This lessens the C
322.
by ryanworl 9y ago
For anyone reading this thread and thinking about server side rendering their single page app: do not go down that route unless you’re willing to also implement proper chunking! If you SSR your pages, but still need to load a giant bundle,
323.
by ryanworl 9y ago
Every 10GB chunk of the database is on six different disks in 3 AZs.
324.
by ryanworl 9y ago
You’re definitely not wrong, but if you simply can’t operate a system to store everything for some reason, sampling is a lot better than just doing an aggregation on one dimension and throwing everything else away. For anyone reading this w
325.
by ryanworl 9y ago
You can use sampling to both store every dimension of a data point and to not store an unwieldy amount of data.
326.
by ryanworl 9y ago
No, I don’t, sorry. What I can promise you is that you’ll thank yourself for implementing it! There is hardly any additional complexity here because you’d probably be uploading the derived content somewhere anyway. Now you’re just putting i
327.
by ryanworl 9y ago
You would use two buckets in this case. Input bucket gets consumed by worker processes to do the transcoding (and validation) and then they upload into the output bucket. The output bucket is what you serve to clients (hopefully with a CDN
328.
by ryanworl 9y ago
They also have a podcast which covers their biggest stories of the week with interviews of reporters who wrote them. It is free and not very long, so if you like podcasts I would check it out.
329.
by ryanworl 9y ago
I want to build this. Can you send an email to the address in my profile? I would love to talk to you for 5 minutes about it if you have time.
330.
by ryanworl 9y ago
This varies by country, but for the United States, the law requires you to honor opt out requests. A link is one mechanism to honor such a request. Another way is to remove a recipient from a list after they reply stating their wish to be u

More ›