3 ms·

Possibly because the app is running on Lambda? Cloudwatch logging is not great for Lambdas.

by guitarbill 9y ago

szalonypiekarz 9y ago
In that case, why not Lambda -> Cloudwatch Logs -> Firehose -> S3 ? No bespoke logging solution is needed.
- joemaller1 9y ago
  Don’t assume the author made an informed but poor choice, they likely made something work with the tools they know or found. It’s really difficult to keep up with all the various AWS offerings if your not part of a team that’s immersed in it. Constructively suggesting alternates is welcome, I’ve got some new tools to read up on this morning.
jcims 9y ago
Why is that?
throwaway2016a 9y ago
> Cloudwatch logging is not great for Lambdas. I run some Lambda systems at scale and CloudWatch Logs has worked excellent for me. It was completely painless and CloudWatch logs has a lot of tooling that comes with it (backups, streaming, indexing with Elastic Search, etc). What part of it has not worked well for you?
- guitarbill 9y ago
  Mainly searching and exporting. Exporting logs to S3 manually for debugging/post-mortem is harder than it needs to be - maybe this isn't a huge use-case, but we like to attach logs to tickets. And searching is very limited. Yes, you can stream them to ElasticSearch => LogStash/Kibana, etc but that's extra steps, and I can do similar things completely without CloudWatch Logs, e.g. Sentry and SumoLogic integrate fine with Lambdas also. I'm not hating on CloudWatch Logs, and alarms are good. It's just not a fully featured logging solution, and it sounds like you agree with that?
  - throwaway2016a 9y ago
    I agree you need more tools around it to make it a full logging solution. Doing data visualization is more or less impossible without extra tools. But I've overall liked it. Search is powerful enough to do something like say, "get me all messages with this trace GUID" and S3 exports can be automated[1] and you could probably use the "task-name" to associate with your ticketing system. [1] http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasks.html http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3Ex...

Possibly because the app is running on Lambda? Cloudwatch logging is not great for Lambdas.

szalonypiekarz 9y ago

In that case, why not Lambda -> Cloudwatch Logs -> Firehose -> S3 ? No bespoke logging solution is needed.

joemaller1 9y ago
Don’t assume the author made an informed but poor choice, they likely made something work with the tools they know or found. It’s really difficult to keep up with all the various AWS offerings if your not part of a team that’s immersed in it. Constructively suggesting alternates is welcome, I’ve got some new tools to read up on this morning.

jcims 9y ago

Why is that?

throwaway2016a 9y ago

> Cloudwatch logging is not great for Lambdas. I run some Lambda systems at scale and CloudWatch Logs has worked excellent for me. It was completely painless and CloudWatch logs has a lot of tooling that comes with it (backups, streaming, indexing with Elastic Search, etc). What part of it has not worked well for you?

guitarbill 9y ago
Mainly searching and exporting. Exporting logs to S3 manually for debugging/post-mortem is harder than it needs to be - maybe this isn't a huge use-case, but we like to attach logs to tickets. And searching is very limited. Yes, you can stream them to ElasticSearch => LogStash/Kibana, etc but that's extra steps, and I can do similar things completely without CloudWatch Logs, e.g. Sentry and SumoLogic integrate fine with Lambdas also. I'm not hating on CloudWatch Logs, and alarms are good. It's just not a fully featured logging solution, and it sounds like you agree with that?
- throwaway2016a 9y ago
  I agree you need more tools around it to make it a full logging solution. Doing data visualization is more or less impossible without extra tools. But I've overall liked it. Search is powerful enough to do something like say, "get me all messages with this trace GUID" and S3 exports can be automated[1] and you could probably use the "task-name" to associate with your ticketing system. [1] http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3ExportTasks.html http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3Ex...