12 ms·
Tell HN: By default, New Relic will start collecting production log data on 5/3
Unless you've been very careful, your production logs almost certainly contain secrets or personally identifying information.
I was surprised (and annoyed) to receive the email below from New Relic, stating that on 5/3 they will start ingesting all production log data, by default.
To make matters worse, if you provisioned New Relic through Heroku, you can only opt out (by enabling High Security Mode) if you contact support. And if you're on the free plan, you can't open support tickets so have to ask on the community forum.
> New Relic APM agents will collect log data starting 5/3!
We’ve been hard at work making improvements to our APM and logging capabilities, and when you update certain APM agents starting May 3, 2022, logs will be automatically collected and sent to New Relic One. Logs are a critical telemetry type for observability and this new feature will help you troubleshoot your applications faster.
You probably have a few questions—including how to customize your logs ingest—so we’re including a FAQ below.
FAQ:
Q: Why did you make this change?
A: Logs are a critical telemetry data type but they are messy. This improvement allows users to send contextualized log data to New Relic without any additional setup. Relevant log data is now surfaced and correlated with other application telemetry automatically, reducing the need to switch context or run log queries when troubleshooting your applications.
Q: Which APM agents will have automatic logs collection and ingest upon upgrade?
A: Starting May 3, 2022, when users upgrade to the latest version of the Java, Ruby, and .NET agent, log ingest will be enabled by default, unless High Security Mode is enabled or you have enabled the logs toggle for your accounts (more information on this below). We expect to enable application logs for Node.js, Python, and Go by July and PHP by September.
Q: I have already implemented logs in context. What should I do?
A: We recommend only using manual OR automatic log forwarding. For more information, check out this documentation.
Q: I already use a third-party log forwarder, or forward logs via the New Relic infrastructure agent. What should I do?
A: To avoid duplicating log data, consult this documentation.
Q: What does this mean for my New Relic bill?
A: Collecting application logs means that more data will be ingested into the platform, at your standard ingest rate. The APM agent samples logs to ensure optimal agent performance. You can increase or decrease an application’s log volume as desired. Learn more here.
Q: I am concerned about sensitive log data being sent to New Relic. What should I do?
A: No logs of any kind will be collected if High Security Mode is enabled on the agent, even after the agent is upgraded. If you do want to use New Relic Logs, it is also possible to configure drop filters to ensure sensitive data is not stored in New Relic. If you have not enabled High Security Mode, but still do not want to send logs to New Relic, see the next question.
Q: I do not want New Relic to collect or ingest logs, even after I upgrade my agents. What should I do?
A: You can either configure the agent config file locally on a machine to disable it, or you can disable logs ingest for APM agents at the account level with a toggle in the New Relic data management hub. The toggle can be flipped before ever setting up an APM agent that forwards log data.
Q: Where can I learn more?
A: Check out our documentation, read the Explorer’s Hub post, reach out to your account team, or contact New Relic Support.
- darkwater 4y agoI was going to post "this must be a thing only in the USA, no way they are doing this to European customers due to GDPR", then I went to check my work email and bam, there it is. Gonna set the environment variables to disable this first thing in the morning tomorrow.
- logging4all 4y agoI have a free account and I can disable the automatic log collection on this page: https://one.newrelic.com/data-management-hub https://one.newrelic.com/data-management-hub just click on "log settings" in the left-hand pane. Doc page with details: https://docs.newrelic.com/docs/logs/logs-context/disable-automatic-logging/#disable-ui https://docs.newrelic.com/docs/logs/logs-context/disable-aut...
- speedgoose 4y agoA bit strange to have this feature enabled after an upgrade. The following statement is now wrong: > New Relic services are designed to receive and process telemetry data on the performance of applications, systems, and infrastructure, which typically do not contain any personal data. Customers generally send very little additional personal data to our platform. https://newrelic.com/blog/how-to-relic/how-demise-of-privacy-shield-affects-you https://newrelic.com/blog/how-to-relic/how-demise-of-privacy... On the topic of privacy and GDPR, I find the previous link a good summary of the arguments used by American companies that cannot respect GDPR. Microsoft or Google say the same things in a lot more words.
- jiveturkey 4y agoIf you are logging Personal Data, is that even wise, wrt GDPR? Logs have a funny way of escaping, NEWR notwithstanding. I'm not a Privacy Engineer, but surely everyone understands that you don't log raw passwords. Similarly, if you are bound by GDPR, shouldn't you not log PD?
- mhitza 4y agoFrom my experience most "shops" don't disable IP address logging in their load balancers/web servers, which is personal data under the GDPR and California privacy act.