4 ms·

I see it as a tiered approach - Detailed Process Logs saved to a db, Audits, Regular logs streamed to your favorite log dump, Monitoring, Health Checks, Metrics

by csours 5y ago

I see it as a tiered approach - Detailed Process Logs saved to a db, Audits, Regular logs streamed to your favorite log dump, Monitoring, Health Checks, Metrics, and Usage Statistics are all part of Observability. If it is a critical process, use detailed process logs. If it is an extremely critical process add Auditing.

mmanciop 5y ago
Yep, pretty much. There is plenty different types of telemetry one can use, and which ones are important for one application depends both on the nature of that application, and who operates them.
nivertech 5y ago
Regular logs (text files) > Detailed Process Logs (DB) > Audits (read-only/tamper-proof) What's the difference between Regular logs, Detailed Process Logs and Audits? Are Audits more compliance related, while detailed logs are business only? It's not clear to me why to have a special detailed process log, if you can you can write important data to Audits, and less important to regular logs, i.e. once can use different log levels for business data vs engineering/debug data. Can you please give an example?
- csours 5y ago
  Regular logs are just whatever your app logs normally (engineering/debug). Process logs are business processes that you need to report on or answer for later. Audits are another business layer for safety or life critical data, or adversarial data (handling money based transactions). They could be compliance related. Auditing is a three step process - Ask a question, figure out how to answer it, and make sure you've answered it. So specifically, if you have a magic bean shop and you want to log transfer of the beans, that would be a process log. Then at the end of the day, you would send a manifest of all the beans sent that day. The audit process would look at the final location of the beans, and also at the process log to answer your audit questions. --- I think a lot of people make a hard distinction between business logging and engineering logging, and I'm not sure that makes sense.

by csours 5y ago

mmanciop 5y ago

Yep, pretty much. There is plenty different types of telemetry one can use, and which ones are important for one application depends both on the nature of that application, and who operates them.

nivertech 5y ago

Regular logs (text files) > Detailed Process Logs (DB) > Audits (read-only/tamper-proof) What's the difference between Regular logs, Detailed Process Logs and Audits? Are Audits more compliance related, while detailed logs are business only? It's not clear to me why to have a special detailed process log, if you can you can write important data to Audits, and less important to regular logs, i.e. once can use different log levels for business data vs engineering/debug data. Can you please give an example?

csours 5y ago
Regular logs are just whatever your app logs normally (engineering/debug). Process logs are business processes that you need to report on or answer for later. Audits are another business layer for safety or life critical data, or adversarial data (handling money based transactions). They could be compliance related. Auditing is a three step process - Ask a question, figure out how to answer it, and make sure you've answered it. So specifically, if you have a magic bean shop and you want to log transfer of the beans, that would be a process log. Then at the end of the day, you would send a manifest of all the beans sent that day. The audit process would look at the final location of the beans, and also at the process log to answer your audit questions. --- I think a lot of people make a hard distinction between business logging and engineering logging, and I'm not sure that makes sense.