3 ms·

I see the ability to use machine learning algorithms to take the digital exhaust of a system and post process it into an algorithm for anticipating the outputs

I see the ability to use machine learning algorithms to take the digital exhaust of a system and post process it into an algorithm for anticipating the outputs of that process to be quite powerful, and is having an impact on a lot of systems today. Could you elaborate on that ? What does that mean ?

ChuckMcM 8y ago
Sure, here is a trivial example of the concept. Let's say you have a system of comments like you do here on HN with upvotes and users and topics. Let's pretend that system just runs "open loop" as it were and things are posted and up voted and down voted and commented on. The operation of the system leaves behind a trail of its activity (some explicit like new comments, some implicit like karma changes). Collectively this data set is the "exhaust" or "wake" (in the sense of a boat wake that shows up when a boat passes by) of the system running. A machine learning system can process that data set and make predictions about future behavior of the system. People do this all the time to predict things like "when is the best time to post" "what happens when something makes the front page" etc. Internally predictions can be made about vote fraud or spam or what have you. You can use machine learning in an active feedback mode where its predictive outputs can then be tied to administrative controls to achieve a desired improvement. Let's say you wanted to remove the 'front page bias' of articles posted at a particular time. If you have a tool for boosting or demoting stories in a fairly fine grained way, and you can deduce the natural tendency of stories based on submission time to reach the front page, you can have your code 'boost' stories that are submitted during "weak" times and 'attenuate' stories that are submitted during "strong" times. The effect on the system is surfacing a wider variety of stories and a better user experience (for readers at least). At the meta level, this is no different than control system theory with an error amplifier and a summation amplifier tied into a feedback loop. But Machine Learning brings with it the ability to characterize, and base feedback on, variables that aren't really visible to people nor completely independent. Coding the variables, their interrelationships and their impact on the output is a time consuming and non-economic thing to do, so it isn't done. But with machine learning, the ability to do that analysis algorithmically and just train for the outputs of interest, is quite inexpensive. Makes it a very powerful hammer in everything from language translation to truck load planning to autonomous driving.

ChuckMcM 8y ago

Sure, here is a trivial example of the concept. Let's say you have a system of comments like you do here on HN with upvotes and users and topics. Let's pretend that system just runs "open loop" as it were and things are posted and up voted and down voted and commented on. The operation of the system leaves behind a trail of its activity (some explicit like new comments, some implicit like karma changes). Collectively this data set is the "exhaust" or "wake" (in the sense of a boat wake that shows up when a boat passes by) of the system running. A machine learning system can process that data set and make predictions about future behavior of the system. People do this all the time to predict things like "when is the best time to post" "what happens when something makes the front page" etc. Internally predictions can be made about vote fraud or spam or what have you. You can use machine learning in an active feedback mode where its predictive outputs can then be tied to administrative controls to achieve a desired improvement. Let's say you wanted to remove the 'front page bias' of articles posted at a particular time. If you have a tool for boosting or demoting stories in a fairly fine grained way, and you can deduce the natural tendency of stories based on submission time to reach the front page, you can have your code 'boost' stories that are submitted during "weak" times and 'attenuate' stories that are submitted during "strong" times. The effect on the system is surfacing a wider variety of stories and a better user experience (for readers at least). At the meta level, this is no different than control system theory with an error amplifier and a summation amplifier tied into a feedback loop. But Machine Learning brings with it the ability to characterize, and base feedback on, variables that aren't really visible to people nor completely independent. Coding the variables, their interrelationships and their impact on the output is a time consuming and non-economic thing to do, so it isn't done. But with machine learning, the ability to do that analysis algorithmically and just train for the outputs of interest, is quite inexpensive. Makes it a very powerful hammer in everything from language translation to truck load planning to autonomous driving.