4 ms·

Cool video. I have no idea what it's describing though. Can anyone generalize it?

by bprater 16y ago

deleted 16y ago
[deleted]
nas 16y ago
How about more concrete? It could be applied to a pattern recognition problem. For example, classifying email messages as spam and not-spam. The dots in 2-D space represent some characteristics of the message (a poor example would be number of words, average number of letters in each word).
j-g-faustus 16y ago
Classification. The goal is to find the simplest description that will separate two sets of data (here: the points inside the circle from the ones outside). Linear classifiers - in 2D, a straight line that separates the two groups - are preferred when possible since linear is as simple as it gets. http://en.wikipedia.org/wiki/Linear_classifier http://en.wikipedia.org/wiki/Linear_classifier The video shows how a classification task that cannot be handled by a linear classifier in 2D can still be described by a linear equation (a flat plane) by transforming the data to 3D.
lotharbot 16y ago
It's describing a technique for classifying data into one of two categories (A or B) statistically, based on existing observations of data from categories A and B. If we could just draw a straight line or plane that separated all of the A observations from the B observations (that is, A and B are "linearly separable"), keeping as far away from both as possible, we could say that any observation that hits the line is 50/50, and anything on one side or the other has a particular statistical change of being either A or B depending on how far away it is. (The fact that it's straight makes the statistics easy.) But not all data sets can be separated easily with a straight line or plane; sometimes they're best separated with some kind of curve ("non-linearly separable"). The "trick" here is to create a higher-dimensional curved surface -- basically twisting our original space where A and B are -- in such a way that now we can make a straight cut between A and B on the twisted surface. The way our cut intersects the new curved surface corresponds to the best separating curve in the original space. (The hard part is picking the mathematical formula or "kernel" to create the twisted surface.) So, for example, in the video all of the A points were outside of a particular circle and all of the B points were inside the circle. We create the curved surface above and copy all of the A and B points over. This allows us to separate A from B with a straight plane; its intersection with our curved space corresponds to a circle in the original space. See also: http://en.wikipedia.org/wiki/Support_vector_machine http://en.wikipedia.org/wiki/Support_vector_machine
smanek 16y ago
It's been a few years, but I'll take a shot. 1. SVMs are useful for categorizing items. 2. SVMs can only categorize on 'linearly separable' sets. That is, if you were to plot the items from the two sets on a plane, it's possible to distinguish the two sets by drawing a line on the graph and everything above the line is in one set, while everything below is in the other. 3. The kernel trick is adding dimensions to a problem, so even though the problem isn't linearly separable in two dimensions, it might be in 3 by setting the right values in your extra dimension.
seiji 16y ago
Hand-wavey non-math details: Imagine a wall with illuminated colored shadows positioned randomly on it. The colored shadows are created by semi-transparent colored balls some distance from the wall. Each set of balls of the same color are the same distance from the wall (e.g. the red ones are 1m back, the blue ones are 2m back, etc). Now, you are just looking at the wall. You don't know there is anything not on the wall (i.e. the higher dimensional objects projecting the pattern on the two dimensional surface). You just see colors interspersed seemingly randomly. In order to make sense of all the two dimensional randomness, you can up-project the points to a higher dimensional space (i.e. try to re-create what projected them down in the first place). That's the kernel trick. You're projecting lower dimensional data into a higher dimensional space. The goal of all this is to make your data linearly separable (i.e. so you can draw a line (2d) or plane (3d) or hyperplane (4d+) between you classes (colors) of data). You can't separate the colors when they are random points on the wall, but you sure a heck can separate the colors when you see the colored balls grouped by individual color at specific offsets from the wall. That's a very simple 2d to 3d projection where you would draw a plane to separate your colors. For more interesting classes of data, you'll probably end up in much higher dimensional spaces.

Cool video. I have no idea what it's describing though. Can anyone generalize it?

by bprater 16y ago

Cool video. I have no idea what it's describing though. Can anyone generalize it?

deleted 16y ago

[deleted]

nas 16y ago

How about more concrete? It could be applied to a pattern recognition problem. For example, classifying email messages as spam and not-spam. The dots in 2-D space represent some characteristics of the message (a poor example would be number of words, average number of letters in each word).

j-g-faustus 16y ago

Classification. The goal is to find the simplest description that will separate two sets of data (here: the points inside the circle from the ones outside). Linear classifiers - in 2D, a straight line that separates the two groups - are preferred when possible since linear is as simple as it gets. http://en.wikipedia.org/wiki/Linear_classifier http://en.wikipedia.org/wiki/Linear_classifier The video shows how a classification task that cannot be handled by a linear classifier in 2D can still be described by a linear equation (a flat plane) by transforming the data to 3D.

lotharbot 16y ago

It's describing a technique for classifying data into one of two categories (A or B) statistically, based on existing observations of data from categories A and B. If we could just draw a straight line or plane that separated all of the A observations from the B observations (that is, A and B are "linearly separable"), keeping as far away from both as possible, we could say that any observation that hits the line is 50/50, and anything on one side or the other has a particular statistical change of being either A or B depending on how far away it is. (The fact that it's straight makes the statistics easy.) But not all data sets can be separated easily with a straight line or plane; sometimes they're best separated with some kind of curve ("non-linearly separable"). The "trick" here is to create a higher-dimensional curved surface -- basically twisting our original space where A and B are -- in such a way that now we can make a straight cut between A and B on the twisted surface. The way our cut intersects the new curved surface corresponds to the best separating curve in the original space. (The hard part is picking the mathematical formula or "kernel" to create the twisted surface.) So, for example, in the video all of the A points were outside of a particular circle and all of the B points were inside the circle. We create the curved surface above and copy all of the A and B points over. This allows us to separate A from B with a straight plane; its intersection with our curved space corresponds to a circle in the original space. See also: http://en.wikipedia.org/wiki/Support_vector_machine http://en.wikipedia.org/wiki/Support_vector_machine

smanek 16y ago

It's been a few years, but I'll take a shot. 1. SVMs are useful for categorizing items. 2. SVMs can only categorize on 'linearly separable' sets. That is, if you were to plot the items from the two sets on a plane, it's possible to distinguish the two sets by drawing a line on the graph and everything above the line is in one set, while everything below is in the other. 3. The kernel trick is adding dimensions to a problem, so even though the problem isn't linearly separable in two dimensions, it might be in 3 by setting the right values in your extra dimension.

seiji 16y ago

Hand-wavey non-math details: Imagine a wall with illuminated colored shadows positioned randomly on it. The colored shadows are created by semi-transparent colored balls some distance from the wall. Each set of balls of the same color are the same distance from the wall (e.g. the red ones are 1m back, the blue ones are 2m back, etc). Now, you are just looking at the wall. You don't know there is anything not on the wall (i.e. the higher dimensional objects projecting the pattern on the two dimensional surface). You just see colors interspersed seemingly randomly. In order to make sense of all the two dimensional randomness, you can up-project the points to a higher dimensional space (i.e. try to re-create what projected them down in the first place). That's the kernel trick. You're projecting lower dimensional data into a higher dimensional space. The goal of all this is to make your data linearly separable (i.e. so you can draw a line (2d) or plane (3d) or hyperplane (4d+) between you classes (colors) of data). You can't separate the colors when they are random points on the wall, but you sure a heck can separate the colors when you see the colored balls grouped by individual color at specific offsets from the wall. That's a very simple 2d to 3d projection where you would draw a plane to separate your colors. For more interesting classes of data, you'll probably end up in much higher dimensional spaces.