2 ms·
Let’s say I wanted to classify email subject lines as “likely spam” vs “likely legitimate”. As input, I have billions of examples of each. Would BERT help me b
by ttul 8y ago
Let’s say I wanted to classify email subject lines as “likely spam” vs “likely legitimate”. As input, I have billions of examples of each.
Would BERT help me by first enabling me to transform the input subject lines into vectors in a high dimensional vector space which could then be the inputs into a relatively shallow network that does the classification?
- brokensegue 8y agosure
- jean- 8y agoAbsolutely. If your only input is the subject line, then you're dealing with a single-sentence classification task. You'd need to take the "class label" vector from the top layer of BERT (labelled "C" in Fig 2b of the paper) and then feed that to your own classifier. For the experiments in paper they actually fine-tuned BERT on the downstream task, but I reckon you'd get acceptable performance by just keeping it fixed and using its outputs as features for a shallow classifier.