Bag of words and TF-IDF

Bag-of-words counts each word in a document. TF-IDF also lowers the weight of words that appear everywhere.

• TF = count of the word ÷ words in the document.
• IDF = log(documents ÷ documents containing the word).
• Word order is lost: 'dog bites man' = 'man bites dog'.

import math
tf = 2 / 10                  # 2 times in a 10-word document
idf = math.log10(100 / 10)   # in 10 of 100 documents
print(round(tf * idf, 2))

0.2

Remember: TF-IDF + logistic regression is a strong, cheap baseline.

Common mistake: Expecting bag-of-words to understand meaning or word order.