Turn text into numbers a model can use: clean, split, count or embed.
- CleanLowercase, strip noise, normalise.
- TokeniseSplit text into words or sub-words.
- VectoriseBag of words, TF-IDF or embeddings.
- ModelClassify, tag or search the vectors.
Explain 1/5
Source: Speech and Language Processing (Jurafsky and Martin)