AI Tokens
💡AI doesn’t see raw sentences—it sees tokens as numbers. Why Tokens? Unlike the traditional way of storing and retrieving data, the AI Model shall build relationships between the text and generate text with the right context, so it requires a special data structure (Vectors) to store it in