tokenize

== English == === Alternative forms === tokenise === Etymology === From token +‎ -ize. === Pronunciation === (General American) IPA(key): /ˈtoʊ.kən.aɪz/ === Verb === tokenize (third-person singular simple present tokenizes, present participle tokenizing, simple past and past participle tokenized) (transitive, computing) To reduce to a token or set of tokens by lexical analysis. Coordinate terms: codify, encode (transitive, computing) To substitute sensitive data with meaningless placeholders. (transitive) To treat as a token minority. ==== Derived terms ==== tokenizable tokenization tokenizer ==== Related terms ==== tokenism === Anagrams === ketonize