Text

Text Tokenizer

Split text into words , sentences , paragraphs or lines . Every token is listed and numbered, with length statistics and a frequency breakdown of anything that repeats.

Load
Split by
Words
33
tokens found
Unique
28
85% distinct
Avg length
4.2
characters per token
Total chars
137
excluding separators
Shortest
A
1 chars
Longest
paragraph
9 chars
Tokens
1The2quick3brown4fox5jumps6over7the8lazy9dog10The11dog12barks13Pack14my15box16with17five18dozen19liquor20jugs21How22vexingly23quick24daft25zebras26jump27A28second29line30in31the32same33paragraph
Most frequentcase-insensitive · only tokens appearing more than once
the
4
dog
2
quick
2
Share

marduc812

© 202620260824_1c411cc