MarqoomText
Definitions, names, and the boundaries between concepts.
Character
Definition: An abstract unit of text in a digital encoding; it need not correspond to a single linguistic letter.
Purpose: Used where an offset, a length or a comparison must be exact: text-path tests, search indexes and the allowlist of permitted characters.
| field | value |
|---|---|
code | character |
plural | characters |
kind | unit |
| Origin | borrowed |
| Tier | core |
| Version | v0.1 |
- A character is a unit of encoding; a letter is a linguistic unit, and one letter may be carried by more than one character.
- A character is not a glyph: the glyph is produced by the font, the character is carried by the text.
Related: letter, glyph, codepoint, grapheme
Codepoint
Definition: A numeric value defined by an encoding standard such as Unicode.
Purpose: Used where the exact encoding of a character or mark matters: normalisation rules, font mappings and the allowlist of permitted codepoints.
| field | value |
|---|---|
code | codepoint |
plural | codepoints |
kind | unit |
| Origin | borrowed |
| Tier | core |
| Version | v0.1 |
Glyph
Definition: The visual shape a font produces to represent a letter, a character, or a group of them.
Purpose: Used for fonts, rasm, rendering and the placement of visual shapes.
| field | value |
|---|---|
code | glyph |
plural | glyphs |
kind | unit |
| Origin | borrowed |
| Tier | core |
| Version | v0.1 |
- A glyph is produced by the font; it is neither the letter nor the character, and one character may be drawn by many glyphs.
Related: character, letter, font
Grapheme
Definition: A written unit that a user perceives as one unit, which may consist of more than one codepoint.
Purpose: Used for visual segmentation, editing and text selection, where the codepoint is not the right unit.
| field | value |
|---|---|
code | grapheme |
plural | graphemes |
kind | unit |
| Origin | borrowed |
| Tier | core |
| Version | v0.1 |
Letter — الحرف
Definition: A letter of the alphabet as a linguistic unit of writing.
Purpose: Used for data that deals with linguistic letters, without confusing them with their digital or visual representations.
| field | value |
|---|---|
code | letter |
plural | letters |
kind | unit |
| Origin | borrowed |
| Tier | core |
| Version | v0.1 |
| Vocalized | الحَرْف |
| Other spellings | harf |
- A letter is a linguistic unit; a character is a unit of encoding, and one letter may be written with one character or more.
- A letter is a unit of writing; harf al-mana is a part of speech.
Related: character, glyph, particle, ijam, word
Sources:
Token
Definition: A unit produced by segmenting the text according to a declared method. It may equal a word, be part of one, or span more than one.
Purpose: Used to tie automated analysis to the text where the word boundary is not the segmentation boundary, and to keep what a segmenter produces distinct from what a reader counts as a word.
| field | value |
|---|---|
code | token |
plural | tokens |
kind | unit |
| Origin | borrowed |
| Tier | core |
| Version | v0.1 |
- A word is a unit the reader recognises; a token is a unit a segmentation method produces, so changing the method changes the number of tokens and not the number of words.
Word — الكلمة
Definition: A unit of the text that a reader recognises as one word, independent of how a segmenter splits the text.
Purpose: Used to attach word-level data: root, morphology, irab, tajwid, audio alignment and visual position.
| field | value |
|---|---|
code | word |
plural | words |
kind | unit |
| Origin | borrowed |
| Tier | core |
| Version | v0.1 |
| Vocalized | الكَلِمَة |
| Other spellings | kalimah |
- A word is a unit the reader recognises; a token is a unit a segmentation method produces, so changing the method changes the number of tokens and not the number of words.
Related: token, morpheme, letter, word_key, ayah, irab, mutashabihat, word_by_word_translation, word_meanings, word_timing