|
Data.Text | Portability | GHC | Stability | experimental | Maintainer | bos@serpentine.com, rtomharper@googlemail.com,
duncan@haskell.org |
|
|
|
|
|
Description |
A time and space-efficient implementation of Unicode text using
packed Word16 arrays. Suitable for performance critical use, both
in terms of large data quantities and high speed.
This module is intended to be imported qualified, to avoid name
clashes with Prelude functions, e.g.
import qualified Data.Text as T
|
|
Synopsis |
|
data Text | | pack :: String -> Text | | unpack :: Text -> String | | singleton :: Char -> Text | | empty :: Text | | cons :: Char -> Text -> Text | | snoc :: Text -> Char -> Text | | append :: Text -> Text -> Text | | uncons :: Text -> Maybe (Char, Text) | | head :: Text -> Char | | last :: Text -> Char | | tail :: Text -> Text | | init :: Text -> Text | | null :: Text -> Bool | | length :: Text -> Int | | compareLength :: Text -> Int -> Ordering | | map :: (Char -> Char) -> Text -> Text | | intercalate :: Text -> [Text] -> Text | | intersperse :: Char -> Text -> Text | | transpose :: [Text] -> [Text] | | reverse :: Text -> Text | | replace :: Text -> Text -> Text -> Text | | toCaseFold :: Text -> Text | | toLower :: Text -> Text | | toUpper :: Text -> Text | | justifyLeft :: Int -> Char -> Text -> Text | | justifyRight :: Int -> Char -> Text -> Text | | center :: Int -> Char -> Text -> Text | | foldl :: (a -> Char -> a) -> a -> Text -> a | | foldl' :: (a -> Char -> a) -> a -> Text -> a | | foldl1 :: (Char -> Char -> Char) -> Text -> Char | | foldl1' :: (Char -> Char -> Char) -> Text -> Char | | foldr :: (Char -> a -> a) -> a -> Text -> a | | foldr1 :: (Char -> Char -> Char) -> Text -> Char | | concat :: [Text] -> Text | | concatMap :: (Char -> Text) -> Text -> Text | | any :: (Char -> Bool) -> Text -> Bool | | all :: (Char -> Bool) -> Text -> Bool | | maximum :: Text -> Char | | minimum :: Text -> Char | | scanl :: (Char -> Char -> Char) -> Char -> Text -> Text | | scanl1 :: (Char -> Char -> Char) -> Text -> Text | | scanr :: (Char -> Char -> Char) -> Char -> Text -> Text | | scanr1 :: (Char -> Char -> Char) -> Text -> Text | | mapAccumL :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text) | | mapAccumR :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text) | | replicate :: Int -> Text -> Text | | unfoldr :: (a -> Maybe (Char, a)) -> a -> Text | | unfoldrN :: Int -> (a -> Maybe (Char, a)) -> a -> Text | | take :: Int -> Text -> Text | | drop :: Int -> Text -> Text | | takeWhile :: (Char -> Bool) -> Text -> Text | | dropWhile :: (Char -> Bool) -> Text -> Text | | dropWhileEnd :: (Char -> Bool) -> Text -> Text | | dropAround :: (Char -> Bool) -> Text -> Text | | strip :: Text -> Text | | stripStart :: Text -> Text | | stripEnd :: Text -> Text | | splitAt :: Int -> Text -> (Text, Text) | | spanBy :: (Char -> Bool) -> Text -> (Text, Text) | | break :: Text -> Text -> (Text, Text) | | breakEnd :: Text -> Text -> (Text, Text) | | breakBy :: (Char -> Bool) -> Text -> (Text, Text) | | group :: Text -> [Text] | | groupBy :: (Char -> Char -> Bool) -> Text -> [Text] | | inits :: Text -> [Text] | | tails :: Text -> [Text] | | split :: Text -> Text -> [Text] | | splitBy :: (Char -> Bool) -> Text -> [Text] | | chunksOf :: Int -> Text -> [Text] | | lines :: Text -> [Text] | | words :: Text -> [Text] | | unlines :: [Text] -> Text | | unwords :: [Text] -> Text | | isPrefixOf :: Text -> Text -> Bool | | isSuffixOf :: Text -> Text -> Bool | | isInfixOf :: Text -> Text -> Bool | | stripPrefix :: Text -> Text -> Maybe Text | | stripSuffix :: Text -> Text -> Maybe Text | | filter :: (Char -> Bool) -> Text -> Text | | find :: Text -> Text -> [(Text, Text)] | | findBy :: (Char -> Bool) -> Text -> Maybe Char | | partitionBy :: (Char -> Bool) -> Text -> (Text, Text) | | index :: Text -> Int -> Char | | findIndex :: (Char -> Bool) -> Text -> Maybe Int | | count :: Text -> Text -> Int | | zip :: Text -> Text -> [(Char, Char)] | | zipWith :: (Char -> Char -> Char) -> Text -> Text -> Text |
|
|
|
Strict vs lazy types
|
|
This package provides both strict and lazy Text types. The
strict type is provided by the Data.Text package, while the lazy
type is provided by the Data.Text.Lazy package. Internally, the
lazy Text type consists of a list of strict chunks.
The strict Text type requires that an entire string fit into
memory at once. The lazy Text type is capable of streaming
strings that are larger than memory using a small memory footprint.
In many cases, the overhead of chunked streaming makes the lazy
Text type slower than its strict counterpart, but this is not
always the case. Sometimes, the time complexity of a function in
one module may be different from the other, due to their differing
internal structures.
Each module provides an almost identical API, with the main
difference being that the strict module uses Int values for
lengths and counts, while the lazy module uses Int64 lengths.
|
|
Fusion
|
|
Most of the functions in this module are subject to fusion,
meaning that a pipeline of such functions will usually allocate at
most one Text value.
As an example, consider the following pipeline:
import Data.Text as T
import Data.Text.Encoding as E
countChars :: ByteString -> Int
countChars = T.length . T.toUpper . E.decodeUtf8
From the type signatures involved, this looks like it should
allocate one ByteString value, and two Text values. However,
when a module is compiled with optimisation enabled under GHC, the
two intermediate Text values will be optimised away, and the
function will be compiled down to a single loop over the source
ByteString.
Functions that can be fused by the compiler are marked with the
phrase "Subject to fusion".
|
|
Types
|
|
|
A space efficient, packed, unboxed Unicode text type.
| Instances | |
|
|
Creation and elimination
|
|
|
O(n) Convert a String into a Text. Subject to fusion.
|
|
|
O(n) Convert a Text into a String. Subject to fusion.
|
|
|
O(1) Convert a character into a Text.
Subject to fusion.
|
|
|
O(1) The empty Text.
|
|
Basic interface
|
|
|
O(n) Adds a character to the front of a Text. This function
is more costly than its List counterpart because it requires
copying a new array. Subject to fusion.
|
|
|
O(n) Adds a character to the end of a Text. This copies the
entire array in the process, unless fused. Subject to fusion.
|
|
|
O(n) Appends one Text to the other by copying both of them
into a new Text. Subject to fusion.
|
|
|
O(1) Returns the first character and rest of a Text, or
Nothing if empty. Subject to fusion.
|
|
|
O(1) Returns the first character of a Text, which must be
non-empty. Subject to fusion.
|
|
|
O(1) Returns the last character of a Text, which must be
non-empty. Subject to fusion.
|
|
|
O(1) Returns all characters after the head of a Text, which
must be non-empty. Subject to fusion.
|
|
|
O(1) Returns all but the last character of a Text, which must
be non-empty. Subject to fusion.
|
|
|
O(1) Tests whether a Text is empty or not. Subject to
fusion.
|
|
|
O(n) Returns the number of characters in a Text.
Subject to fusion.
|
|
|
O(n) Compare the count of characters in a Text to a number.
Subject to fusion.
This function gives the same answer as comparing against the result
of length, but can short circuit if the count of characters is
greater than the number, and hence be more efficient.
|
|
Transformations
|
|
|
O(n) map f t is the Text obtained by applying f to
each element of t. Subject to fusion.
|
|
|
O(n) The intercalate function takes a Text and a list of
Texts and concatenates the list after interspersing the first
argument between each element of the list.
|
|
|
O(n) The intersperse function takes a character and places it
between the characters of a Text. Subject to fusion.
|
|
|
O(n) The transpose function transposes the rows and columns
of its Text argument. Note that this function uses pack,
unpack, and the list version of transpose, and is thus not very
efficient.
|
|
|
O(n) Reverse the characters of a string. Subject to fusion.
|
|
|
:: Text | Text to search for
| -> Text | Replacement text
| -> Text | Input text
| -> Text | | O(m+n) Replace every occurrence of one substring with another.
In (unlikely) bad cases, this function's time complexity degrades
towards O(n*m).
|
|
|
Case conversion
|
|
When case converting Text values, do not use combinators like
map toUpper to case convert each character of a string
individually, as this gives incorrect results according to the
rules of some writing systems. The whole-string case conversion
functions from this module, such as toUpper, obey the correct
case conversion rules. As a result, these functions may map one
input character to two or three output characters. For examples,
see the documentation of each function.
Note: In some languages, case conversion is a locale- and
context-dependent operation. The case conversion functions in this
module are not locale sensitive. Programs that require locale
sensitivity should use appropriate versions of the case mapping
functions from the text-icu package.
|
|
|
O(n) Convert a string to folded case. This function is mainly
useful for performing caseless (also known as case insensitive)
string comparisons.
A string x is a caseless match for a string y if and only if:
toCaseFold x == toCaseFold y The result string may be longer than the input string, and may
differ from applying toLower to the input string. For instance,
the Armenian small ligature "ﬓ" (men now, U+FB13) is case
folded to the sequence "մ" (men, U+0574) followed by
"ն" (now, U+0576), while the Greek "µ" (micro sign,
U+00B5) is case folded to "μ" (small letter mu, U+03BC)
instead of itself.
|
|
|
O(n) Convert a string to lower case, using simple case
conversion. The result string may be longer than the input string.
For instance, "İ" (Latin capital letter I with dot above,
U+0130) maps to the sequence "i" (Latin small letter i, U+0069) followed
by " ̇" (combining dot above, U+0307).
|
|
|
O(n) Convert a string to upper case, using simple case
conversion. The result string may be longer than the input string.
For instance, the German "ß" (eszett, U+00DF) maps to the
two-letter sequence "SS".
|
|
Justification
|
|
|
O(n) Left-justify a string to the given length, using the
specified fill character on the right. Subject to fusion. Examples:
justifyLeft 7 'x' "foo" == "fooxxxx"
justifyLeft 3 'x' "foobar" == "foobar"
|
|
|
O(n) Right-justify a string to the given length, using the
specified fill character on the left. Examples:
justifyRight 7 'x' "bar" == "xxxxbar"
justifyRight 3 'x' "foobar" == "foobar"
|
|
|
O(n) Center a string to the given length, using the
specified fill character on either side. Examples:
center 8 'x' "HS" = "xxxHSxxx"
|
|
Folds
|
|
|
O(n) foldl, applied to a binary operator, a starting value
(typically the left-identity of the operator), and a Text,
reduces the Text using the binary operator, from left to right.
Subject to fusion.
|
|
|
O(n) A strict version of foldl. Subject to fusion.
|
|
|
O(n) A variant of foldl that has no starting value argument,
and thus must be applied to a non-empty Text. Subject to fusion.
|
|
|
O(n) A strict version of foldl1. Subject to fusion.
|
|
|
O(n) foldr, applied to a binary operator, a starting value
(typically the right-identity of the operator), and a Text,
reduces the Text using the binary operator, from right to left.
Subject to fusion.
|
|
|
O(n) A variant of foldr that has no starting value argument,
and thust must be applied to a non-empty Text. Subject to
fusion.
|
|
Special folds
|
|
|
O(n) Concatenate a list of Texts.
|
|
|
O(n) Map a function over a Text that results in a Text, and
concatenate the results.
|
|
|
O(n) any p t determines whether any character in the
Text t satisifes the predicate p. Subject to fusion.
|
|
|
O(n) all p t determines whether all characters in the
Text t satisify the predicate p. Subject to fusion.
|
|
|
O(n) maximum returns the maximum value from a Text, which
must be non-empty. Subject to fusion.
|
|
|
O(n) minimum returns the minimum value from a Text, which
must be non-empty. Subject to fusion.
|
|
Construction
|
|
Scans
|
|
|
O(n) scanl is similar to foldl, but returns a list of
successive reduced values from the left. Subject to fusion.
scanl f z [x1, x2, ...] == [z, z `f` x1, (z `f` x1) `f` x2, ...]
Note that
last (scanl f z xs) == foldl f z xs.
|
|
|
O(n) scanl1 is a variant of scanl that has no starting
value argument. Subject to fusion.
scanl1 f [x1, x2, ...] == [x1, x1 `f` x2, ...]
|
|
|
O(n) scanr is the right-to-left dual of scanl.
scanr f v == reverse . scanl (flip f) v . reverse
|
|
|
O(n) scanr1 is a variant of scanr that has no starting
value argument. Subject to fusion.
|
|
Accumulating maps
|
|
|
O(n) Like a combination of map and foldl'. Applies a
function to each element of a Text, passing an accumulating
parameter from left to right, and returns a final Text.
|
|
|
The mapAccumR function behaves like a combination of map and
a strict foldr; it applies a function to each element of a
Text, passing an accumulating parameter from right to left, and
returning a final value of this accumulator together with the new
Text.
|
|
Generation and unfolding
|
|
|
O(n*m) replicate n t is a Text consisting of the input
t repeated n times.
|
|
|
O(n), where n is the length of the result. The unfoldr
function is analogous to the List unfoldr. unfoldr builds a
Text from a seed value. The function takes the element and
returns Nothing if it is done producing the Text, otherwise
Just (a,b). In this case, a is the next Char in the
string, and b is the seed value for further production. Subject
to fusion.
|
|
|
O(n) Like unfoldr, unfoldrN builds a Text from a seed
value. However, the length of the result should be limited by the
first argument to unfoldrN. This function is more efficient than
unfoldr when the maximum length of the result is known and
correct, otherwise its performance is similar to unfoldr. Subject
to fusion.
|
|
Substrings
|
|
Breaking strings
|
|
|
O(n) take n, applied to a Text, returns the prefix of the
Text of length n, or the Text itself if n is greater than
the length of the Text. Subject to fusion.
|
|
|
O(n) drop n, applied to a Text, returns the suffix of the
Text after the first n characters, or the empty Text if n
is greater than the length of the Text. Subject to fusion.
|
|
|
O(n) takeWhile, applied to a predicate p and a Text,
returns the longest prefix (possibly empty) of elements that
satisfy p. Subject to fusion.
|
|
|
O(n) dropWhile p t returns the suffix remaining after
takeWhile p t. Subject to fusion.
|
|
|
O(n) dropWhileEnd p t returns the prefix remaining after
dropping characters that fail the predicate p from the end of
t. Subject to fusion.
Examples:
dropWhileEnd (=='.') "foo..." == "foo"
|
|
|
O(n) dropAround p t returns the substring remaining after
dropping characters that fail the predicate p from both the
beginning and end of t. Subject to fusion.
|
|
|
O(n) Remove leading and trailing white space from a string.
Equivalent to:
dropAround isSpace
|
|
|
O(n) Remove leading white space from a string. Equivalent to:
dropWhile isSpace
|
|
|
O(n) Remove trailing white space from a string. Equivalent to:
dropWhileEnd isSpace
|
|
|
O(n) splitAt n t returns a pair whose first element is a
prefix of t of length n, and whose second is the remainder of
the string. It is equivalent to (take n t, drop n t).
|
|
|
O(n) spanBy, applied to a predicate p and text t, returns
a pair whose first element is the longest prefix (possibly empty)
of t of elements that satisfy p, and whose second is the
remainder of the list.
|
|
|
O(n+m) Find the first instance of needle (which must be
non-null) in haystack. The first element of the returned tuple
is the prefix of haystack before needle is matched. The second
is the remainder of haystack, starting with the match.
Examples:
break "::" "a::b::c" ==> ("a", "::b::c")
break "/" "foobar" ==> ("foobar", "")
Laws:
append prefix match == haystack
where (prefix, match) = break needle haystack
If you need to break a string by a substring repeatedly (e.g. you
want to break on every instance of a substring), use find
instead, as it has lower startup overhead.
In (unlikely) bad cases, this function's time complexity degrades
towards O(n*m).
|
|
|
O(n+m) Similar to break, but searches from the end of the string.
The first element of the returned tuple is the prefix of haystack
up to and including the last match of needle. The second is the
remainder of haystack, following the match.
breakEnd "::" "a::b::c" ==> ("a::b::", "c")
|
|
|
O(n) breakBy is like spanBy, but the prefix returned is
over elements that fail the predicate p.
|
|
|
O(n) Group characters in a string by equality.
|
|
|
O(n) Group characters in a string according to a predicate.
|
|
|
O(n) Return all initial segments of the given Text, shortest
first.
|
|
|
O(n) Return all final segments of the given Text, longest
first.
|
|
Breaking into many substrings
|
|
Splitting functions in this library do not perform character-wise
copies to create substrings; they just construct new Texts that
are slices of the original.
|
|
|
O(m+n) Break a Text into pieces separated by the first
Text argument, consuming the delimiter. An empty delimiter is
invalid, and will cause an error to be raised.
Examples:
split "\r\n" "a\r\nb\r\nd\r\ne" == ["a","b","d","e"]
split "aaa" "aaaXaaaXaaaXaaa" == ["","X","X","X",""]
split "x" "x" == ["",""]
and
intercalate s . split s == id
split (singleton c) == splitBy (==c)
In (unlikely) bad cases, this function's time complexity degrades
towards O(n*m).
|
|
|
O(n) Splits a Text into components delimited by separators,
where the predicate returns True for a separator element. The
resulting components do not contain the separators. Two adjacent
separators result in an empty component in the output. eg.
splitBy (=='a') "aabbaca" == ["","","bb","c",""]
splitBy (=='a') "" == [""]
|
|
|
O(n) Splits a Text into components of length k. The last
element may be shorter than the other chunks, depending on the
length of the input. Examples:
chunksOf 3 "foobarbaz" == ["foo","bar","baz"]
chunksOf 4 "haskell.org" == ["hask","ell.","org"]
|
|
Breaking into lines and words
|
|
|
O(n) Breaks a Text up into a list of Texts at
newline Chars. The resulting strings do not contain newlines.
|
|
|
O(n) Breaks a Text up into a list of words, delimited by Chars
representing white space.
|
|
|
O(n) Joins lines, after appending a terminating newline to
each.
|
|
|
O(n) Joins words using single space characters.
|
|
Predicates
|
|
|
O(n) The isPrefixOf function takes two Texts and returns
True iff the first is a prefix of the second. This function is
subject to fusion.
|
|
|
O(n) The isSuffixOf function takes two Texts and returns
True iff the first is a suffix of the second.
|
|
|
O(n+m) The isInfixOf function takes two Texts and returns
True iff the first is contained, wholly and intact, anywhere
within the second.
In (unlikely) bad cases, this function's time complexity degrades
towards O(n*m).
|
|
View patterns
|
|
|
O(n) Returns the suffix of the second string if its prefix
matches the first.
Examples:
stripPrefix "foo" "foobar" == Just "bar"
stripPrefix "foo" "quux" == Nothing
This is particularly useful with the ViewPatterns extension to
GHC, as follows:
{-# LANGUAGE ViewPatterns #-}
import Data.Text as T
fnordLength :: Text -> Int
fnordLength (stripPrefix "fnord" -> Just suf) = T.length suf
fnordLength _ = -1
|
|
|
O(n) Returns the prefix of the second string if its suffix
matches the first.
Examples:
stripSuffix "bar" "foobar" == Just "foo"
stripSuffix "foo" "quux" == Nothing
This is particularly useful with the ViewPatterns extension to
GHC, as follows:
{-# LANGUAGE ViewPatterns #-}
import Data.Text as T
quuxLength :: Text -> Int
quuxLength (stripSuffix "quux" -> Just pre) = T.length pre
quuxLength _ = -1
|
|
Searching
|
|
|
O(n) filter, applied to a predicate and a Text,
returns a Text containing those characters that satisfy the
predicate.
|
|
|
:: Text | needle to search for
| -> Text | haystack in which to search
| -> [(Text, Text)] | | O(n+m) Find all non-overlapping instances of needle in
haystack. Each element of the returned list consists of a pair:
- The entire string prior to the kth match (i.e. the prefix)
- The kth match, followed by the remainder of the string
Examples:
find "::" ""
==> []
find "/" "a/b/c/"
==> [("a", "/b/c/"), ("a/b", "/c/"), ("a/b/c", "/")]
In (unlikely) bad cases, this function's time complexity degrades
towards O(n*m).
The needle parameter may not be empty.
|
|
|
|
O(n) The findBy function takes a predicate and a Text, and
returns the first element in matching the predicate, or Nothing
if there is no such element.
|
|
|
O(n) The partitionBy function takes a predicate and a Text,
and returns the pair of Texts with elements which do and do not
satisfy the predicate, respectively; i.e.
partitionBy p t == (filter p t, filter (not . p) t)
|
|
Indexing
|
|
If you think of a Text value as an array of Char values (which
it is not), you run the risk of writing inefficient code.
An idiom that is common in some languages is to find the numeric
offset of a character or substring, then use that number to split
or trim the searched string. With a Text value, this approach
would require two O(n) operations: one to perform the search, and
one to operate from wherever the search ended.
For example, suppose you have a string that you want to split on
the substring "::", such as "foo::bar::quux". Instead of
searching for the index of "::" and taking the substrings
before and after that index, you would instead use find "::".
|
|
|
O(n) Text index (subscript) operator, starting from 0.
|
|
|
O(n) The findIndex function takes a predicate and a Text
and returns the index of the first element in the Text satisfying
the predicate. Subject to fusion.
|
|
|
O(n+m) The count function returns the number of times the
query string appears in the given Text. An empty query string is
invalid, and will cause an error to be raised.
In (unlikely) bad cases, this function's time complexity degrades
towards O(n*m).
|
|
Zipping and unzipping
|
|
|
O(n) zip takes two Texts and returns a list of
corresponding pairs of bytes. If one input Text is short,
excess elements of the longer Text are discarded. This is
equivalent to a pair of unpack operations.
|
|
|
O(n) zipWith generalises zip by zipping with the function
given as the first argument, instead of a tupling function.
|
|
Produced by Haddock version 2.6.0 |