text-0.8.1.0: An efficient packed Unicode text type.Source codeContentsIndex
Data.Text.Lazy
PortabilityGHC
Stabilityexperimental
Maintainerbos@serpentine.com, rtomharper@googlemail.com, duncan@haskell.org
Contents
Creation and elimination
Basic interface
Transformations
Case conversion
Justification
Folds
Special folds
Construction
Scans
Accumulating maps
Generation and unfolding
Substrings
Breaking strings
Breaking into many substrings
Breaking into lines and words
Predicates
View patterns
Searching
Indexing
Zipping and unzipping
Description

A time and space-efficient implementation of Unicode text using lists of packed arrays. This representation is suitable for high performance use and for streaming large quantities of data. It provides a means to manipulate a large body of text without requiring that the entire content be resident in memory.

Some operations, such as concat, append, reverse and cons, have better complexity than their Data.Text equivalents, due to optimisations resulting from the list spine structure. And for other operations lazy Texts are usually within a few percent of strict ones, but with better heap usage. For data larger than available memory, or if you have tight memory constraints, this module will be the only option.

This module is intended to be imported qualified, to avoid name clashes with Prelude functions. eg.

 import qualified Data.Text.Lazy as L
Synopsis
data Text
pack :: String -> Text
unpack :: Text -> String
singleton :: Char -> Text
empty :: Text
fromChunks :: [Text] -> Text
toChunks :: Text -> [Text]
toStrict :: Text -> Text
fromStrict :: Text -> Text
foldrChunks :: (Text -> a -> a) -> a -> Text -> a
foldlChunks :: (a -> Text -> a) -> a -> Text -> a
cons :: Char -> Text -> Text
snoc :: Text -> Char -> Text
append :: Text -> Text -> Text
uncons :: Text -> Maybe (Char, Text)
head :: Text -> Char
last :: Text -> Char
tail :: Text -> Text
init :: Text -> Text
null :: Text -> Bool
length :: Text -> Int64
compareLength :: Text -> Int64 -> Ordering
map :: (Char -> Char) -> Text -> Text
intercalate :: Text -> [Text] -> Text
intersperse :: Char -> Text -> Text
transpose :: [Text] -> [Text]
reverse :: Text -> Text
replace :: Text -> Text -> Text -> Text
toCaseFold :: Text -> Text
toLower :: Text -> Text
toUpper :: Text -> Text
justifyLeft :: Int64 -> Char -> Text -> Text
justifyRight :: Int64 -> Char -> Text -> Text
center :: Int64 -> Char -> Text -> Text
foldl :: (a -> Char -> a) -> a -> Text -> a
foldl' :: (a -> Char -> a) -> a -> Text -> a
foldl1 :: (Char -> Char -> Char) -> Text -> Char
foldl1' :: (Char -> Char -> Char) -> Text -> Char
foldr :: (Char -> a -> a) -> a -> Text -> a
foldr1 :: (Char -> Char -> Char) -> Text -> Char
concat :: [Text] -> Text
concatMap :: (Char -> Text) -> Text -> Text
any :: (Char -> Bool) -> Text -> Bool
all :: (Char -> Bool) -> Text -> Bool
maximum :: Text -> Char
minimum :: Text -> Char
scanl :: (Char -> Char -> Char) -> Char -> Text -> Text
scanl1 :: (Char -> Char -> Char) -> Text -> Text
scanr :: (Char -> Char -> Char) -> Char -> Text -> Text
scanr1 :: (Char -> Char -> Char) -> Text -> Text
mapAccumL :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text)
mapAccumR :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text)
replicate :: Int64 -> Text -> Text
unfoldr :: (a -> Maybe (Char, a)) -> a -> Text
unfoldrN :: Int64 -> (a -> Maybe (Char, a)) -> a -> Text
take :: Int64 -> Text -> Text
drop :: Int64 -> Text -> Text
takeWhile :: (Char -> Bool) -> Text -> Text
dropWhile :: (Char -> Bool) -> Text -> Text
dropWhileEnd :: (Char -> Bool) -> Text -> Text
dropAround :: (Char -> Bool) -> Text -> Text
strip :: Text -> Text
stripStart :: Text -> Text
stripEnd :: Text -> Text
splitAt :: Int64 -> Text -> (Text, Text)
spanBy :: (Char -> Bool) -> Text -> (Text, Text)
break :: Text -> Text -> (Text, Text)
breakEnd :: Text -> Text -> (Text, Text)
breakBy :: (Char -> Bool) -> Text -> (Text, Text)
group :: Text -> [Text]
groupBy :: (Char -> Char -> Bool) -> Text -> [Text]
inits :: Text -> [Text]
tails :: Text -> [Text]
split :: Text -> Text -> [Text]
splitBy :: (Char -> Bool) -> Text -> [Text]
chunksOf :: Int64 -> Text -> [Text]
lines :: Text -> [Text]
words :: Text -> [Text]
unlines :: [Text] -> Text
unwords :: [Text] -> Text
isPrefixOf :: Text -> Text -> Bool
isSuffixOf :: Text -> Text -> Bool
isInfixOf :: Text -> Text -> Bool
stripPrefix :: Text -> Text -> Maybe Text
stripSuffix :: Text -> Text -> Maybe Text
filter :: (Char -> Bool) -> Text -> Text
find :: Text -> Text -> [(Text, Text)]
findBy :: (Char -> Bool) -> Text -> Maybe Char
partitionBy :: (Char -> Bool) -> Text -> (Text, Text)
index :: Text -> Int64 -> Char
count :: Text -> Text -> Int64
zip :: Text -> Text -> [(Char, Char)]
zipWith :: (Char -> Char -> Char) -> Text -> Text -> Text
Documentation
data Text Source
show/hide Instances
Creation and elimination
pack :: String -> TextSource

O(n) Convert a String into a Text.

This function is subject to array fusion.

unpack :: Text -> StringSource
O(n) Convert a Text into a String. Subject to array fusion.
singleton :: Char -> TextSource
O(1) Convert a character into a Text. Subject to fusion.
empty :: TextSource
Smart constructor for Empty.
fromChunks :: [Text] -> TextSource
O(c) Convert a list of strict Texts into a lazy Text.
toChunks :: Text -> [Text]Source
O(n) Convert a lazy Text into a list of strict Texts.
toStrict :: Text -> TextSource
O(n) Convert a lazy Text into a strict Text.
fromStrict :: Text -> TextSource
O(c) Convert a strict Text into a lazy Text.
foldrChunks :: (Text -> a -> a) -> a -> Text -> aSource
Consume the chunks of a lazy Text with a natural right fold.
foldlChunks :: (a -> Text -> a) -> a -> Text -> aSource
Consume the chunks of a lazy Text with a strict, tail-recursive, accumulating left fold.
Basic interface
cons :: Char -> Text -> TextSource
O(n) Adds a character to the front of a Text. This function is more costly than its List counterpart because it requires copying a new array. Subject to fusion.
snoc :: Text -> Char -> TextSource
O(n) Adds a character to the end of a Text. This copies the entire array in the process, unless fused. Subject to fusion.
append :: Text -> Text -> TextSource
O(n\c)/ Appends one Text to another. Subject to array fusion.
uncons :: Text -> Maybe (Char, Text)Source
O(1) Returns the first character and rest of a Text, or Nothing if empty. Subject to array fusion.
head :: Text -> CharSource
O(1) Returns the first character of a Text, which must be non-empty. Subject to array fusion.
last :: Text -> CharSource
O(1) Returns the last character of a Text, which must be non-empty. Subject to array fusion.
tail :: Text -> TextSource
O(1) Returns all characters after the head of a Text, which must be non-empty. Subject to array fusion.
init :: Text -> TextSource
O(1) Returns all but the last character of a Text, which must be non-empty. Subject to array fusion.
null :: Text -> BoolSource
O(1) Tests whether a Text is empty or not. Subject to array fusion.
length :: Text -> Int64Source
O(n) Returns the number of characters in a Text. Subject to fusion.
compareLength :: Text -> Int64 -> OrderingSource

O(n) Compare the count of characters in a Text to a number. Subject to fusion.

This function gives the same answer as comparing against the result of length, but can short circuit if the count of characters is greater than the number, and hence be more efficient.

Transformations
map :: (Char -> Char) -> Text -> TextSource
O(n) map f t is the Text obtained by applying f to each element of t. Subject to array fusion.
intercalate :: Text -> [Text] -> TextSource
O(n) The intercalate function takes a Text and a list of Texts and concatenates the list after interspersing the first argument between each element of the list.
intersperse :: Char -> Text -> TextSource
O(n) The intersperse function takes a character and places it between the characters of a Text. Subject to array fusion.
transpose :: [Text] -> [Text]Source
O(n) The transpose function transposes the rows and columns of its Text argument. Note that this function uses pack, unpack, and the list version of transpose, and is thus not very efficient.
reverse :: Text -> TextSource
O(n) reverse t returns the elements of t in reverse order.
replaceSource
:: TextText to search for
-> TextReplacement text
-> TextInput text
-> Text

O(m+n) Replace every occurrence of one substring with another.

In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).

Case conversion
With Unicode text, it is incorrect to use combinators like map toUpper to case convert each character of a string individually. Instead, use the whole-string case conversion functions from this module. For correctness in different writing systems, these functions may map one input character to two or three output characters.
toCaseFold :: Text -> TextSource

O(n) Convert a string to folded case. This function is mainly useful for performing caseless (or case insensitive) string comparisons.

A string x is a caseless match for a string y if and only if:

toCaseFold x == toCaseFold y

The result string may be longer than the input string, and may differ from applying toLower to the input string. For instance, the Armenian small ligature men now (U+FB13) is case folded to the bigram men now (U+0574 U+0576), while the micro sign (U+00B5) is case folded to the Greek small letter letter mu (U+03BC) instead of itself.

toLower :: Text -> TextSource
O(n) Convert a string to lower case, using simple case conversion. The result string may be longer than the input string. For instance, the Latin capital letter I with dot above (U+0130) maps to the sequence Latin small letter i (U+0069) followed by combining dot above (U+0307).
toUpper :: Text -> TextSource
O(n) Convert a string to upper case, using simple case conversion. The result string may be longer than the input string. For instance, the German eszett (U+00DF) maps to the two-letter sequence SS.
Justification
justifyLeft :: Int64 -> Char -> Text -> TextSource

O(n) Left-justify a string to the given length, using the specified fill character on the right. Subject to fusion. Examples:

 justifyLeft 7 'x' "foo"    == "fooxxxx"
 justifyLeft 3 'x' "foobar" == "foobar"
justifyRight :: Int64 -> Char -> Text -> TextSource

O(n) Right-justify a string to the given length, using the specified fill character on the left. Examples:

 justifyRight 7 'x' "bar"    == "xxxxbar"
 justifyRight 3 'x' "foobar" == "foobar"
center :: Int64 -> Char -> Text -> TextSource

O(n) Center a string to the given length, using the specified fill character on either side. Examples:

 center 8 'x' "HS" = "xxxHSxxx"
Folds
foldl :: (a -> Char -> a) -> a -> Text -> aSource
O(n) foldl, applied to a binary operator, a starting value (typically the left-identity of the operator), and a Text, reduces the Text using the binary operator, from left to right. Subject to array fusion.
foldl' :: (a -> Char -> a) -> a -> Text -> aSource
O(n) A strict version of foldl. Subject to array fusion.
foldl1 :: (Char -> Char -> Char) -> Text -> CharSource
O(n) A variant of foldl that has no starting value argument, and thus must be applied to a non-empty Text. Subject to array fusion.
foldl1' :: (Char -> Char -> Char) -> Text -> CharSource
O(n) A strict version of foldl1. Subject to array fusion.
foldr :: (Char -> a -> a) -> a -> Text -> aSource
O(n) foldr, applied to a binary operator, a starting value (typically the right-identity of the operator), and a Text, reduces the Text using the binary operator, from right to left. Subject to array fusion.
foldr1 :: (Char -> Char -> Char) -> Text -> CharSource
O(n) A variant of foldr that has no starting value argument, and thust must be applied to a non-empty Text. Subject to array fusion.
Special folds
concat :: [Text] -> TextSource
O(n) Concatenate a list of Texts.
concatMap :: (Char -> Text) -> Text -> TextSource
O(n) Map a function over a Text that results in a Text, and concatenate the results.
any :: (Char -> Bool) -> Text -> BoolSource
O(n) any p t determines whether any character in the Text t satisifes the predicate p. Subject to array fusion.
all :: (Char -> Bool) -> Text -> BoolSource
O(n) all p t determines whether all characters in the Text t satisify the predicate p. Subject to array fusion.
maximum :: Text -> CharSource
O(n) maximum returns the maximum value from a Text, which must be non-empty. Subject to array fusion.
minimum :: Text -> CharSource
O(n) minimum returns the minimum value from a Text, which must be non-empty. Subject to array fusion.
Construction
Scans
scanl :: (Char -> Char -> Char) -> Char -> Text -> TextSource

O(n) scanl is similar to foldl, but returns a list of successive reduced values from the left. This function is subject to array fusion.

 scanl f z [x1, x2, ...] == [z, z `f` x1, (z `f` x1) `f` x2, ...]

Note that

 last (scanl f z xs) == foldl f z xs.
scanl1 :: (Char -> Char -> Char) -> Text -> TextSource

O(n) scanl1 is a variant of scanl that has no starting value argument. This function is subject to array fusion.

 scanl1 f [x1, x2, ...] == [x1, x1 `f` x2, ...]
scanr :: (Char -> Char -> Char) -> Char -> Text -> TextSource

O(n) scanr is the right-to-left dual of scanl.

 scanr f v == reverse . scanl (flip f) v . reverse
scanr1 :: (Char -> Char -> Char) -> Text -> TextSource
O(n) scanr1 is a variant of scanr that has no starting value argument.
Accumulating maps
mapAccumL :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text)Source
O(n) Like a combination of map and foldl'. Applies a function to each element of a Text, passing an accumulating parameter from left to right, and returns a final Text.
mapAccumR :: (a -> Char -> (a, Char)) -> a -> Text -> (a, Text)Source
The mapAccumR function behaves like a combination of map and a strict foldr; it applies a function to each element of a Text, passing an accumulating parameter from right to left, and returning a final value of this accumulator together with the new Text.
Generation and unfolding
replicate :: Int64 -> Text -> TextSource
O(n*m) replicate n t is a Text consisting of the input t repeated n times.
unfoldr :: (a -> Maybe (Char, a)) -> a -> TextSource
O(n), where n is the length of the result. The unfoldr function is analogous to the List unfoldr. unfoldr builds a Text from a seed value. The function takes the element and returns Nothing if it is done producing the Text, otherwise Just (a,b). In this case, a is the next Char in the string, and b is the seed value for further production.
unfoldrN :: Int64 -> (a -> Maybe (Char, a)) -> a -> TextSource
O(n) Like unfoldr, unfoldrN builds a Text from a seed value. However, the length of the result should be limited by the first argument to unfoldrN. This function is more efficient than unfoldr when the maximum length of the result is known and correct, otherwise its performance is similar to unfoldr.
Substrings
Breaking strings
take :: Int64 -> Text -> TextSource
O(n) take n, applied to a Text, returns the prefix of the Text of length n, or the Text itself if n is greater than the length of the Text. Subject to fusion.
drop :: Int64 -> Text -> TextSource
O(n) drop n, applied to a Text, returns the suffix of the Text after the first n characters, or the empty Text if n is greater than the length of the Text. Subject to fusion.
takeWhile :: (Char -> Bool) -> Text -> TextSource
O(n) takeWhile, applied to a predicate p and a Text, returns the longest prefix (possibly empty) of elements that satisfy p. This function is subject to array fusion.
dropWhile :: (Char -> Bool) -> Text -> TextSource
O(n) dropWhile p t returns the suffix remaining after takeWhile p t. This function is subject to array fusion.
dropWhileEnd :: (Char -> Bool) -> Text -> TextSource

O(n) dropWhileEnd p t returns the prefix remaining after dropping characters that fail the predicate p from the end of t. Examples:

 dropWhileEnd (=='.') "foo..." == "foo"
dropAround :: (Char -> Bool) -> Text -> TextSource
O(n) dropAround p t returns the substring remaining after dropping characters that fail the predicate p from both the beginning and end of t. Subject to fusion.
strip :: Text -> TextSource

O(n) Remove leading and trailing white space from a string. Equivalent to:

 dropAround isSpace
stripStart :: Text -> TextSource

O(n) Remove leading white space from a string. Equivalent to:

 dropWhile isSpace
stripEnd :: Text -> TextSource

O(n) Remove trailing white space from a string. Equivalent to:

 dropWhileEnd isSpace
splitAt :: Int64 -> Text -> (Text, Text)Source
O(n) splitAt n t returns a pair whose first element is a prefix of t of length n, and whose second is the remainder of the string. It is equivalent to (take n t, drop n t).
spanBy :: (Char -> Bool) -> Text -> (Text, Text)Source
O(n) spanBy, applied to a predicate p and text t, returns a pair whose first element is the longest prefix (possibly empty) of t of elements that satisfy p, and whose second is the remainder of the list.
break :: Text -> Text -> (Text, Text)Source

O(n+m) Find the first instance of needle (which must be non-null) in haystack. The first element of the returned tuple is the prefix of haystack before needle is matched. The second is the remainder of haystack, starting with the match.

Examples:

 break "::" "a::b::c" ==> ("a", "::b::c")
 break "/" "foobar"   ==> ("foobar", "")

Laws:

 append prefix match == haystack
   where (prefix, match) = break needle haystack

If you need to break a string by a substring repeatedly (e.g. you want to break on every instance of a substring), use find instead, as it has lower startup overhead.

This function is strict in its first argument, and lazy in its second.

In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).

breakEnd :: Text -> Text -> (Text, Text)Source

O(n+m) Similar to break, but searches from the end of the string.

The first element of the returned tuple is the prefix of haystack up to and including the last match of needle. The second is the remainder of haystack, following the match.

 breakEnd "::" "a::b::c" ==> ("a::b::", "c")
breakBy :: (Char -> Bool) -> Text -> (Text, Text)Source
O(n) breakBy is like spanBy, but the prefix returned is over elements that fail the predicate p.
group :: Text -> [Text]Source

The group function takes a Text and returns a list of Texts such that the concatenation of the result is equal to the argument. Moreover, each sublist in the result contains only equal elements. For example,

 group "Mississippi" = ["M","i","ss","i","ss","i","pp","i"]

It is a special case of groupBy, which allows the programmer to supply their own equality test.

groupBy :: (Char -> Char -> Bool) -> Text -> [Text]Source
The groupBy function is the non-overloaded version of group.
inits :: Text -> [Text]Source
O(n) Return all initial segments of the given Text, shortest first.
tails :: Text -> [Text]Source
O(n) Return all final segments of the given Text, longest first.
Breaking into many substrings
Splitting functions in this library do not perform character-wise copies to create substrings; they just construct new Texts that are slices of the original.
splitSource
:: TextText to split on
-> TextInput text
-> [Text]

O(m+n) Break a Text into pieces separated by the first Text argument, consuming the delimiter. An empty delimiter is invalid, and will cause an error to be raised.

Examples:

 split "\r\n" "a\r\nb\r\nd\r\ne" == ["a","b","d","e"]
 split "aaa"  "aaaXaaaXaaaXaaa"  == ["","X","X","X",""]
 split "x"    "x"                == ["",""]

and

 intercalate s . split s         == id
 split (singleton c)             == splitBy (==c)

This function is strict in its first argument, and lazy in its second.

In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).

splitBy :: (Char -> Bool) -> Text -> [Text]Source

O(n) Splits a Text into components delimited by separators, where the predicate returns True for a separator element. The resulting components do not contain the separators. Two adjacent separators result in an empty component in the output. eg.

 splitBy (=='a') "aabbaca" == ["","","bb","c",""]
 splitBy (=='a') []        == [""]
chunksOf :: Int64 -> Text -> [Text]Source

O(n) Splits a Text into components of length k. The last element may be shorter than the other chunks, depending on the length of the input. Examples:

 chunksOf 3 "foobarbaz"   == ["foo","bar","baz"]
 chunksOf 4 "haskell.org" == ["hask","ell.","org"]
Breaking into lines and words
lines :: Text -> [Text]Source
O(n) Breaks a Text up into a list of Texts at newline Chars. The resulting strings do not contain newlines.
words :: Text -> [Text]Source
O(n) Breaks a Text up into a list of words, delimited by Chars representing white space.
unlines :: [Text] -> TextSource
O(n) Joins lines, after appending a terminating newline to each.
unwords :: [Text] -> TextSource
O(n) Joins words using single space characters.
Predicates
isPrefixOf :: Text -> Text -> BoolSource
O(n) The isPrefixOf function takes two Texts and returns True iff the first is a prefix of the second. This function is subject to fusion.
isSuffixOf :: Text -> Text -> BoolSource
O(n) The isSuffixOf function takes two Texts and returns True iff the first is a suffix of the second.
isInfixOf :: Text -> Text -> BoolSource

O(n+m) The isInfixOf function takes two Texts and returns True iff the first is contained, wholly and intact, anywhere within the second.

This function is strict in its first argument, and lazy in its second.

In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).

View patterns
stripPrefix :: Text -> Text -> Maybe TextSource

O(n) Returns the suffix of the second string if its prefix matches the first.

Examples:

 stripPrefix "foo" "foobar" == Just "bar"
 stripPrefix "foo" "quux"   == Nothing

This is particularly useful with the ViewPatterns extension to GHC, as follows:

 {-# LANGUAGE ViewPatterns #-}
 import Data.Text as T

 fnordLength :: Text -> Int
 fnordLength (stripPrefix "fnord" -> Just suf) = T.length suf
 fnordLength _                              = -1
stripSuffix :: Text -> Text -> Maybe TextSource

O(n) Returns the prefix of the second string if its suffix matches the first.

Examples:

 stripSuffix "bar" "foobar" == Just "foo"
 stripSuffix "foo" "quux"   == Nothing

This is particularly useful with the ViewPatterns extension to GHC, as follows:

 {-# LANGUAGE ViewPatterns #-}
 import Data.Text as T

 quuxLength :: Text -> Int
 quuxLength (stripSuffix "quux" -> Just pre) = T.length pre
 quuxLength _                             = -1
Searching
filter :: (Char -> Bool) -> Text -> TextSource
O(n) filter, applied to a predicate and a Text, returns a Text containing those characters that satisfy the predicate.
findSource
:: Textneedle to search for
-> Texthaystack in which to search
-> [(Text, Text)]

O(n+m) Find all non-overlapping instances of needle in haystack. Each element of the returned list consists of a pair:

  • The entire string prior to the kth match (i.e. the prefix)
  • The kth match, followed by the remainder of the string

Examples:

 find "::" ""
 ==> []
 find "/" "a/b/c/"
 ==> [("a", "/b/c/"), ("a/b", "/c/"), ("a/b/c", "/")]

This function is strict in its first argument, and lazy in its second.

In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).

The needle parameter may not be empty.

findBy :: (Char -> Bool) -> Text -> Maybe CharSource
O(n) The findBy function takes a predicate and a Text, and returns the first element in matching the predicate, or Nothing if there is no such element.
partitionBy :: (Char -> Bool) -> Text -> (Text, Text)Source

O(n) The partitionBy function takes a predicate and a Text, and returns the pair of Texts with elements which do and do not satisfy the predicate, respectively; i.e.

 partitionBy p t == (filter p t, filter (not . p) t)
Indexing
index :: Text -> Int64 -> CharSource
O(n) Text index (subscript) operator, starting from 0.
count :: Text -> Text -> Int64Source

O(n+m) The count function returns the number of times the query string appears in the given Text. An empty query string is invalid, and will cause an error to be raised.

In (unlikely) bad cases, this function's time complexity degrades towards O(n*m).

Zipping and unzipping
zip :: Text -> Text -> [(Char, Char)]Source
O(n) zip takes two Texts and returns a list of corresponding pairs of bytes. If one input Text is short, excess elements of the longer Text are discarded. This is equivalent to a pair of unpack operations.
zipWith :: (Char -> Char -> Char) -> Text -> Text -> TextSource
O(n) zipWith generalises zip by zipping with the function given as the first argument, instead of a tupling function.
Produced by Haddock version 2.6.1