selectolax Changelog¶

Version 1.0.1¶

Add the scripting option to LexborHTMLParser, which parses <noscript> the way a browser with scripting enabled does: its content becomes raw text, so a <noscript> in <head> no longer pushes the head elements after it into <body>. The default is unchanged.
Add template_fragments() for reaching the content of <template> elements which are ignored by HTML5 compliant parsers.
Do not replace <-undef> it not longer possible to get it.
The internal <html> wrapper of an HTML fragment is no longer reported as the parent of the fragment’s top-level nodes.
create_tag() no longer returns None for the tags whose start tag the HTML parser drops (html, head, body, caption, col, colgroup, frame, frameset, tbody, td, tfoot, th, thead, tr).
Fix inner_html covering only the first top-level node of an HTML fragment.
Improve the speed of comment extraction
Improve the speed of len(attrs) by counting the attribute list directly
Fix encoding initialization when encoding=True.

Version 1.0.0¶

This release contains breaking changes.

The main change is that the Modest backend is no longer available. It is outdated and unmaintained, contains bugs, and does not follow modern HTML5 standards.

The second is that parse_fragment() is gone. It re-implemented fragment parsing by guessing whether the input was a document or a fragment; LexborHTMLParser(html, is_fragment=True) already does this properly, so the guessing layer has been removed.

The rest of the changes fix corner cases where the bugs were happening rarely, usually when heavily modifying the tree.

Breaking: Remove the Modest backend. selectolax.parser is now a stub that raises ImportError on import. Use the lexbor backend (from selectolax.lexbor import LexborHTMLParser) instead.
Breaking: Remove parse_fragment().
Breaking: Fix css_matches and any_css_matches missing matches outside the first top-level node of an HTML fragment. They now search the same scope as css.
Breaking: Fix tags and strip_tags doing nothing on an HTML fragment. tags returned an empty list and strip_tags removed nothing, because the fragment was not reachable from the document node.
Breaking: Fix the internal <html> wrapper of an HTML fragment being reported as a match.
Breaking: Fix scripts_contain and script_srcs_contain missing matches outside the first top-level node of an HTML fragment. They now search the same scope as css.
Breaking: Fix scripts_contain and script_srcs_contain answering from another scope’s cached result. The cache is keyed by the node the search is rooted at rather than the node it was called on.
Breaking: Fix select() missing matches outside the first top-level node of an HTML fragment. It now searches the same scope as css.
Breaking: Fix traverse() covering only the first top-level node of an HTML fragment, and yielding nothing when the fragment starts with a text node.
Breaking: Fix attribute_longer_than and any_attribute_longer_than returning inconsistent results
Breaking: attributes and attrs now report an attribute’s qualified name instead of its local name. An element carrying both href and xlink:href used to collapse them into a single href key.
Breaking: Fix iter() skipping the remaining children when a node is removed during iteration
Breaking: Fix an HTML fragment whose first node is text dropping the rest of the fragment. LexborHTMLParser('a<span>s</span>', is_fragment=True).text() returned 'a' instead of 'as'.
Fix text() silently returning truncated text instead of raising when a fragment fails to be collected.
Fix scripts_contain and script_srcs_contain sometimes returning wrong results due to HTML mutations.
Fix a single undecodable byte in an untrusted document making html, inner_html, html_pretty, attributes, attrs, id, tag, text_lexbor and text_content raise UnicodeDecodeError. Bytes Lexbor passes through verbatim are now substituted with U+FFFD, the way text() already did.
Fix text_lexbor sometimes holding temporary memory longer than needed
Improve memory consumption, potential stack overflow and slow extraction in merge_text_nodes(). It is now up to 10 times faster.
Improve speed of __eq__
Improve speed of attrs.items() and attrs.values().
Fix the inner_html setter attaching element children to non-element nodes
Fix head and body dangling after setting inner_html on the <html> element
Fix the inner_html setter freeing the replaced children
Fix root going stale on an HTML fragment.
Fix a segfault when walking the tree from the root of an HTML fragment that had been detached from the fragment, for example by unwrap().
Avoid segfaults when hitting OOM
Improve performance of the text method. Text fragments are now concatenated as raw bytes, up to 5x faster.
Fix skip_empty being ignored by text(deep=True)
Fix text() raising UnicodeDecodeError on undecodable bytes when deep=False. It now substitutes U+FFFD, like the deep path always did
Fix handling of the id method on text nodes
Fix attrs[key] = value raising AttributeError instead of TypeError when value is not a string
Prevent segfaults when instantiating LexborNode or LexborAttributes directly.
Fix unwrap() corrupting the tree in some cases.
Fix head and body going stale once <head>/<body> is removed from the document.
Fix attrs reading freed memory when it outlives the node it was obtained from.
Fix memory leak in attrs[key] = None; it leaked the value buffer header on every call.
Fix possible memory leak in clone()
Add encoding=True to LexborHTMLParser, which detects the encoding of bytes input and transcodes it to UTF-8 before parsing.

Version 0.4.13¶

Support lexbor-only builds with --disable-modest
Importing selectolax no longer automatically imports selectolax.parser and selectolax.lexbor
Update lexbor

Version 0.4.12¶

Add options parameter to LexborHTMLParser to pass Lexbor document parsing options (for example, LexborDocumentOptions.WO_EVENTS to disable mutation events). Fixes #221.
Fix LexborHTMLParser.clone() losing nodes for empty HTML fragments
Allow empty elements in text_lexbor. Fixes #220.
Update lexbor

Version 0.4.11¶

Add NULL checks for CSS selectors module to prevent crashes
Fix html5test pretty print serialization mode checks
Update lexbor

Version 0.4.10¶

Do not destroy nodes when stripping tags
Allow building selectolax using older lexbor versions (#218)
Update lexbor. Fixes crashes when parsing HTML (#217).

Version 0.4.9¶

Add an ability to specify tags and namespace for fragmented parser
Add a new serialization mode when pretty printing: html5test
Allow empty HTML fragments
Fix attrs access for non-element nodes
Improve buffer cleanups
Fix duplicate text when doing text(deep=True) on a text node
Improve text concatenation performance
Improve attribute handling

Version 0.4.8¶

Add Add html_pretty, inner_html_pretty methods
Enable free-threading
Improve merge_text_nodes
Update lexbor

Version 0.4.7¶

Fix .text() and iter() for HTML fragments when there are multiple nodes at the root level. Resolves #209.
Update lexbor. Resolves #212.
Breaking changes: Empty tags are now serialized to <div value=""> instead of <div value> (Commit 4530fed).
Improve unwrap_tags and merge_text_nodes.

Version 0.4.6¶

Fix HTML parsing in fragment parser for LexborHTMLParser
Fix memory leak in fragment parser
Improve skip_empty parameter for text methods
Add comment_content method
Minor performance optimizations
Add create_tag method to LexborHTMLParser
Fix advanced selector (.select()) when attributes are empty.

Version 0.4.5¶

Broken release. Not published to PyPi.

Version 0.4.4¶

Add is_fragment parameter to LexborHTMLParser @pygarap
Add the ability to skip empty text nodes for lexbor backend to .text, .iter, .traverse @pygarap
Add new properties to lexbor backend: is_element_node, is_text_node, is_comment_node, is_document_node. @pygarap
Update lexbor library

Version 0.4.3¶

Update lexbor library
Fix missing description on PyPi.

Version 0.4.2¶

Broken release. Not published to PyPi.

Version 0.4.1¶

Fix parsing of CSS selectors that contain Unicode characters.

Version 0.4.0¶

Fix incorrect default value in docstrings for strict argument
Fix incorrect exception handling for any_css_matches
Fix docstring for css_first method
Fix memory leak in merge_text_nodes for lexbor backend
Update lexbor backend
Add .inner_html property. Allows to get and set inner HTML of a node.
Update various docstrings.
Optimize performance forcss_first in lexbor backend
Fix segfaults when accessing attributes. Resolves #135.
Add new .clone method to lexbor backend. Resolve #117.
Improve unicode handling for malformed text. Resolves #138.
Fix segfaults when doing double .decompose. Resolves #179.
Fix sefgaults when doing double .unwrap. Resolves #169.
Fix typo for tag names. Clarify available tag names.

Version 0.3.34¶

Released

Lexbor backend now supports :lexbor-contains("abc" i) CSS pseudo-class to match text nodes.

Version 0.3.33¶

Released

Add merge_text_nodes to lexbor backend. Fixes #170. @amirshukayev
Performance improvements in Cython code. @Vizonex

Version 0.3.32¶

Released

Update lexbor. New version of lexbor fixes bugs with CSS selectors.

Version 0.3.31¶

Released

Improve type hints, add docstrings to type hints
Prevent decomposing of the root node
Unpin Cython version and make it Optional
Allow empty attribute values. Fixes #165.

Version 0.3.30¶

Released

Update lexbor
Expose SelectolaxError exception in lexbor.pyi

Version 0.3.29¶

Released

Feat: Add unwrap empty tags functionality. Fixes #159.

Version 0.3.28¶

Released

Fix: Update lexbor and improve HTML serialization speed. Fixes #153.
Fix: typo in type annotations. Fixes #147.
Fix: Fix incorrect type annotations for LexborHTMLParser.__init__. Fixes #144.

Version 0.3.27¶

Released

Fix: Header detected as head

Version 0.3.26¶

Released

Improve type hints

Version 0.3.25¶

Released

Feat: Add parse_fragment() and create_tag()
Add missing typing for Node.insert_child()
Add Node.parser to access the HTMLParser to which the node belongs

Version 0.3.24¶

Released

Add Node.insert_child method to lexbor and modest backends

Version 0.3.23¶

Released

Add Python 3.13 wheels
Update lexbor

Version 0.3.21¶

Released

Breaking change: lexbor backend now includes the root node when querying CSS selectors. Same as Modest backend.
Fix css_matches and any_css_matches methods for Modest backend on some compilers

Version 0.3.20¶

Released

Fixup for 0.3.19 release
Fix tag order for lexbor backend

Version 0.3.19¶

Released

Increase maximum HTML size to 2.4GB

Version 0.3.18¶

Released

Fix memory leak when using CSS selectors, lexbor backend

Version 0.3.17¶

Released

Update lexbor
Add Python 3.12 wheels

Version 0.3.16¶

Released

Make HTML nodes hashable
Pin Cython version

Version 0.3.15¶

Released

Improve typing. Thanks to @nesb1

Version 0.3.14¶

Released

Fix memory leak for lexbor backend

Version 0.3.13¶

Released

Update lexbor

Version 0.3.12¶

Released

Update lexbor
Add Python 3.11 wheels

Version 0.3.11¶

Released

Fix out-of-bounds bug for merge_text_nodes method.

Version 0.3.10¶

Released

This release does not contain any changes. Due to a typo in the version number (#70), we need to make a new release.

Version 0.3.9¶

Released

Remove trailing separator when using text(deep=True, separator='x').
Add a new merge_text_nodes method for Modest backend.

Version 0.3.8¶

Released

Fix incorrect text handling when using text(deep=True) on a text node.

Version 0.3.7¶

Released

Fix return type of HTMLParser.tags

Version 0.3.6¶

Released

Improve text handling
Add binary builds for Python 3.10 and ARM on MacOS and Linux

Version 0.3.5¶

Released

Add type annotations

Version 0.3.4¶

Released

Fix HTMLParser.html

Version 0.3.3¶

Released

Use document for the HTMLParser.html, LexborHTMLParser.html root properties

Version 0.3.2¶

Released

Fix selector method for lexbor
Improve text extraction for lexbor

Version 0.3.1¶

Released

Fix setup.py for Windows

Version 0.3.0¶

Released

Added lexbor backend
Fix cloning for Modest backend

Version 0.2.14¶

Released

Added advanced Selector (the select method)
Improved speed of strip_tags
Added clone method for the HtmlParser object
Exposed detect_encoding, decode_errors, use_meta_tags, raw_html attributes for HtmlParser
Added sget method to the attrs property

Version 0.2.13¶

Released

Don’t throw exception when encoding text as UTF-8 bytes fails (#40).
Fix Node.attrs.items() causes (#39).

Version 0.2.12¶

Released

Build wheels Apple Silicon

Version 0.2.11¶

Released

Fix strip argument is ignored for the root node (#35).
Fix CSS parser hangs on a bad CSS selector (#36).

Version 0.2.10¶

Released

Fix root node property (#32). The root property now points to the html tag.

Version 0.2.9¶

Released

Fix README for PyPI

Version 0.2.8¶

Released

Add wheels for Python 3.9

Version 0.2.7¶

Released

Add raw_value attribute for Node objects (#22)
Improve node modification operations

Version 0.2.6¶

Released

Fix dependency on the source Node when inserting to or modifying destination Node

Version 0.2.5¶

Released

Allow to pass Node instances to replace_with, insert_before and insert_after methods
Added insert_before and insert_after methods

Version 0.2.4¶

Released

Set maximum input size to 80MB
Update modest

Version 0.2.3¶

Released

Rebuild PyPi wheels to support Python 3.8 and manylinux2010

Version 0.2.2¶

Released

Fix node comparison

Version 0.2.1¶

Released

Add optional include_text parameter for the iter and traverse methods

Version 0.2.0¶

Released

Fix iter() does not yield text nodes
Switch from TravisCI to Github Actions
Build and ship wheels for Windows, MacOS and Linux using Azure Pipelines
Add unwrap and unwrap_tags method (#7)
Add replace_with method (#13)
Add attrs property
Add traverse method