Fork of pulldown-cmark with support for bidi
Ви не можете вибрати більше 25 тем Теми мають розпочинатися з літери або цифри, можуть містити дефіси (-) і не повинні перевищувати 35 символів.
 
 
 
Trinity Pointard 58514a67a5 improvements on bidi 3 роки тому
benches Optimize smart punct by creating a dedicated ItemBody 4 роки тому
examples Clean up new callback design a bit 4 роки тому
fuzzer Release 0.7.0 (#427) 4 роки тому
specs Support inline attributes split over multiple lines inside blockquotes 4 роки тому
src improvements on bidi 3 роки тому
tests improvements on bidi 3 роки тому
third_party Finish initial implementation of smart punctuation 4 роки тому
tools Tweak code generating code to produce rustmft compliant code when possible 5 роки тому
.gitignore Remove Cargo.lock from .gitignore 5 роки тому
CONTRIBUTING.md CONTRIBUTING: add "Getting familiar with the project" 5 роки тому
Cargo.lock Bump version to 0.8.0; update dependencies 4 роки тому
Cargo.toml Bump version to 0.8.0; update dependencies 4 роки тому
LICENSE Change license to MIT 9 роки тому
README.md Bump version to 0.8.0; update dependencies 4 роки тому
azure-pipelines.yml Bump MSRV to 1.36 4 роки тому
build.rs Finish initial implementation of smart punctuation 4 роки тому

README.md

pulldown-cmark

Build Status Docs Crates.io

Documentation

This library is a pull parser for CommonMark, written in Rust. It comes with a simple command-line tool, useful for rendering to HTML, and is also designed to be easy to use from as a library.

It is designed to be:

  • Fast; a bare minimum of allocation and copying
  • Safe; written in pure Rust with no unsafe blocks
  • Versatile; in particular source-maps are supported
  • Correct; the goal is 100% compliance with the CommonMark spec

Further, it optionally supports parsing footnotes, Github flavored tables, Github flavored task lists and strikethrough.

Rustc 1.36 or newer is required to build the crate.

Why a pull parser?

There are many parsers for Markdown and its variants, but to my knowledge none use pull parsing. Pull parsing has become popular for XML, especially for memory-conscious applications, because it uses dramatically less memory than constructing a document tree, but is much easier to use than push parsers. Push parsers are notoriously difficult to use, and also often error-prone because of the need for user to delicately juggle state in a series of callbacks.

In a clean design, the parsing and rendering stages are neatly separated, but this is often sacrificed in the name of performance and expedience. Many Markdown implementations mix parsing and rendering together, and even designs that try to separate them (such as the popular hoedown), make the assumption that the rendering process can be fully represented as a serialized string.

Pull parsing is in some sense the most versatile architecture. It's possible to drive a push interface, also with minimal memory, and quite straightforward to construct an AST. Another advantage is that source-map information (the mapping between parsed blocks and offsets within the source text) is readily available; you can call into_offset_iter() to create an iterator that yields (Event, Range) pairs, where the second element is the event's corresponding range in the source document.

While manipulating ASTs is the most flexible way to transform documents, operating on iterators is surprisingly easy, and quite efficient. Here, for example, is the code to transform soft line breaks into hard breaks:

let parser = parser.map(|event| match event {
	Event::SoftBreak => Event::HardBreak,
	_ => event
});

Or expanding an abbreviation in text:

let parser = parser.map(|event| match event {
	Event::Text(text) => Event::Text(text.replace("abbr", "abbreviation").into()),
	_ => event
});

Another simple example is code to determine the max nesting level:

let mut max_nesting = 0;
let mut level = 0;
for event in parser {
	match event {
		Event::Start(_) => {
			level += 1;
			max_nesting = std::cmp::max(max_nesting, level);
		}
		Event::End(_) => level -= 1,
		_ => ()
	}
}

There are some basic but fully functional examples of the usage of the crate in the examples directory of this repository.

Using Rust idiomatically

A lot of the internal scanning code is written at a pretty low level (it pretty much scans byte patterns for the bits of syntax), but the external interface is designed to be idiomatic Rust.

Pull parsers are at heart an iterator of events (start and end tags, text, and other bits and pieces). The parser data structure implements the Rust Iterator trait directly, and Event is an enum. Thus, you can use the full power and expressivity of Rust's iterator infrastructure, including for loops and map (as in the examples above), collecting the events into a vector (for recording, playback, and manipulation), and more.

Further, the Text event (representing text) is a small copy-on-write string. The vast majority of text fragments are just slices of the source document. For these, copy-on-write gives a convenient representation that requires no allocation or copying, but allocated strings are available when they're needed. Thus, when rendering text to HTML, most text is copied just once, from the source document to the HTML buffer.

When using the pulldown-cmark's own HTML renderer, make sure to write to a buffered target like a Vec<u8> or String. Since it performs many (very) small writes, writing directly to stdout, files, or sockets is detrimental to performance. Such writers can be wrapped in a BufWriter.

Build options

By default, the binary is built as well. If you don't want/need it, then build like this:

> cargo build --no-default-features

Or put in your Cargo.toml file:

pulldown-cmark = { version = "0.8", default-features = false }

SIMD accelerated scanners are available for the x64 platform from version 0.5 onwards. To enable them, build with simd feature:

> cargo build --release --features simd

Or add the feature to your project's Cargo.toml:

pulldown-cmark = { version = "0.8", default-features = false, features = ["simd"] }

Authors

The main author is Raph Levien. The implementation of the new design (v0.3+) was completed by Marcus Klaas de Vries.

Contributions

We gladly accept contributions via GitHub pull requests. Please see CONTRIBUTING.md for more details.