blingfire
cargo install blingfire

blingfire

Rust wrapper for the BlingFire tokenization library

by reinfer

1.0.0 (see all)License:MIT
cargo install blingfire
Readme

Build Status Documentation Crate

BlingFire in Rust

blingfire is a thin Rust wrapper for the BlingFire tokenization library.

Add the library to Cargo.toml to get started

cargo add blingfire

The library exposes two functions text_to_words and text_to_sentences

use blingfire;

fn main() {
    let mut parsed = String::new();

    blingfire::text_to_words("Cat,sat on   the mat.", &mut parsed).unwrap();
    assert_eq!(parsed.as_str(), "Cat , sat on the mat .");

    blingfire::text_to_sentences("Cat sat. Dog barked.", &mut parsed).unwrap();
    assert_eq!(parsed.as_str(), "Cat sat.\nDog barked.");
}

The code is licensed under the MIT License.

GitHub Stars

12

LAST COMMIT

2yrs ago

MAINTAINERS

1

CONTRIBUTORS

2

OPEN ISSUES

0

OPEN PRs

0
VersionTagPublished
1.0.0
2yrs ago
0.3.0
3yrs ago
0.2.1
3yrs ago
0.2.0
3yrs ago
No alternatives found
No tutorials found
Add a tutorial