İşleme Grafiği

22 İşleme (f6d169567c922683be52f48164a2eabd7291107b)

Yazar SHA1 Mesaj Tarih
Ana Gelez f6d169567c
Introduce Searcher::new
This function does what was previously done in main:
create a search index, and try to recover for errors
if possible.

This commit also fixes plume-cli to use the new Searcher
API (that depends on a DbPool, not on a single Connection).
4 yıl önce
Ana Gelez eaad38ad2c
Avoid unwrapping, return an Error instead 4 yıl önce
Mina Galić 962fbcc503
plume-model: refactor Searcher to have its own DbPool
this way, we don't need to pass along a conn into the function.
This should make splitting PlumeRocket up into its components a little
easier. This PR is another attempt at fixing #799.
4 yıl önce
Mina Galić f945e18bf1
Revert "Add an SearcherActor, wrapping Searcher & DbPool"
This reverts commit 0757f84397fc4d4ecba8a904e9d25f1b243b4292.
We'll attempt to use a strategy in #807
4 yıl önce
Mina Galić 5e17636b9d
Add an SearcherActor, wrapping Searcher & DbPool
and implement AddDocument message!
This code compiles, but hasn't been put to use yet.
4 yıl önce
Kitaiti Makoto 484659fde2 Run cargo fmt 4 yıl önce
Kitaiti Makoto c5d03d300b Cause IndexInvalidDataError when search index is invalid 4 yıl önce
KITAITI Makoto 92a386277b
Switchable tokenizer (#776)
* [REFACTORING]Rename whitespace_tokenizer to tag_tokenizer for
registration

Name representing its purpose is preferred.

* Add lindera-tantivy to plume-model's dependencies

* Install lindera-tantivy

* Add SearchTokenizerConfig struct

* Add search tokenizers to config option

* Use CONFIG for tokenizers

* Use enum to hold tokenizer config instead of initializing on config phase

* Use guard instead of duplicate default values

* Use as_deref() instead of guard

* Move SearchTokenizer from plume-models to plume-models::search::tokenizer

* Rename SearchTokenizer to TokenizerKind

* Define SearchTokenierConfig::determine_tokenizer()

* Use determine_tokenizer in SearchTokenizerConfig::init()

* Pass tokenizer config to Searcher methods

* Add LowerCase filter to Lindera tokenizer

* Add test for Lindera tokenizer

* Define SEARCH_LANG env to specify tokenizers set

* Run cargo fmt

* Make Lindera tokenizer optional

* Fix typos
4 yıl önce
KITAITI Makoto ef70cb93e6
Upgrade Tantivy to v0.12.0 (#771)
* Upgrade Tantivy to 0.12.0

* Follow Tantivy Tokenizer's new type definition

* Wrap tokenizers with TextAnalyzer to use filter methods

* Replace async IndexWriter::garbage_collect_files with sync functions

* Update Cargo.toml
4 yıl önce
Ana Gelez 5f8d6b8e0e Rust 2018! (#726) 4 yıl önce
fdb-hiroshima 3663bffe5c Update compiler (#725)
* update compiler to recent nightly

* cargo fmt

* fix clippy

but ructe code is still complaining

* update circleci and rocket_csrf

* fix last clippy problems

* cargo fmt

* fix build issue

was caused be improper handling of recursive template (comment) by old ructe, which is fixed on newer version
4 yıl önce
fdb-hiroshima bf7603d439
don't index drafts (#656) 5 yıl önce
fdb-hiroshima 49bb8cb0bc
import migrations and don't require diesel_cli for admins (#555)
* import migrations via macro

* panic on database not to the latest migration

* add subcommand to plm

* create migration that run tantivy index creation

* remove diesel_cli from places it was

* use our migration system for tests

* create table __diesel_schema_migrations if needed
5 yıl önce
fdb-hiroshima 26fc2cde5d
use circleci instead of travis (#511)
Upgrade rust-toolchain
Upgrade Tantivy
Warning: tantivy now use a different file format, search index may need to be deleted and recreated to work as intended
try to run compile twice in a row because sometime oomk is evil
5 yıl önce
Atul Bhosale b945d1f602 Run 'cargo fmt' to format code (#489) 5 yıl önce
Igor Galić 732f514da7 Refactor with the help of Clippy (#462)
We add clippy as our build — also rectifying the missing `plume-cli` build!

In the next step we follow clippy's advise and fix some of the "simple" mistakes in our code, such as style or map usage.

Finally, we refactor some hard bits that need extraction of new types, or refactoring of function call-types, especially those that thread thru macros, and, of course functions with ~15 parameters should probably be rethought.
5 yıl önce
Baptiste Gelez fe6e69d7c4
Add a fqn field to blogs and users (#457)
Fixes #319
5 yıl önce
fdb-hiroshima 944a9d08a7
Update dependencies (#428)
* Bump serde from 1.0.82 to 1.0.84
* Bump ructe from 0.5.4 to 0.5.6
* Bump colored from 1.6.1 to 1.7.0
* Bump itertools from 0.7.11 to 0.8.0
* Bump ammonia from 1.2.0 to 2.0.0
* Bump hyper from 0.12.19 to 0.12.20
* Bump serde_derive from 1.0.80 to 1.0.84
* Bump stdweb from 0.4.12 to 0.4.13
* Bump rsass from 0.9.4 to 0.9.6
* Bump failure from 0.1.3 to 0.1.5
* Bump reqwest from 0.9.5 to 0.9.8
* Bump serde_derive from 0.7.1 to 0.8.0
5 yıl önce
Baptiste Gelez 80a4dae8bd
Avoid panics (#392)
- Use `Result` as much as possible
- Display errors instead of panicking

TODO (maybe in another PR? this one is already quite big):
- Find a way to merge Ructe/ErrorPage types, so that we can have routes returning `Result<X, ErrorPage>` instead of panicking when we have an `Error`
- Display more details about the error, to make it easier to debug

(sorry, this isn't going to be fun to review, the diff is huge, but it is always the same changes)
5 yıl önce
Trinity Pointard d6e220cc8d Actually index post_id in tantivy
Otherwise you can't remove/edit a document
5 yıl önce
Trinity Pointard 39deede935 Verify signature date
Fix #286
Remove indexed from post_id search field as it was added by mistake
5 yıl önce
fdb-hiroshima 449641d158
Add a search engine into Plume (#324)
* Add search engine to the model

Add a Tantivy based search engine to the model
Implement most required functions for it

* Implement indexing and plm subcommands

Implement indexation on insert, update and delete
Modify func args to get the indexer where required
Add subcommand to initialize, refill and unlock search db

* Move to a new threadpool engine allowing scheduling

* Autocommit search index every half an hour

* Implement front part of search

Add default fields for search
Add new routes and templates for search and result
Implement FromFormValue for Page to reuse it on search result pagination
Add optional query parameters to paginate template's macro
Update to newer rocket_csrf, don't get csrf token on GET forms

* Handle process termination to release lock

Handle process termination
Add tests to search

* Add proper support for advanced search

Add an advanced search form to /search, in template and route
Modify Tantivy schema, add new tokenizer for some properties
Create new String query parser
Create Tantivy query AST from our own

* Split search.rs, add comment and tests

Split search.rs into multiple submodules
Add comments and tests for Query
Make user@domain be treated as one could assume
5 yıl önce