~alip/jja

v0.9.1 9 months ago .tar.gz browse log

Release v0.9.1

- upgrade `nix` crate from `0.27` to `0.28`.
- upgrade `tempfile` crate from `3.8` to `3.10`.
- upgrade `rust-embed` crate from `8.1` to `8.4`.
- upgrade `rusqlite` crate from `0.30` to `0.31`.
- upgrade `rayon` crate from `1.8` to `1.10`.
- upgrade `hostname` crate from `0.3` to `0.4`.
- Add support for restoring BrainLearn experience files from Lichess
  evaluations export JSON.
- `TotalPlyCount` PGN tag is now supported in make filters.
- fix `--win,draw,loss-factor` calculation for make
- Add 50-moves detection and 3-position repetition detection to the play
  subcommand.
- upgrade `once_cell` crate from `1.18` to `1.19`.
- upgrade `rayon` crate from `1.7` to `1.8`.
- upgrade `zstd` crate from `0.12` to `0.13`.
- upgrade `rusqlite` crate from `0.29` to `0.30`.
- upgrade `dialoguer` crate from `0.10` to `0.11`.
- upgrade `i18n-embed` crate from `0.13` to `0.14`.
- upgrade `rust-embed` crate from `6.8` to `8.0`.
- upgrade `built` crate from `0.6` to `0.7`.
- fix quote matching at word boundaries when no wildcard characters are given.
- merge learned `-O<u16>, --outlier-threshold=<u16>` to specify an outlier threshold
  where moves with weights higher than the given threshold are filtered out from the
  merged output book. This stands from the fact that if the weight differences of
  the same move in two different books have a high difference, calculating any
  average may not yield meaningful results. The default is 0, which disables this
  feature.
- merge learned `--strategy dynmid` which stands for
  `jja::merge::MergeStrategy::DynamicMidpoint`.
- merge learned `--strategy wdist` which stands for
  `jja::merge::MergeStrategy::WeightedDistance`.
- upgrade `nix` crate from `0.26` to `0.27` and add the `signal` feature.
- merge learned `--strategy wmedian` which stands for
  `jja::merge::MergeStrategy::WeightedMedian`.
- merge learned `--strategy entropy` which stands for
  `jja::merge::MergeStrategy::Entropy`.
- merge learned `--strategy quadratic` which stands for
`jja::merge::MergeStrategy::QuadraticMean`. This strategy merges using the Quadratic
Mean (Root Mean Square) approach. The Quadratic Mean (also known as the Root Mean
Square) is a statistical measure of the magnitude of a set of numbers. It offers a
more balanced view, especially when dealing with numbers of varying magnitudes.
- merge learned `--strategy lavg` which stands for
`jja::merge::MergeStrategy::LogarithmicAverage`. This strategy merges using a
logarithmic averaging approach. Given that logarithmic functions compress large
values and expand small values, we can use them to get a merge strategy that's
sensitive to differences in smaller weights while being more resistant to
disparities in larger weights.

v0.9.0 1 year, 5 months ago .tar.gz browse log

Release v0.9.0

- merge cutoff specified using `-c, --cutoff` is now applied after the merge
  strategy assigns new weights. This way, cutoff may also be used to filter out
  unwanted entries after the merge.
- the default value of `edit --color-weight-blue` has been changed from `32768` to
  `65280` for consistency with `--nag-weight-good`.
- merge learned new merge strategy `jja::merge::MergeStrategy::Sort` to calculate
  weight based on the relative position of the move entry in sorted move entries.
- merge learned new merge strategy `jja::merge::MergeStrategy::GeometricScaling` to
  calculate the weight using geometric scaling: The geometric scale focuses on
  multiplying numbers together rather than adding, which can help in equalizing
  disparities.
- merge learned new merge strategy `jja::merge::MergeStrategy::HarmonicMean` to
  calculate the harmonic mean weight of the identical move entries in both books.
  This approach tends to favor more balanced weights and is less influenced by
  extreme values.
- merge learned new merge strategy `jja::merge::MergeStrategy::Sigmoid` to remap the
  weights in a non-linear fashion using the sigmoid function. The idea here is to
  diminish the influence of extreme values, which might be causing the
  dissatisfaction in previous strategies.
- edit now properly prioritizes CTG move entries with higher combined color, NAG,
  performance score such that the ordering is more close to the original.
- edit learned new commandline flags `-C, --no-colors`, and `-N, --no-nags` to avoid
  using CTG move colours and NAGs to assign PolyGlot move weights or Arena move
  priorities.
- merge and make commands now outputs `info` on the output book after successful run.
- merge strategy defaults to the new `pavg` strategy rather than `sum` now.
- merge learned `--strategy pavg` which stands for
  `jja::merge::MergeStrategy::PercentageAverage`. This strategy merges by computing
  the weighted average of percentage weights, taking into account the total number
  of entries in each book. This approach gives higher importance to moves from
  larger books versus smaller ones, ensuring that the resultant weights reflect the
  relative contributions of each book based on its size.
- merge learned `--rescale` to rescale weights of merged book entries globally to
  fit into 16 bits. This is similar to `edit --rescale` but works on the final
  merged book rather than the input books.
- make now uses `estimate-num-keys` property of the temporary RocksDB database to
  determine the _approximate_ unique position count rather than bulk scanning the
  database twice which improves performance.
- make now uses read ahead during the bulk scan of the temporary RocksDB database.
  This increases performance significantly especially on spinning disks.
  The read ahead size may be configured using the `--read-ahead=<SIZE>` commandline
  option. This defaults to `4 MB` and may be disabled by passing `0` as size.
- make now uses asynchronous I/O during the bulk scan of the temporary RocksDB
  database. This increases performance significantly especially with the `io_uring`
  feature enabled. This may be disabled using the new `--sync` commandline argument
  or using the environment variable `JJA_SYNC`.
- drop `smallvec` crate usage, and remove dependency.
- replace `humansize` with `bytefmt` crate.
- `jja::abk::traverse_tree` function now returns a `BTreeMap<u64,
  Vec<CompactBookEntry>` rather than a `BTreeMap<u64, Vec<BookEntry>` which is a
  **breaking change**. Moreover this function now sorts the value vectors using
  binary search which makes `abk->bin` both more memory efficient and faster.
- `jja::ctg::ByteBoard::from_position` has been replaced with the function
  `from_board` which is a **breaking change**.
- `jja::ctgbook::CtgBook::process_move` function now utilizes binary search for move
  table lookup improving efficiency. `jja::ctg::MOVETABLE`'s `encoding` element has
  changed its type from `char` to `u8` which is a **breaking change**.
- edit converts CTG books to BIN books with 20% less memory usage using the new compact
  `jja::polyglot::CompactBookEntry` data structure.
- `jja::abk::PackedSBookMoveEntry` has been renamed to `CompactSBookMoveEntry` which
  is a **breaking change**.
- drop the unused functions `jja::ctgbook::CtgBook::extract_all{,2}` and the unused
  type `jja::ctgbook::CtgTree` both of which are **breaking changes**.
- rename `jja::ctgbook::CtgBook::extract_map{,2}` to `extract_abk{,2}` for
  consistency which is a **breaking change**.
- use `extract_bin` in `CTG->BIN` to improve efficiency
- implement new functions `jja::ctgbook::CtgBook::extract_bin{,2}`
- use `human-panic` crate for user friendly panic messages.

v0.8.1 1 year, 5 months ago .tar.gz browse log

Release v0.8.1

- upgrade `tempfile` crate from `3.7` to `3.8`.
- upgrade `clap` crate from `4.3` to `4.4`.
- info learned to print the sha256 checksum of the chess file.
- fix a memory leak in builds with the `i18n` feature disabled.
- edit now converts CTG books to ABK books around 5% times faster using 30% less
  memory thanks to the new public functions `jja::ctgbook::CtgBook::extract_map{,2}`
  which directly returns a `BTreeMap<u64, Vec<SBookMoveEntry>>` rather than processing a
  `CtgTree` into a `SBookMoveEntryHashMap` as an additional step.
- info now prints information about the size of a single entry.
- info now prints information about ABK number of entries.
- New public function `jja::abkbook::AbkBook::total_entries()` to calculate the
  number of entries in an Arena opening book file.
- Various memory efficiency improvements for `CTG->ABK` conversion.
- edit now reads from standard input rather than spawning the default editor if the
  standard input does not refer to a terminal, ie is not a TTY. This is useful for
  programmatic editing of opening books. find output may be piped to edit which
  makes it practical to write entries from an opening book to another opening book
  of the same type. Finally this also gives users who are unable to use the editor
  functionality a chance to edit their opening books by submitting a custom CSV
  file via standard input.
- replace the final `lazy_static!` usages with `once_cell` and drop the dependency
  on `lazy_static` crate. Our usage of `once_cell` is currently in Rust nightly and
  is hopefully soon going to land in stable Rust.

v0.8.0 1 year, 6 months ago .tar.gz browse log

Release v0.8.0

- new `perft` subcommand to count legal move paths of a given length
- upgrade `pgn-reader` crate from `0.24` to `0.25`
- upgrade `shakmaty` crate from `0.25` to `0.26`
- upgrade `tempfile` create from `3.6` to `3.7`
- **important fix** in PGN generation to avoid skip some lines due to the incorrect
  usage of the repetition tracker. This effects PGN generation from all supported
  opening book formats (`abk`, `bin`, `ctg`, `exp`, `obk`) so users are highly
  recommended to re-export their previously exported PGNs using `jja edit
  book.{abk,bin,ctg,exp,obk} -o book.pgn`.
- bump MSRV (minimal supported Rust version) from `1.64` to `1.70`.
- quote now matches at word boundaries unless the given quote pattern includes the
  wildcard characters `.`, `?`, or `*`.
- jja now ignores invalid castling rights and en passant squares in PGN games' FEN
  header rather than skipping the game.
- edit learned to dump all positions in a PGN file to an output EPD file. Compressed
  PGN files are supported, hence the usage is
  `jja edit source.pgn{,.bzip2,gz,lz4,xz,zst} -o output.epd`.
- dump learned `-e`, `--elements` to specify a list of elements to dump which
  defaults to `id, position`. This option works only for PGN dumps.
- new subcommand `probe` which can be used to probe Syzygy tablebases up to 7
  pieces. This is almost functionally identical to the awesome Fathom tool, but also
  offers some alternative modes such as `--test --fast` to skip walking the DTZ line
  at the cost of misevaluating `MaybeWin` and `MaybeLoss` positions.
- edit learned two new command line options `--max-ply=<PLY>`, and
  `--look-ahead=<PLY>`. The former limits PGN generation to a certain number of
  plies, and defaults to `1024`. The latter is used to specify the number of plies
  to look ahead on PolyGlot book lookup misses during PGN generation which is useful
  to generate PGNs of book generated by `--only-black|white`. Currently
  `--look-ahead` only supports `0`, and `1` as argument, of which `0` being the
  default value, other values will generate an `unimplemented` panic which directs
  the user to report a bug. This change comes along with a change in the public
  function signature of `jja::polyglotbook::PolyGlotBook::write_pgn` which is a
  **breaking change**.
- The function `jja::polyglot::from_move` now takes argument a *reference* to a
  `shakmarty::Move` rather than a `shakmaty::Move` which is a **breaking change**.
- The default values of some weight conversion arguments to `jja edit` have been
  changed to be more intuitive, in that default value of `--color-weight-green` has
  been changed from `10000` to `65520`, `--color-weight-blue` from `1000` to
  `32768`, `--nag-weight-good` from `9000` to `65280`, `--nag-weight-hard` from
  `10000` to `65520`, `--nag-weight-interesting` from `7500` to `61440`,
  `--nag-weight--forced` from `10000` to `65520`, and `--nag-weight-only` from
  `10000` to `65520`. The new defaults are empirical, and advanced users are
  recommended to use `--color-weight-*`, and `--nag-weight-*` arguments with `jja
  edit` for fine-tuning during `CTG->BIN` or `CTG->ABK` conversions.
- **Important fix** in `jja::ctgbook::CtgBook::read_page` function to prevent panics
  due to out-of-bounds access. This fixes search & conversion with some huge CTG
  books.
- The function `jja::abkbook::AbkBook::traverse_book_and_merge` function now returns
  nothing rather than `Result<(), Box<dyn std::error::Error>>` which is a **breaking
  change**.
- The function `jja::chess::lines_from_tree` now returns a `Table` rather than a
  `Result<Table, Box<dyn std::error::Error>>` which is a **breaking change**.
- Use `anyhow` crate for error handling. This is mainly used in the main code, and
  not the library code but there are points where the library code is changed, and
  there are **breaking changes**: `jja::pgnbook::create_opening_book` now returns an
  `anyhow::Result<GameBase>`, rather than
  `Result<GameBase, Box<dyn std::error::Error>>`. Similarly the function
  `jja::pgn::pgn_dump` now returns `anyhow::Result<()>` rather than
  `Result<(), Box<dyn std::error::Error>>`.
- The functions `jja::system::get_progress_{bar,spinner}` now return a `ProgressBar`
  rather than a `Result<ProgressBar, Box<dyn std::error::Error>>`. These functions
  now panic when there is an error in the template, which shouldn't happen normally.
  This change is a **breaking change**.
- The function `jja::system::edit_tempfile` now returns `EditTempfileError` on error
  rather than `Box<dyn std::error::Error>`. Moreover the `EditTempfileError` enum
  has a new member `EditTempfileError::InputOutputError(std::io::Error)`. These
  changes are in the public API and hence are **breaking changes**.
- restore learned experimental EPD output support. This may be used in a pipeline
  with dump, e.g: `jja dump -fcsv file.pgn | jja restore file.epd` to create an
  Extended Position Description file of all the positions in the given Portable Game
  Notation file. The EPD entries include the Zobrist hash of the positions in the `id`
  field.
- fix deserializing of Chess960 castling rights in `jja::chess::deserialize_chess`.
- `jja::chess::deserialize_chess` function now panics on invalid piece indexes
  rather than silently continuing.
- `hash` subcommand has been renamed to `digest`. The former will work as an alias
  until the next major version bump.

v0.7.1 1 year, 7 months ago .tar.gz browse log

Release v0.7.1

- fix build with `i18n` feature disabled.
- enable the lint `#![deny(clippy::cast_lossless)]` for jja library and fix
  offending code.

v0.7.0 1 year, 7 months ago .tar.gz browse log

Release v0.7.0

- enable the lint `#![deny(clippy::cast_precision_loss)]` for jja library and fix
  offending code.
- edit learned to use buffered I/O, rather than direct I/O during PGN conversion to
  improve efficiency, especially with huge opening books.
- edit learned to display a progress bar during PGN conversion.
- CTG interface has seen many **breaking change**s to make the interface leaner,
  with regards to the recent mmap changes. As a result, the function
  `jja::ctg::find_piece` returns an `Option<i32>` rather than a `Result<i32,
  std::io::Error>`, the functions `jja::ctgbook::CtgBook::extract_all{,2}` return a
  `CtgTree` rather than a `Result<CtgTree, std::io::Error>`, and the function
  `jja::ctgbook::CtgBook::lookup_moves` returns an `Option<Vec<CtgEntry>>` rather
  than a `Result<Option<Vec<CtgEntry>>, std:io::Error>`.
- edit now uses a much more efficient PGN converter implementation which does not
  load the whole tree of variations in memory. This makes it possible to convert
  huge opening book files into PGN without running out of system memory. Moreover,
  the PGN converter now respects the `-f, --fen`, and `-p, --pgn` arguments of `jja
  edit` so it is possible to convert a subtree of the opening book into PGN. See the
  respective issues [#14](https://todo.sr.ht/~alip/jja/14), and
  [#16](https://todo.sr.ht/~alip/jja/16) for more details. This change deprecates
  the function `jja::chess::pgn_from_tree`, it is recommended to use the new
  `write_pgn` function of the respective opening book file.
- `jja::polyglotbook::PolyGlotBook` implementation has seen many improvements to be
  a leaner interface. `PolyGlotBook::get{,_key}` functions no longer panic on
  invalid indexes, rather return an `Option<BookEntry>` rather than a `BookEntry`.
  This implementation avoids an ugly hack to map a dummy anonymous memory region for
  PolyGlot books with zero-size to allow creating PolyGlot books from scratch (e.g:
  `touch new-file.bin && jja edit -i new-file.bin`). As these functions are public,
  this is a **breaking change**.
- merge now supports merging BrainLearn experience files together. The option
  `--weight-cutoff` has been renamed to `--cutoff` and now supports filtering out by
  min depth in BrainLearn experience files.
- edit now supports editing BrainLearn experience files. In-place editing of such
  files is also supported.
- reduce minimum supported Rust version (MSRV) from `1.70` to `1.64` for portability.
- bring back the dependency on `is_terminal` crate rather than depending on >=rust-1.70.
- downgrade `pgn-reader` crate from `0.25` to `0.24`.
- downgrade `shakmaty` crate from `0.26` to `0.25`.
- upgrade `regex` crate from `1.8` to `1.9`.
- upgrade `smallvec` crate from `1.10` to `1.11`.
- The `progress_bar` member of `jja::obkbook::ObkBook` has been removed, in return
  the public functions `jja::obkbook::ObkBook::{tree,traverse_tree}` require an
  optional reference to a progress bar now. This avoids a needless clone of the
  progress bar and it is a **breaking change**. Moreover, the function
  `jja::obkbook::ObkBook::read_moves` has been renamed to `load` which is again a
  **breaking change**.
- find now memory maps the OBK opening book files rather than reading the whole file
  into memory at once for efficiency. This caused a change in public function
  signatures of `jja::obkbook::ObkBook::{read_moves,traverse_tree,tree}` which is a
  **breaking change**.
- dump learned `binary` format to dump PGNs in PostgreSQL binary output format. This
  brings in a dependency on crate `pgcopy`.
- find now memory maps the PolyGlot opening book files rather than maintaining a
  `BufReader<File>` handle to them. This removes the `book` public member of
  `jja::polyglotbook::PolyGlotBook`, and changes signatures of public functions
  `jja::polyglotbook::PolyGlotBook::{lookup_moves,tree}` which is a **breaking
  change**. This also changes names of the public functions
  `jja::polyglotbook::PolyGlotBook::{find_book_key,read_book_entry,read_book_key}`
  to `jja::polyglotbook::PolyGlotBook::{find,get,get_key}` respectively which is
  again a **breaking change**.  Moreover `jja::polyglotbook::PolyGlotBook`'s default
  iterator implementation has been changed to iterate over single book entries. New
  function introduced `jja::polyglotbook::PolyGlotBook::into_iter_grouped()` may be
  used to iterate over entries grouped by key.
- check for whether standard error is a TTY, rather than standard output when
  displaying progress bars. This allows commands such as dump to display progress
  bars during execution.
- optimize the stockfish hash function implementation, making it almost double as
  fast.
- hash learned `-B`, `--benchmark`, and `-I`, `--benchmark-iterations` to benchmark
  Stockfish and Zobrist hash functions. This brings in a dependency on
  `benchmarking` crate.
- Avoid translating CSV headers of ABK entries in `jja find` output. Note, this
  change is for CSV output only which is printed when the output is not a TTY or the
  option `--porcelain=csv` is given.
- The `progress_bar` member of `jja::ctgbook::CtgBook` has been removed, in return
  the public functions `jja::ctgbook::CtgBook::extract_all{,2}` require an optional
  reference to a progress bar now. This avoids a needless clone of the progress bar
  and it is a **breaking change**.
- avoid needless conversion to and from EPD in `jja::ctgbook::CtgBook` functions
  improving efficiency. The function `jja::ctgbook::CtgBook::lookup_moves` now
  accepts a `&dyn shakmaty::Position` rather than an EPD string which is a
  **breaking change**.
- drop unused public functions `jja::ctg::find_piece`, `jja::ctg::decode_fen_board`,
  `jja::ctg::invert_board`, `jja::ctg::needs_flipping`, `jja::ctg::flip_board`
  which is a **breaking change**.
- quote now also accepts a search term as a case-insensitive regular expression as
  well as a quote index.
- The `progress_bar` member of `jja::abkbook::AbkBook` has been removed, in return
  many public functions of `jja::abkbook::AbkBook` require an optional reference to
  a progress bar now. This avoids a needless clone of the progress bar and it is a
  **breaking change**.
- find now memory maps the ABK opening book files rather than reading the whole file
  into memory at once for efficiency. `jja::abkbook::AbkBook` no longer implements
  `Clone` which is a **breaking change**.
- edit now uses buffered writing when converting books to the ABK opening book
  format which improves efficiency. The function `jja::AbkBook::write_file` has been
  changed to take an argument a `BufWriter<W: Seek + Write>` rather than a `File`
  which is a **breaking change**.
- the positions table, `p`, in jja-0 databases now have an index on id, `p_idx` so
  as to be able to query for Zobrist hash collisions more efficiently.
- info prints file type in uppercase rather than lowercase now.
- edit learned to print Polyglot book information after a successful CTG conversion,
  like we already do for CTG to ABK conversions.
- CTG gained support for the full range of NAGs, `$0 - $255`, thus edit no longer
  panics when stumbling upon a previously unsupported NAG. Note, only the move
  assessments, `$1 - $9`, are used in Polyglot weight and ABK priority calculation
  during edit. Other NAGs are merely used for display for the find subcommand. Note,
  this changes the `jja::ctg::Nag` public type, and hence is a **breaking change**.
- dump learned `-f=<FORMAT>`, `--format=<FORMAT>` argument to choose the dump format
  of PGN dumps. This option has no effect on non-PGN dumps. The function
  `jja::pgn::pgn2csv` has been renamed to `jja::pgn::pgn_dump` which is a
  **breaking change**.
- prioritize CTG coloured move recommendations over Numeric Annotation Glpyhs (NAGs)
  in ABK priority calculation. edit learned three command-line parameters which are
  `--color-priority-green`, `--color-priority-blue`, and `--color-priority-red`.
  Their default values are `9`, `5`, and `1` respectively.
- make learned `--min-wins` which can be used to filter moves by their win count.
  The default value is `0` which has no effect.
- prioritize CTG coloured recommendations over Numeric Annotation Glpyhs (NAGs) in
  Polyglot weight calculation. edit learned three command-line parameters which are
  `--color-weight-green`, `--color-weight-blue`, and `--color-weight-red`. Their
  default values are `10000`, `1000`, and `1` respectively.
- CTG numerical annotation glyph `7` which is a forced move is now supported.
  Previously, we mistakenly used `8` which was an only move, not a forced move.
  Although the distinction is not really clear, we've implemented `ctg::Nag::Only`
  in addition to `ctg::Nag::Forced`, and the corresponding command-line flags for
  jja edit are `--nag-weight-only=<WEIGHT>`, and `--nag-priority-only=<PRIO>`. Both
  defaults are identical to the default values of `--nag-weight-forced`, and
  `--nag-priority-forced`.
- **important fix** `jja::CtgBook::search_position` function from mistakenly missing
  some positions causing some huge CTG books, larger than ~2,5-3G, to be seen as
  having 0 positions during edit, or causing position lookups to fail during find. A
  multiplication overflow in `jja::CtgBook::read_page` function is also fixed.
- make informs user about the `--{win,draw,loss}-factor` values during filtering as
  they're also important in determining weight, and preserval of entries.
- fix a bug in make where negative values in `--draw-factor`, and `--loss-factor`
  was not counted as deficits.
- dump and restore learned an experimental PGN to an array of chess position Zobrist
  keys and setups in streaming JSON. restore can save these dump into a sqlite3
  database with `.jja-0` extension. The format of this file is **experimental**, and
  is subject to change. Once this format is stable, the extension `.jja-1` is going
  to be used. We're using this format currently only to detect Zobrist hash
  collisions.
- use the [`XorShift`](https://www.jstatsoft.org/v08/i14/paper) random number
  generator to randomly pick moves during random playouts using the play command. This
  algorithm is cryptographically insecure but is very fast. See the benchmark
  [here](https://git.sr.ht/~alip/jja/commit/b2659d05c883f33d03d3e5c717759580b528919b)
- **breaking change**: `jja::quote::print_quote` now requires a second argument which
  is a boolean which specifies whether the output should be formatted with ANSI
  colour codes or not. By default, when the standard output is not a TTY,
  `print_quote` will now print quote, and author information without styling.
- fix PGN dumps to produce proper JSON arrays, previously the array markers `[]`
  were erroneously not printed out.

v0.6.1 1 year, 7 months ago .tar.gz browse log

Release v0.6.1

- fix deserializing of promotions of BrainLearn experience file entries.
- edit learned to convert BrainLearn experience files to PGN. Note a lot of memory
  may be required in converting these files. This is planned to be improved in the
  future.
- fix deserializing of castling moves of BrainLearn experience file entries.
- fix `jja restore` which incorrectly wrote BrainLearn entries in big-endian rather
  than little-endian.
- fix sorting of BrainLearn entries in `jja find` tree output.
- upgrade `rust-embed` crate from `6.7` to `6.8`.
- upgrade `num_cpus` crate from `1.15` to `1.16`.
- find can now correctly query BrainLearn experience files using Stockfish
  compatible Zobrist hashes.
- hash learned `-S, --stockfish` to generate Stockfish compatible Zobrist hashes.
- New public module `jja::stockfish`, and new public function
  `jja::stockfish::stockfish_hash` to generate Stockfish compatible Zobrist hashes.
  Before calling this function `jja::stokfish::zobrist::init` function must be
  called once to compute hashtables used in Zobrist hashing.
- New public functions `jja::chess::{de,}serialize_chess` to serialize/deserialize a
  `shakmaty::Chess` instance to/from an array of 5 unsigned 64-bit numbers. `jja
  dump` uses this functionality in PGN dumps.

v0.6.0 1 year, 7 months ago .tar.gz browse log

Release v0.6.0

Changes:

- `jja::pgnfilt::Operator` and `jja::pgnfilt::LogicalOperator` implements `Eq` as
  well as `PartialEq` now.
- new `restore` command to accompany the `dump` command which restores JSON
  serialized PolyGlot or BrainLearn file entries into the given output file.
- fix `SIGPIPE` handling on UNIX systems so that `jja` does not panic when the
  output is piped to another program such as a pager.
- find learned `-z <HASH>, --hash=<HASH>` to query PolyGlot opening books and
  BrainLearn experience files by Zobrist hash.
- readonly support for BrainLearn experience file format. The subcommands `info`,
  `dump`, and `find` are able to handle files in BrainLearn experience file format
  with the extension `.exp`.
- improve polyglot key lookup by reading only the key rather than the whole entry
  from file. The new public function `polyglotbook::PolyGlotBook::read_book_key` is
  used for that.
- new `dump` command to dump the full contents of a PolyGlot opening book or a PGN
  file. The dump format of the PolyGlot opening book is JSON, whereas for PGN files
  this is CSV.
- **breaking change**: `polyglotbook::PolyglotBook::lookup_moves` has been changed
  to take a zobrist hash of a chess position as an argument rather than the
  `shakmaty::Chess` position itself.
- hash learned `--signed` to print Zobrist hashes as signed decimal numbers.
- new module `jja::file` which exports utilities for binary file i/o.
- **breaking change**: `jja::polyglot::entry_{from,to}_file` have been renamed to
`jja::polyglot::bin_entry_{from,to}_file`.
- use the [`XorShift`](https://www.jstatsoft.org/v08/i14/paper) random number
  generator to randomly pick moves during book matches using the match command. This
  algorithm is cryptographically insecure but is very fast.
- fix match command from panicing on certain cases when there is a book lookup miss.
- Print more detailed build information on long version, `--version` output.
- Memory map CTG files to speed up random access in return for increased memory
  usage. This brings a new dependency upon the crate `memmap`.
- use buffered read/write in interacting with PolyGlot books which reduces the
  read/write system calls by a huge margin and thereby improves performance. The
  `book` member of `PolyGlotBook` is now a `BufReader<File>` rather than a `File`
  which is a **breaking change**.
- **important fix** for calculating PolyGlot compatible Zobrist hashes wrt.
en-passant legality. In PolyGlot format, en-passant moves are only pseudo-legal
whereas previously the `jja::hash::zobrist_hash` function mistakenly checked
for full legality.
- **breaking change**: new type `ctgbook::CtgTree` which holds the new return value
  of the functions `CtgBook::extract_all`, and `CtgBook::extract_all2`. The `tree`
  element of `CtgEntry` which were used by these functions has also been dropped and
  the functions have been implemented in a much more performant way using
  considerably less memory. As a result, most ctg to abk/polyglot conversions are
  almost double as fast.
- **breaking change**: `ctgbook::CtgEntry` member uci's type has been changed from
  `String` to `shakmaty::uci::Uci`, and the `nags` member has been renamed to `nag`
  and its type has been changed from `Option<String>` to `Option<Nag>`.
- ctg move comment entries were parsed and silently discarded, they're no longer
  parsed. Moreover, `ctgbook::CtgEntry` no longer has a `comment` member which is a
  **breaking change**.
- **breaking change**: `ctg::colored_uci` function now accepts a
  `shakmaty::uci::Uci` rather than a UCI string. The order of the function arguments
  is also changed.
- ctg: new type `Ctg::Nag` to abstract CTG NAG (Numeric Annotation Glyph) entries
- obk entries with zero weight, are now assigned weight `1` during PolyGlot
  conversion to prevent skipping these entries. We plan to make this
  user-configurable in the future.
- use the standard `IsTerminal` trait, and drop the dependency on `is_terminal`
  crate
- use the standard `to_le_bytes()`, `to_be_bytes()`, and drop the dependency on
  `byteorder` crate
- hash learned `-e=<MODE>`, `--enpassant-mode=<MODE>` to select en-passant mode when
  to include the en-passant square in Zobrist hash calculation.
- **important fix** for encoding castling moves during book conversions to the
  PolyGlot format. See details in the [respective issue](https://todo.sr.ht/~alip/jja/13)
- **breaking change**: `polyglot.from_uci` now expects a `bool` argument
  to correctly encode castling positions by determining whether the king to move is
  on their starting square.
- bump minimum supported Rust version (MSRV) from `1.64` to `1.70` due to `shakmaty` bump
- drop dependency on the unmaintained and insecure `chrono` crate
- `PolyGlotBook` has two new public functions: `find_book_key`, and
  `read_book_entry`
- hash learned `-x`, `--hex` to print hash as hexadecimal rather than decimal
- upgrade `pgn-reader` crate from `0.24` to `0.25`
- upgrade `shakmaty` crate from `0.25` to `0.26`
- optimize various ctg functions
- info learned to print the total number of positions for `ctg` opening books
- **breaking change**: `CtgBook::num_entries` has been renamed to `total_positions`.
- **breaking change**: `CtgBook::total_pages` function accepts a reference to
  `self`, rather than consuming `self`. The return type is `usize` now.
- **breaking change**: Drop `close` functions of `CtgBook` and `PolyGlotBook`,
  improve `CtgBook` to close the cto file immediately after open, and keep a `File`,
  rather than an `Option<File>` in `CtgBook`. The now unused `path` element of
  `CtgBook` is also dropped.
- find: simplify & optimize polyglot entry lookup
- display license, version and author information in `--help` output
- set minimum supported Rust version (MSRV) to `1.64` as determined by `cargo-msrv`
- fix a bug which caused key and epd to be displayed incorrectly in editor screen
- edit: implement `--rescale` for PolyGlot books. When specified, edit will rescale
  weights of all entries in the book, rather than a single entry. This is useful to
  quickly correct/optimize PolyGlot books which were generated without weight scaling.
- fix build with `i18n` feature disabled
- upgrade `tempfile` crate from `3.5` to `3.6`
- upgrade `rust-embed` crate from `6.6` to `6.7`
- upgrade `once_cell` crate from `1.17` to `1.18`
- upgrade `ctrlc` crate from `3.3` to `3.4`
- **important fix** CTG to Polyglot weight conversion which caused all entries in
  CTG books with missing performance information to be skipped from the output
  PolyGlot book. Read more about it in the [respective
  issue](https://todo.sr.ht/~alip/jja/12).
- make learned `-H`, `--hashcode` argument to skip duplicate games based on the
  **HashCode** PGN tag. PGN files may be tagged using `pgn-extract --addhashcode`

v0.5.0 1 year, 8 months ago .tar.gz browse log

Preparation for v0.5.0

- **important fix, for encoding of castling in polyglot books**. Previously, we
  encoded castling as e1g1, e1c1, e8g8, and e8c8, whereas the correct encoding is
  e1h1, e1a1, e8h8, and e8a8.
- when printing version with `--version`, prefer `git` version over package version
  for git builds
- upgrade `clap` crate from `4.2` to `4.3`
- upgrade `ctrlc` crate from `3.2` to `3.3`
- upgrade `rocksdb` crate to `0.21.0` which bundles RocksDB-8.1.1
- document all the public code and enable the lint `#[deny(missing_docs)]`
- edit learned to export opening books in PGN format, use an output file with `pgn`
  extension to export an opening book to a PGN
- add back the build dependency upon the [built](https://crates.io/crates/built) crate
- find learned `-l <max-ply>`, `--line=<max-ply>` to display lines from the opening
  book as a table of opening variations reverse-sorted by cumulative weight
- make `--min-score` now accepts floating point values as argument rather than an
  unsigned 64-bit integer
- make learned `--win-factor`, `--draw-factor`, and `--loss-factor` to specify
  respective factors during score calculation, the defaults, `2`, `1`, and `0`
  respectively, resembles the original `polyglot` tool
- find `--tree=<max-ply>` no longer panics on broken pipe, so it's more convenient
  to use with a pager
- make learned `-p, --min-pieces` to specify the minimum number of pieces on the
  board for a position to be included in the book, defaults to `8`
- merge learned `-c, --weight-cutoff` to specify the minimum weight of entries to be
  included in the book
- merge learned about merge strategies `avg` and `wavg` to merge using average
  weight or weighted average weight respectively; the weighted averages should be
  specified for `wavg` using `-w, --weight1`, and `-W, --weight2`
- in-place editing of polyglot files allows editing empty books which makes it
  practical to create polyglot book from scratch
- merge learned about merge strategies `max`, `min`, `ours`, and `sum`, default is
  `sum` which adds together move weights, `max` picks the one with the maximum
  weight, `min` picks up the one with the minimum weight, and `ours` always picks
  the entries from the first book
- translate the README to German language
- turn i18n support into a feature which defaults to on. For static linking this
  must be disabled as embed makes use of proc macros.
- drop depedency on unused [libc](https://crates.io/crates/libc) crate

v0.4.1 1 year, 10 months ago .tar.gz browse log

Release v0.4.1

- fix [docs.rs](https://docs.rs/jja) build
- update [shakmaty](https://crates.io/crates/shakmaty), and
  [pgn-reader](https://crates.io/crates/pgn-reader) creates
- add initial translation to Turkish language
- add initial translation to German language
- use gettext for i18n
1 / 2

Branches

main
0ca30300 — Ali Polatel 26 days ago
shakmaty-ctg-experiment
Do not follow this link