quote: new entry, thx to Xenguy
preparation for v0.9.1
cargo clippy
This is jja, a command line utility to interact with various chess file formats. It is still in its early stages of development. The initial intention of the author was to convert their opening books which were saved with ChessBase's proprietary CTG format to the free and open PolyGlot format. Overtime they intend to add support for other chess file formats ( cbh, si4, si5 and so on).
I've started hacking this on International Women's Day 2023, a day to honor the achievements of women and advocate for their rights worldwide. As a person of Kurdish heritage, I am particularly moved by the slogan "Woman, Life, Freedom", which has become a symbol of resistance against oppression and a call for equality. In the spirit of free software and free speech, I strive to contribute to the creation of a more just and inclusive society, where every human being is granted the freedom to express themselves and pursue their dreams. I also honor the memory of Mahsa Amini, whose tragic death reminds us of the urgent need to fight for women's freedom and safety.
More on Wikipedia, WikiPedia::Woman, Life, Freedom
As of version 0.7.0, jja supports reading/querying:
bin
abk
ctg
obk
(version 1 and 2, w\o,with text notes)exp
opening book files, whereas it supports writing/converting to:
bin
abk
exp
opening book files.
As of version 0.5.0, jja supports exporting all the supported opening book
formats to PGN. To
use this functionality, specify an output file with pgn
extension as an argument
to jja edit
.
As of version 0.8.0, jja supports exporting all positions in a
PGN file into an
output EPD file.
To use this functionality, specify an input file with pgn
extension (or a
compressed pgn: pgn.bz2
, pgn.gz
, pgn.lz4
, pgn.xz
or pgn.zst
) and an output
file with epd
extension as arguments to jja edit
.
During opening book conversion, jja uses the information provided in various
input opening book formats to come up with a move weight which accompanies the
move in the PolyGlot opening file.
jja also writes some custom numbers in the learn field, such as
NAGs during ctg
conversion or priority during abk
conversion. You may disable this custom
usage using --no-learn
as it may confuse other software making use of this field.
Note, Arena, aka abk
, opening book file
writing support is only supported from
ChessBase, aka ctg
books. Use the command
line flags --author
, --comment
, --probability-priority
, --probability-games
,
--probability-win-percent
to configure ABK
header metadata. Game statistics (minimum number of games/wins, win percentages for
both sides) are managed automatically by jja.
In-place editing for Arena opening books is
also possible using -i, --in-place=SUFFIX
command line option. Conversion from
PolyGlot, aka bin
, and
ChessMaster, aka obk
opening books
to Arena, aka abk
opening book files is
planned for a future release.
jja info
to get brief information about the chess file.jja find
to search for a position in a chess file.jja edit
to edit opening book files and convert to
PolyGlot files or
Arena files.jja make
to compile
PGN files into
PolyGlot opening books.jja dump
to dump a PolyGlot
or BrainLearn file as a stream of
JSON arrays.jja restore
to restore a PolyGlot
or BrainLearn file from a stream of
JSON arrays.jja merge
to merge two PolyGlot
opening books.jja match
to arrange book matches using random playouts.jja play
to make random playouts, optionally using books.jja digest
to calculate Zobrist hash of a given chess position.jja perft
to calculate node count from a given position up to a given depth.jja open
to browse ECO
classification.jja quote
to print a chess quote.jja determines the type of the file using its file extension. Files with the
extension .bin
are considered PolyGlot
books. Files with the extension .ctg
are considered
ChessBase books. Files with the extension
.abk
are considered Arena books. Files
with extension .obk
are considered
ChessMaster books. Files with
extension .exp
are considered BrainLearn
experience files.
By default if the standard output is a
TTY, jja will display
information using fancy tables. Use --porcelain
command line option to get the
output in CSV
(comma-separated values) format instead.
To compile from source, use cargo install jja
. This requires the Rust
Toolchain to be installed. In addition you are going to need
OpenSSL libraries on
UNIX systems. Moreover you need
liburing on Linux. If
you're on a Linux system older than
5.1 or you are unable to install
liburing for another reason, you may disable
the feature by building jja with cargo install jja --no-default-features
.
As an alternative, release builds of jja are hosted on chesswob.org for 64-bit Linux and Windows. These versions are signed by GnuPG, using key D076A377FB27DE70. To install, acquire the latest version from chesswob.org, verify the checksum and the GnuPG signature:
$> export JJA_VERSION=0.7.1
$> export JJA_FLAVOUR=glibc
$> curl https://keybase.io/alip/pgp_keys.asc | gpg --import
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13292 100 13292 0 0 13535 0 --:--:-- --:--:-- --:--:-- 26584
gpg: key D076A377FB27DE70: public key "Ali Polatel (Caissa AI) <alip@caissa.ai>" imported
gpg: Total number processed: 1
gpg: imported: 1
$> for f in jja-${JJA_VERSION}-${JJA_FLAVOUR}.bin{,.sha512sum,.sha512sum.asc}; do wget -q https://chesswob.org/jja/${f}; done
$> gpg --verify jja-${JJA_VERSION}.bin.sha512sum.asc jja-${JJA_VERSION}.bin.sha512sum
gpg: Signature made Sun Mar 19 20:52:41 2023 CET
gpg: using RSA key 5DF763560390A149AC6C14C7D076A377FB27DE70
gpg: Good signature from "Ali Polatel (Caissa AI) ...
$> sha512sum -c jja-${JJA_VERSION}.bin.sha512sum
jja: OK
$> sudo install -m755 jja-${JJA_VERSION}-${JJA_FLAVOUR}.bin /usr/local/bin
Finally you may download the builds of the latest git version via the
SourceHut build server. There're three flavours,
windows
, linux-glibc
, and linux-musl
. Simply browse to the latest build and
download the artifact listed on the left. Note: these artifacts are kept for 90
days.
jja is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.
Hey you, out there beyond the wall,
Breaking bottles in the hall,
Can you help me?
Report bugs to jja's bug tracker at https://todo.sr.ht/~alip/jja/:
Since version 0.4.0, jja can make
PolyGlot books out of
PGN
files. This feature is similar to polyglot make-book
with the following
differences:
--no-scale
to disable.The filter expression string should contain filter conditions, which consist of a tag name, a comparison operator, and a value. The following operators are supported:
>
(greater than)>=
(greater than or equal to)<
(less than)<=
(less than or equal to)=
(equal to)!=
(not equal to)=~
(regex match, case insensitive)!~
(negated regex match, case insensitive)Filter conditions can be combined using the following logical operators:
AND
(logical AND)OR
(logical OR)Example:
--filter="Event =~ World AND White =~ Carlsen AND ( Result = 1-0 OR ECO = B33 )"
Supported tags are Event, Site, Date, UTCDate, Round, Black, White, Result, BlackElo, WhiteElo, BlackRatingDiff, WhiteRatingDiff, BlackTitle, WhiteTitle, ECO, Opening, TimeControl, Termination, TotalPlyCount, and ScidFlags.
In addition to these are four special variables, namely, Player, Elo, Title, and RatingDiff. These variables may be used to match the relevant header from either one of the sides. E.g the filter:
--filter="Player =~ Carlsen"
is functionally equivalent to
--filter="( White =~ Carlsen OR Black =~ Carlsen )"
Note: The filtering is designed to be simple and fast. The tokens, including parantheses are split by whitespace. Quoting values is not allowed. For more sophisticated filtering needs, use pgn-extract.
Scid uses one character flags, DWBMENPTKQ!?U123456
, for each field where:
D
- DeletedW
- White openingB
- Black openingM
- MiddlegameE
- EndgameN
- NoveltyP
- Pawn structureT
- TacticsK
- Kingside playQ
- Queenside play!
- Brilliancy?
- BlunderU
- User-defined1..6
- Custom flagsIt is ill-advised to rely on the order of the characters flags.
Use a regex match if/when you can.
--max-open-files=-1
to keep
files always open.4
, are
aimed at speedy conversion with relatively moderate space usage. If you run out
of space during conversion, try to use an algorithm like
Zstd with an "ultra" level, ie. a level
greater or equal to 20
.--filter=<expr>
,
--max-ply=<ply>
, --min-pieces=<piece-count>
, --only-white
, and
--only-black
.Thanks to Steinar H. Gunderson, for publishing the CTG
Specification,
and authoring the remoteglot tool:
The CTG probing code in jja is very
directly ported from their
C probing code,
and the specification has been an enormous help in clearing up various rough edges.
Thanks to Fabien Letouzey, the author of the original
PolyGlot software: The
PolyGlot probing, book making and merging
code in jja is mostly ported from their respective
C code. Thanks to
Michel Van den Bergh, the author of pg_utils, a collection of tools to
interact with PolyGlot opening books: The
PolyGlot book editing code of jja
uses many ideas and code excerpts from pg_utils. Thanks to Peter Österlund,
the author of DroidFish: The
ABK opening book interface code in jja
makes use of ideas and code excerpts from
DroidFish. Thanks to Jens
Nissen, the author of ChessX: The
CTG and
ABK probing codes in jja use ideas and
code excerpts from ChessX. Thanks to
LiChess, the best chess website on the planet. The quote
command of jja has a selection of quotes imported from the
LiChess codebase. Thanks to Shane Hudson, the author of
Scid vs. PC: The jja eco
command uses the
ECO classification which has been done by
the Scid project. In addition, the
PolyGlot editing code of jja uses
ideas and code from Scid. Thanks to Marcus
Bufett, the author of
chess-tactics-cli: The
chessboard displaying code in PolyGlot and
ABK edit screens is borrowed from
chess-tactics-cli.
nix
crate from 0.27
to 0.28
.tempfile
crate from 3.8
to 3.10
.rust-embed
crate from 8.1
to 8.4
.rusqlite
crate from 0.30
to 0.31
.rayon
crate from 1.8
to 1.10
.hostname
crate from 0.3
to 0.4
.TotalPlyCount
PGN tag is now supported in make filters.--win,draw,loss-factor
calculation for makeonce_cell
crate from 1.18
to 1.19
.rayon
crate from 1.7
to 1.8
.zstd
crate from 0.12
to 0.13
.rusqlite
crate from 0.29
to 0.30
.dialoguer
crate from 0.10
to 0.11
.i18n-embed
crate from 0.13
to 0.14
.rust-embed
crate from 6.8
to 8.0
.built
crate from 0.6
to 0.7
.-O<u16>, --outlier-threshold=<u16>
to specify an outlier threshold
where moves with weights higher than the given threshold are filtered out from the
merged output book. This stands from the fact that if the weight differences of
the same move in two different books have a high difference, calculating any
average may not yield meaningful results. The default is 0, which disables this
feature.--strategy dynmid
which stands for
jja::merge::MergeStrategy::DynamicMidpoint
.--strategy wdist
which stands for
jja::merge::MergeStrategy::WeightedDistance
.nix
crate from 0.26
to 0.27
and add the signal
feature.--strategy wmedian
which stands for
jja::merge::MergeStrategy::WeightedMedian
.--strategy entropy
which stands for
jja::merge::MergeStrategy::Entropy
.--strategy quadratic
which stands for
jja::merge::MergeStrategy::QuadraticMean
. This strategy merges using the Quadratic
Mean (Root Mean Square) approach. The Quadratic Mean (also known as the Root Mean
Square) is a statistical measure of the magnitude of a set of numbers. It offers a
more balanced view, especially when dealing with numbers of varying magnitudes.--strategy lavg
which stands for
jja::merge::MergeStrategy::LogarithmicAverage
. This strategy merges using a
logarithmic averaging approach. Given that logarithmic functions compress large
values and expand small values, we can use them to get a merge strategy that's
sensitive to differences in smaller weights while being more resistant to
disparities in larger weights.-c, --cutoff
is now applied after the merge
strategy assigns new weights. This way, cutoff may also be used to filter out
unwanted entries after the merge.edit --color-weight-blue
has been changed from 32768
to
65280
for consistency with --nag-weight-good
.jja::merge::MergeStrategy::Sort
to calculate
weight based on the relative position of the move entry in sorted move entries.jja::merge::MergeStrategy::GeometricScaling
to
calculate the weight using geometric scaling: The geometric scale focuses on
multiplying numbers together rather than adding, which can help in equalizing
disparities.jja::merge::MergeStrategy::HarmonicMean
to
calculate the harmonic mean weight of the identical move entries in both books.
This approach tends to favor more balanced weights and is less influenced by
extreme values.jja::merge::MergeStrategy::Sigmoid
to remap the
weights in a non-linear fashion using the sigmoid function. The idea here is to
diminish the influence of extreme values, which might be causing the
dissatisfaction in previous strategies.-C, --no-colors
, and -N, --no-nags
to avoid
using CTG move colours and NAGs to assign PolyGlot move weights or Arena move
priorities.info
on the output book after successful run.pavg
strategy rather than sum
now.--strategy pavg
which stands for
jja::merge::MergeStrategy::PercentageAverage
. This strategy merges by computing
the weighted average of percentage weights, taking into account the total number
of entries in each book. This approach gives higher importance to moves from
larger books versus smaller ones, ensuring that the resultant weights reflect the
relative contributions of each book based on its size.--rescale
to rescale weights of merged book entries globally to
fit into 16 bits. This is similar to edit --rescale
but works on the final
merged book rather than the input books.estimate-num-keys
property of the temporary RocksDB database to
determine the approximate unique position count rather than bulk scanning the
database twice which improves performance.--read-ahead=<SIZE>
commandline
option. This defaults to 4 MB
and may be disabled by passing 0
as size.io_uring
feature enabled. This may be disabled using the new --sync
commandline argument
or using the environment variable JJA_SYNC
.smallvec
crate usage, and remove dependency.humansize
with bytefmt
crate.jja::abk::traverse_tree
function now returns a BTreeMap<u64, Vec<CompactBookEntry>
rather than a BTreeMap<u64, Vec<BookEntry>
which is a
breaking change. Moreover this function now sorts the value vectors using
binary search which makes abk->bin
both more memory efficient and faster.jja::ctg::ByteBoard::from_position
has been replaced with the function
from_board
which is a breaking change.jja::ctgbook::CtgBook::process_move
function now utilizes binary search for move
table lookup improving efficiency. jja::ctg::MOVETABLE
's encoding
element has
changed its type from char
to u8
which is a breaking change.jja::polyglot::CompactBookEntry
data structure.jja::abk::PackedSBookMoveEntry
has been renamed to CompactSBookMoveEntry
which
is a breaking change.jja::ctgbook::CtgBook::extract_all{,2}
and the unused
type jja::ctgbook::CtgTree
both of which are breaking changes.jja::ctgbook::CtgBook::extract_map{,2}
to extract_abk{,2}
for
consistency which is a breaking change.extract_bin
in CTG->BIN
to improve efficiencyjja::ctgbook::CtgBook::extract_bin{,2}
human-panic
crate for user friendly panic messages.tempfile
crate from 3.7
to 3.8
.clap
crate from 4.3
to 4.4
.i18n
feature disabled.jja::ctgbook::CtgBook::extract_map{,2}
which directly returns a BTreeMap<u64, Vec<SBookMoveEntry>>
rather than processing a
CtgTree
into a SBookMoveEntryHashMap
as an additional step.jja::abkbook::AbkBook::total_entries()
to calculate the
number of entries in an Arena opening book file.CTG->ABK
conversion.lazy_static!
usages with once_cell
and drop the dependency
on lazy_static
crate. Our usage of once_cell
is currently in Rust nightly and
is hopefully soon going to land in stable Rust.perft
subcommand to count legal move paths of a given lengthpgn-reader
crate from 0.24
to 0.25
shakmaty
crate from 0.25
to 0.26
tempfile
create from 3.6
to 3.7
abk
, bin
, ctg
, exp
, obk
) so users are highly
recommended to re-export their previously exported PGNs using jja edit book.{abk,bin,ctg,exp,obk} -o book.pgn
.1.64
to 1.70
..
, ?
, or *
.jja edit source.pgn{,.bzip2,gz,lz4,xz,zst} -o output.epd
.-e
, --elements
to specify a list of elements to dump which
defaults to id, position
. This option works only for PGN dumps.probe
which can be used to probe Syzygy tablebases up to 7
pieces. This is almost functionally identical to the awesome Fathom tool, but also
offers some alternative modes such as --test --fast
to skip walking the DTZ line
at the cost of misevaluating MaybeWin
and MaybeLoss
positions.--max-ply=<PLY>
, and
--look-ahead=<PLY>
. The former limits PGN generation to a certain number of
plies, and defaults to 1024
. The latter is used to specify the number of plies
to look ahead on PolyGlot book lookup misses during PGN generation which is useful
to generate PGNs of book generated by --only-black|white
. Currently
--look-ahead
only supports 0
, and 1
as argument, of which 0
being the
default value, other values will generate an unimplemented
panic which directs
the user to report a bug. This change comes along with a change in the public
function signature of jja::polyglotbook::PolyGlotBook::write_pgn
which is a
breaking change.jja::polyglot::from_move
now takes argument a reference to a
shakmarty::Move
rather than a shakmaty::Move
which is a breaking change.jja edit
have been
changed to be more intuitive, in that default value of --color-weight-green
has
been changed from 10000
to 65520
, --color-weight-blue
from 1000
to
32768
, --nag-weight-good
from 9000
to 65280
, --nag-weight-hard
from
10000
to 65520
, --nag-weight-interesting
from 7500
to 61440
,
--nag-weight--forced
from 10000
to 65520
, and --nag-weight-only
from
10000
to 65520
. The new defaults are empirical, and advanced users are
recommended to use --color-weight-*
, and --nag-weight-*
arguments with jja edit
for fine-tuning during CTG->BIN
or CTG->ABK
conversions.jja::ctgbook::CtgBook::read_page
function to prevent panics
due to out-of-bounds access. This fixes search & conversion with some huge CTG
books.jja::abkbook::AbkBook::traverse_book_and_merge
function now returns
nothing rather than Result<(), Box<dyn std::error::Error>>
which is a breaking
change.jja::chess::lines_from_tree
now returns a Table
rather than a
Result<Table, Box<dyn std::error::Error>>
which is a breaking change.anyhow
crate for error handling. This is mainly used in the main code, and
not the library code but there are points where the library code is changed, and
there are breaking changes: jja::pgnbook::create_opening_book
now returns an
anyhow::Result<GameBase>
, rather than
Result<GameBase, Box<dyn std::error::Error>>
. Similarly the function
jja::pgn::pgn_dump
now returns anyhow::Result<()>
rather than
Result<(), Box<dyn std::error::Error>>
.jja::system::get_progress_{bar,spinner}
now return a ProgressBar
rather than a Result<ProgressBar, Box<dyn std::error::Error>>
. These functions
now panic when there is an error in the template, which shouldn't happen normally.
This change is a breaking change.jja::system::edit_tempfile
now returns EditTempfileError
on error
rather than Box<dyn std::error::Error>
. Moreover the EditTempfileError
enum
has a new member EditTempfileError::InputOutputError(std::io::Error)
. These
changes are in the public API and hence are breaking changes.jja dump -fcsv file.pgn | jja restore file.epd
to create an
Extended Position Description file of all the positions in the given Portable Game
Notation file. The EPD entries include the Zobrist hash of the positions in the id
field.jja::chess::deserialize_chess
.jja::chess::deserialize_chess
function now panics on invalid piece indexes
rather than silently continuing.hash
subcommand has been renamed to digest
. The former will work as an alias
until the next major version bump.i18n
feature disabled.#![deny(clippy::cast_lossless)]
for jja library and fix
offending code.#![deny(clippy::cast_precision_loss)]
for jja library and fix
offending code.jja::ctg::find_piece
returns an Option<i32>
rather than a Result<i32, std::io::Error>
, the functions jja::ctgbook::CtgBook::extract_all{,2}
return a
CtgTree
rather than a Result<CtgTree, std::io::Error>
, and the function
jja::ctgbook::CtgBook::lookup_moves
returns an Option<Vec<CtgEntry>>
rather
than a Result<Option<Vec<CtgEntry>>, std:io::Error>
.-f, --fen
, and -p, --pgn
arguments of jja edit
so it is possible to convert a subtree of the opening book into PGN. See the
respective issues #14, and
#16 for more details. This change deprecates
the function jja::chess::pgn_from_tree
, it is recommended to use the new
write_pgn
function of the respective opening book file.jja::polyglotbook::PolyGlotBook
implementation has seen many improvements to be
a leaner interface. PolyGlotBook::get{,_key}
functions no longer panic on
invalid indexes, rather return an Option<BookEntry>
rather than a BookEntry
.
This implementation avoids an ugly hack to map a dummy anonymous memory region for
PolyGlot books with zero-size to allow creating PolyGlot books from scratch (e.g:
touch new-file.bin && jja edit -i new-file.bin
). As these functions are public,
this is a breaking change.--weight-cutoff
has been renamed to --cutoff
and now supports filtering out by
min depth in BrainLearn experience files.1.70
to 1.64
for portability.is_terminal
crate rather than depending on >=rust-1.70.pgn-reader
crate from 0.25
to 0.24
.shakmaty
crate from 0.26
to 0.25
.regex
crate from 1.8
to 1.9
.smallvec
crate from 1.10
to 1.11
.progress_bar
member of jja::obkbook::ObkBook
has been removed, in return
the public functions jja::obkbook::ObkBook::{tree,traverse_tree}
require an
optional reference to a progress bar now. This avoids a needless clone of the
progress bar and it is a breaking change. Moreover, the function
jja::obkbook::ObkBook::read_moves
has been renamed to load
which is again a
breaking change.jja::obkbook::ObkBook::{read_moves,traverse_tree,tree}
which is a
breaking change.binary
format to dump PGNs in PostgreSQL binary output format. This
brings in a dependency on crate pgcopy
.BufReader<File>
handle to them. This removes the book
public member of
jja::polyglotbook::PolyGlotBook
, and changes signatures of public functions
jja::polyglotbook::PolyGlotBook::{lookup_moves,tree}
which is a breaking
change. This also changes names of the public functions
jja::polyglotbook::PolyGlotBook::{find_book_key,read_book_entry,read_book_key}
to jja::polyglotbook::PolyGlotBook::{find,get,get_key}
respectively which is
again a breaking change. Moreover jja::polyglotbook::PolyGlotBook
's default
iterator implementation has been changed to iterate over single book entries. New
function introduced jja::polyglotbook::PolyGlotBook::into_iter_grouped()
may be
used to iterate over entries grouped by key.-B
, --benchmark
, and -I
, --benchmark-iterations
to benchmark
Stockfish and Zobrist hash functions. This brings in a dependency on
benchmarking
crate.jja find
output. Note, this
change is for CSV output only which is printed when the output is not a TTY or the
option --porcelain=csv
is given.progress_bar
member of jja::ctgbook::CtgBook
has been removed, in return
the public functions jja::ctgbook::CtgBook::extract_all{,2}
require an optional
reference to a progress bar now. This avoids a needless clone of the progress bar
and it is a breaking change.jja::ctgbook::CtgBook
functions
improving efficiency. The function jja::ctgbook::CtgBook::lookup_moves
now
accepts a &dyn shakmaty::Position
rather than an EPD string which is a
breaking change.jja::ctg::find_piece
, jja::ctg::decode_fen_board
,
jja::ctg::invert_board
, jja::ctg::needs_flipping
, jja::ctg::flip_board
which is a breaking change.progress_bar
member of jja::abkbook::AbkBook
has been removed, in return
many public functions of jja::abkbook::AbkBook
require an optional reference to
a progress bar now. This avoids a needless clone of the progress bar and it is a
breaking change.jja::abkbook::AbkBook
no longer implements
Clone
which is a breaking change.jja::AbkBook::write_file
has been
changed to take an argument a BufWriter<W: Seek + Write>
rather than a File
which is a breaking change.p
, in jja-0 databases now have an index on id, p_idx
so
as to be able to query for Zobrist hash collisions more efficiently.$0 - $255
, thus edit no longer
panics when stumbling upon a previously unsupported NAG. Note, only the move
assessments, $1 - $9
, are used in Polyglot weight and ABK priority calculation
during edit. Other NAGs are merely used for display for the find subcommand. Note,
this changes the jja::ctg::Nag
public type, and hence is a breaking change.-f=<FORMAT>
, --format=<FORMAT>
argument to choose the dump format
of PGN dumps. This option has no effect on non-PGN dumps. The function
jja::pgn::pgn2csv
has been renamed to jja::pgn::pgn_dump
which is a
breaking change.--color-priority-green
, --color-priority-blue
, and --color-priority-red
.
Their default values are 9
, 5
, and 1
respectively.--min-wins
which can be used to filter moves by their win count.
The default value is 0
which has no effect.--color-weight-green
, --color-weight-blue
, and --color-weight-red
. Their
default values are 10000
, 1000
, and 1
respectively.7
which is a forced move is now supported.
Previously, we mistakenly used 8
which was an only move, not a forced move.
Although the distinction is not really clear, we've implemented ctg::Nag::Only
in addition to ctg::Nag::Forced
, and the corresponding command-line flags for
jja edit are --nag-weight-only=<WEIGHT>
, and --nag-priority-only=<PRIO>
. Both
defaults are identical to the default values of --nag-weight-forced
, and
--nag-priority-forced
.jja::CtgBook::search_position
function from mistakenly missing
some positions causing some huge CTG books, larger than ~2,5-3G, to be seen as
having 0 positions during edit, or causing position lookups to fail during find. A
multiplication overflow in jja::CtgBook::read_page
function is also fixed.--{win,draw,loss}-factor
values during filtering as
they're also important in determining weight, and preserval of entries.--draw-factor
, and --loss-factor
was not counted as deficits..jja-0
extension. The format of this file is experimental, and
is subject to change. Once this format is stable, the extension .jja-1
is going
to be used. We're using this format currently only to detect Zobrist hash
collisions.XorShift
random number
generator to randomly pick moves during random playouts using the play command. This
algorithm is cryptographically insecure but is very fast. See the benchmark
herejja::quote::print_quote
now requires a second argument which
is a boolean which specifies whether the output should be formatted with ANSI
colour codes or not. By default, when the standard output is not a TTY,
print_quote
will now print quote, and author information without styling.[]
were erroneously not printed out.jja restore
which incorrectly wrote BrainLearn entries in big-endian rather
than little-endian.jja find
tree output.rust-embed
crate from 6.7
to 6.8
.num_cpus
crate from 1.15
to 1.16
.-S, --stockfish
to generate Stockfish compatible Zobrist hashes.jja::stockfish
, and new public function
jja::stockfish::stockfish_hash
to generate Stockfish compatible Zobrist hashes.
Before calling this function jja::stokfish::zobrist::init
function must be
called once to compute hashtables used in Zobrist hashing.jja::chess::{de,}serialize_chess
to serialize/deserialize a
shakmaty::Chess
instance to/from an array of 5 unsigned 64-bit numbers. jja dump
uses this functionality in PGN dumps.jja::pgnfilt::Operator
and jja::pgnfilt::LogicalOperator
implements Eq
as
well as PartialEq
now.restore
command to accompany the dump
command which restores JSON
serialized PolyGlot or BrainLearn file entries into the given output file.SIGPIPE
handling on UNIX systems so that jja
does not panic when the
output is piped to another program such as a pager.-z <HASH>, --hash=<HASH>
to query PolyGlot opening books and
BrainLearn experience files by Zobrist hash.info
,
dump
, and find
are able to handle files in BrainLearn experience file format
with the extension .exp
.polyglotbook::PolyGlotBook::read_book_key
is
used for that.dump
command to dump the full contents of a PolyGlot opening book or a PGN
file. The dump format of the PolyGlot opening book is JSON, whereas for PGN files
this is CSV.polyglotbook::PolyglotBook::lookup_moves
has been changed
to take a zobrist hash of a chess position as an argument rather than the
shakmaty::Chess
position itself.--signed
to print Zobrist hashes as signed decimal numbers.jja::file
which exports utilities for binary file i/o.jja::polyglot::entry_{from,to}_file
have been renamed to
jja::polyglot::bin_entry_{from,to}_file
.XorShift
random number
generator to randomly pick moves during book matches using the match command. This
algorithm is cryptographically insecure but is very fast.--version
output.memmap
.book
member of PolyGlotBook
is now a BufReader<File>
rather than a File
which is a breaking change.jja::hash::zobrist_hash
function mistakenly checked
for full legality.ctgbook::CtgTree
which holds the new return value
of the functions CtgBook::extract_all
, and CtgBook::extract_all2
. The tree
element of CtgEntry
which were used by these functions has also been dropped and
the functions have been implemented in a much more performant way using
considerably less memory. As a result, most ctg to abk/polyglot conversions are
almost double as fast.ctgbook::CtgEntry
member uci's type has been changed from
String
to shakmaty::uci::Uci
, and the nags
member has been renamed to nag
and its type has been changed from Option<String>
to Option<Nag>
.ctgbook::CtgEntry
no longer has a comment
member which is a
breaking change.ctg::colored_uci
function now accepts a
shakmaty::uci::Uci
rather than a UCI string. The order of the function arguments
is also changed.Ctg::Nag
to abstract CTG NAG (Numeric Annotation Glyph) entries1
during PolyGlot
conversion to prevent skipping these entries. We plan to make this
user-configurable in the future.IsTerminal
trait, and drop the dependency on is_terminal
crateto_le_bytes()
, to_be_bytes()
, and drop the dependency on
byteorder
crate-e=<MODE>
, --enpassant-mode=<MODE>
to select en-passant mode when
to include the en-passant square in Zobrist hash calculation.polyglot.from_uci
now expects a bool
argument
to correctly encode castling positions by determining whether the king to move is
on their starting square.1.64
to 1.70
due to shakmaty
bumpchrono
cratePolyGlotBook
has two new public functions: find_book_key
, and
read_book_entry
-x
, --hex
to print hash as hexadecimal rather than decimalpgn-reader
crate from 0.24
to 0.25
shakmaty
crate from 0.25
to 0.26
ctg
opening booksCtgBook::num_entries
has been renamed to total_positions
.CtgBook::total_pages
function accepts a reference to
self
, rather than consuming self
. The return type is usize
now.close
functions of CtgBook
and PolyGlotBook
,
improve CtgBook
to close the cto file immediately after open, and keep a File
,
rather than an Option<File>
in CtgBook
. The now unused path
element of
CtgBook
is also dropped.--help
output1.64
as determined by cargo-msrv
--rescale
for PolyGlot books. When specified, edit will rescale
weights of all entries in the book, rather than a single entry. This is useful to
quickly correct/optimize PolyGlot books which were generated without weight scaling.i18n
feature disabledtempfile
crate from 3.5
to 3.6
rust-embed
crate from 6.6
to 6.7
once_cell
crate from 1.17
to 1.18
ctrlc
crate from 3.3
to 3.4
-H
, --hashcode
argument to skip duplicate games based on the
HashCode PGN tag. PGN files may be tagged using pgn-extract --addhashcode
--version
, prefer git
version over package version
for git buildsclap
crate from 4.2
to 4.3
ctrlc
crate from 3.2
to 3.3
rocksdb
crate to 0.21.0
which bundles RocksDB-8.1.1#[deny(missing_docs)]
pgn
extension to export an opening book to a PGN-l <max-ply>
, --line=<max-ply>
to display lines from the opening
book as a table of opening variations reverse-sorted by cumulative weight--min-score
now accepts floating point values as argument rather than an
unsigned 64-bit integer--win-factor
, --draw-factor
, and --loss-factor
to specify
respective factors during score calculation, the defaults, 2
, 1
, and 0
respectively, resembles the original polyglot
tool--tree=<max-ply>
no longer panics on broken pipe, so it's more convenient
to use with a pager-p, --min-pieces
to specify the minimum number of pieces on the
board for a position to be included in the book, defaults to 8
-c, --weight-cutoff
to specify the minimum weight of entries to be
included in the bookavg
and wavg
to merge using average
weight or weighted average weight respectively; the weighted averages should be
specified for wavg
using -w, --weight1
, and -W, --weight2
max
, min
, ours
, and sum
, default is
sum
which adds together move weights, max
picks the one with the maximum
weight, min
picks up the one with the minimum weight, and ours
always picks
the entries from the first book-i
, --irreversible
to prefer irreversible moves during random
playouts. Pawn moves, captures, moves that destroy castling rights, and moves that
cede en-passant are irreversible.-B <games>
, --batch-size=<games>
to determine write batch size in
number of games--no-default-features
on installation.ScidFlags
, a set of character flags used by the
Scid software, which may be used in filter
expressions--debug
flag to print information on matching filter expressions--max-open-files=<int>
to specify the maximum number of open files
per thread opened by the temporary RocksDB database.-T=<threads>
, --threads=<threads>
or the
JJA_NPROC
environment variable to overridePlayer
, Elo
, Title
, and RatingDiff
to match the relevant field from either colour--filter=<expression>
to filter PGN games by headers. The filtering
is designed to be simple and fast. The tokens, including parantheses are split by
whitespace. Quoting variables is not allowed. See jja make --help
for more
information on Filter Expressions-t <max-ply>
, --tree=<max-ply>
to display lines from the opening
book as a tree using the nice termtree crate--pgn
commandline option-0
, --null
--no-scale
--compression={none,bzip2,lz4,lz4hc,snappy,zlib,zstd}
,--compression-level=<level>
,
defaults to lz4
, level 4-T
, --threads
to override, works transparently with compressed PGN files (zstd, bzip2, gunzip, lz4)