~q3cpma/mangadex-tools

Diverse Mangadex tools
Fix cover downloading
Fix mdex_dl for chapter URLs
Adapt to MDex API change (why don't these monkeys maintain a changelog page for their important API changes?)

refs

master
browse  log 

clone

read-only
https://git.sr.ht/~q3cpma/mangadex-tools
read/write
git@git.sr.ht:~q3cpma/mangadex-tools

You can also use your local clone with git send-email.

                        MangaDex tools
                        ==============

        Overview
        --------

Some useful tools to interface with MangaDex via its JSON API:

+----------------------------------------------------------------+
| NAME                                                           |
|     mdex_dl.tcl - download MangaDex chapters                   |
|                                                                |
| SYNOPSIS                                                       |
|     mdex_dl.tcl [OPTION]... MANGA_URL [CHAPTER_NUM...]         |
|     mdex_dl.tcl [OPTION]... CHAPTER_URL...                     |
|     mdex_dl.tcl [OPTION]... MANGA_URL covers [VOLUME_NUM...]   |
|                                                                |
| DESCRIPTION                                                    |
|     Download each of the specified items (chapters or covers). |
|     If no item number list is specified, download all of them. |
|                                                                |
| OPTIONS                                                        |
|     -proxy PROXY_URL                                           |
|         Set the curl HTTP/HTTPS proxy.                         |
|                                                                |
|     -lang LANG_CODE                                            |
|         Only download chapters in this language.               |
|         Defaults to "en".                                      |
|                                                                |
|     -help                                                      |
|         Print this help message and exit.                      |
|                                                                |
+----------------------------------------------------------------+

+-----------------------------------------------------------------------------------+
| NAME                                                                              |
|     mdex_monitor.tcl - monitor MangaDex manga updates                             |
|                                                                                   |
| SYNOPSIS                                                                          |
|     mdex_monitor.tcl [OPTION]... CATALOG                                          |
|                                                                                   |
| DESCRIPTION                                                                       |
|     Read mangas to monitor from CATALOG, a file using the following syntax:       |
|           CATALOG = ITEM ITEM...                                                  |
|           ITEM = MANGA_ID | "{" MANGA_ID OPTION VALUE OPTION VALUE... "}"         |
|                                                                                   |
|     Comments are everything from a "#" to a newline or end of file.               |
|     Since this is actually a literal Tcl list, all the spaces used in these       |
|     syntax rules are runs of one or more whitespace characters ([[:space:]]+ in   |
|     POSIX EREs).                                                                  |
|                                                                                   |
|     The following item OPTIONs are available:                                     |
|           autodl                                                                  |
|               If VALUE is 1, new chapters for this manga are downloaded to the    |
|               directory specified via the -autodl-dir option. If the -autodl      |
|               option is set, using a value of 0 disables it.                      |
|                                                                                   |
|           group                                                                   |
|               Only download chapters having VALUE matching one of their group     |
|               names.                                                              |
|                                                                                   |
|           title                                                                   |
|               Use VALUE as title instead of the MangaDex provided one.            |
|                                                                                   |
|     For each item:                                                                |
|           Query MangaDex for the item's chapter list.                             |
|                                                                                   |
|           If this is the first run since it was added to CATALOG, nothing is done |
|           except for the creation of its (empty) Atom feed.                       |
|                                                                                   |
|           Else, if there are new chapters since the last run, the Atom feed is    |
|           updated and those chapters are downloaded if autodl is currently        |
|           enabled.                                                                |
|                                                                                   |
|     A database holding the last chapter timestamp for each item is maintained     |
|     next to CATALOG.                                                              |
|                                                                                   |
| OPTIONS                                                                           |
|     -proxy PROXY_URL                                                              |
|         Set the curl HTTP/HTTPS proxy.                                            |
|                                                                                   |
|     -lang LANG_CODE                                                               |
|         Only monitor new chapters in this language.                               |
|         Defaults to "en".                                                         |
|                                                                                   |
|     -autodl                                                                       |
|         Set the "autodl" option for every catalog item.                           |
|                                                                                   |
|     -autodl-dir DIRECTORY                                                         |
|         Where to auto download new chapters.                                      |
|         Defaults to the same directory as CATALOG.                                |
|                                                                                   |
|     -feed-dir DIRECTORY                                                           |
|         Where to store the chapter update feeds.                                  |
|         Defaults to the same directory as CATALOG.                                |
|                                                                                   |
|     -single-feed                                                                  |
|         Produce a single feed instead of one per manga.                           |
|                                                                                   |
|     -help                                                                         |
|         Print this help message and exit.                                         |
|                                                                                   |
+-----------------------------------------------------------------------------------+

+---------------------------------------------------------------------------------+
| NAME                                                                            |
|     mdex_convert_catalog.tcl - update monitor config files for new MangaDex API |
|                                                                                 |
| SYNOPSIS                                                                        |
|     mdex_convert_catalog.tcl [OPTION]... CATALOG_PATH                           |
|                                                                                 |
| DESCRIPTION                                                                     |
|     Convert the catalog found at CATALOG_PATH along with its corresponding      |
|     timestamp database to work with the new MangaDex API.                       |
|     Before that, the old versions are backuped to .old suffixed files.          |
|                                                                                 |
| OPTIONS                                                                         |
|     -proxy PROXY_URL                                                            |
|         Set the curl HTTP/HTTPS proxy.                                          |
|                                                                                 |
|     -help                                                                       |
|         Print this help message and exit.                                       |
|                                                                                 |
+---------------------------------------------------------------------------------+


        Examples
        --------

Download all the english chapters of Grappler Baki:
    $ mdex_dl.tcl https://mangadex.org/title/ea3122bb-0c28-4669-8686-d6df1274512f

Idem but only the chapters 11 and 12:
    $ mdex_dl.tcl https://mangadex.org/title/ea3122bb-0c28-4669-8686-d6df1274512f 11 12

or using chapter URLs:
    $ mdex_dl.tcl https://mangadex.org/chapter/{b0bf931e-da4f-4bdc-b03e-70d4eff3c323 \
          https://mangadex.org/chapter/39978fce-46c5-4b62-8130-00356573a619/1

Download the covers for volume 1, 2 and 3:
    $ mdex_dl.tcl https://mangadex.org/title/ea3122bb-0c28-4669-8686-d6df1274512f covers 1 2 3

Monitoring example with two mangas already monitored (including a new chapter for the first one)
and another being added:
    $ ls -1F manga_dir/
    Baki Dou (2018)_ea3122bb-0c28-4669-8686-d6df1274512f.xml
    Baki the Grappler_5e9a2a3e-f992-4af4-a887-cd130460c9e5.xml
    catalog.tcllist
    timestamps.tcldict
    $ cat manga_dir/catalog.tcllist
    5e9a2a3e-f992-4af4-a887-cd130460c9e5  # Baki Dou (2018)
    {ea3122bb-0c28-4669-8686-d6df1274512f
        autodl 1
        title "Baki the Grappler"
        group "TheSpinyBackTeam"
    }
    $ echo '{f1c716b5-e82f-4625-a646-280cfa66abad title "City Hunter"}' >>manga_dir/catalog.tcllist
    $ mdex_monitor.tcl -autodl manga_dir/catalog.tcllist
    [1/3] Processing manga 5e9a2a3e-f992-4af4-a887-cd130460c9e5...
    Downloading manga feed JSON...
    [1/1] Downloading Baki Dou (2018) - c093 [TheSpinyBackTeam]...
    Downloading @Home server URL JSON...
    <...curl output...>
    [2/3] Processing manga ea3122bb-0c28-4669-8686-d6df1274512f...
    Downloading manga feed JSON...
    No new chapters
    [3/3] Processing manga f1c716b5-e82f-4625-a646-280cfa66abad...
    Downloading manga feed JSON...
    New catalog item, monitoring chapter updates from now on
    $ ls -1F manga_dir/
    Baki Dou (2018) - c093 [TheSpinyBackTeam]/
    Baki Dou (2018)_ea3122bb-0c28-4669-8686-d6df1274512f.xml
    Baki the Grappler_5e9a2a3e-f992-4af4-a887-cd130460c9e5.xml
    City Hunter_ea3122bb-0c28-4669-8686-d6df1274512f.xml
    catalog.tcllist
    timestamps.tcldict

Idem using a single feed:
    ...
    $ mdex_monitor.tcl -single-feed -auto-dl manga_dir/catalog.tcllist
    ...
    $ ls -1F manga_dir/
    Baki Dou (2018) - c093 [TheSpinyBackTeam]/
    catalog.tcllist
    mangadex.xml
    timestamps.tcldict


        Dependencies
        ------------

* Tcl 8.6
* tcllib (json module)
* curl >= 7.67.0
* tdom (mdex_monitor.tcl only)