~mil/mobroute

v0.1.0 a month ago .tar.gz browse log

This is the first tagged release of Mobroute, the minimal
GTFS router with automatic feed ingestion. For more details
on the project you can see the README, project page at
http://sr.ht/~mil/mobroute, and initial release announcement at:
https://lists.sr.ht/~mil/mepo-devel/%3C9e35583d-c97c-4978-a06a-fd501cf548b4%40app.fastmail.com%3E

Since the initial release announcement referenced above, the major
features changed include: (1) SQLite integration via the new Mobsql
project and (2) Miscellaneous updates including JSON & GPX output formats,
reworking the logging system, and routing algorithm improvements. Details
below:

1. SQLite integration via Mobsql
All logic for downloading and caching GTFS archives from the Mobility
Database catalog has been migrated to the new Mobsql project (also 0.1
tagged, see http://git.sr.ht/~mil/mobsql). The purpose behind the split
between Mobsql and Mobroute is that from now on Mobroute will exclusively
be responsible for routing, stops, stoptimes, and auxiliary functionality
dependent on read-only access from a local SQLite database. Meanwhile all
Mobility Database GTFS fetching, import, caching and DB write-specific
access functionality will be handled by Mobsql. Accordingly Mobroute
no longer functions based handling of raw GTFS zip archives but rather
assumes GTFS data imported to a local SQLite database. Mobsql's purpose is
to serve as a general-purpose ETL & caching layer to import GTFS-to-SQL
featuring ingestion from links from the Mobility Database catalog and
can also be used independently of Mobroute.

The usage story in terms of end-user experience for Mobroute in this
change is fairly transparent as Mobsql is now used as a dependency (so
upon performing routing requests the relevant GTFS sources are fetched,
imported to a local SQLite database, via Mobsql; and then loaded from the
same SQLite database to memory in Mobroute's application logic). Rather
then specifying the `-gtfsregex` flag as previously required; a new
`-f` flag has been added to Mobroute which passes through to Mobsql's
filter functionality (see http://git.sr.ht/~mil/mobsql). This filter
functionality provides a broad and flexible mode for addressing one or
multiple GTFS sources.

While prior to this update, Mobroute used GTFS archives directly through
the go-gtfs package which necessitated loading the entire GTFS zip archive
to memory; with the use of Mobsql, specific select queries are used for
loading this data from tables.  Having the load process be indexable
via SQL queries opens the door for future optimizations (such as loading
only stop-times for the day the route being calculated, within a certain
timespan, and similar). Some broad loading optimizations have been
added in this release, however in future releases these optimizations
will be built upon to further refine and limit only loading the needed
data to memory allowing running Mobroute route queries with lower memory
requirements and more efficiently.

2. Misc Updates: JSON & GPX formats, Logging rework, algorithm improvements
Aside from the update to use an intermediate caching SQLite database
via Mobsql, there have been a few miscellaneous features added to
mention here.

Firstly, the default output format has been updated to be JSON rather
then ASCII (tables). This is generally simpler for debugging and reduces
code boilerplate between commands. The CSV output format has replaced the
former ASCII format which reduces complexity in output formatting. And
finally, a GPX format has been added for the route command.  One of
the driving factors behind adding support for the GPX format is to
allow testing on Android mobile map applications such as Organic Maps
by way of using Mobroute in its current state via Termux (see README
for guide to follow with Termux) to generate routes in GPX to view in
such applications.

Secondly, STDERR logging has been reworked to group all log messages
into three categories: info, warn, and debug. You can use the flags
-vi, -vw, and -vd for these categories respectively; or -vvv to enable
all log categories. Info messages are enabled by default and provides
helpful general-purpose logging information, warn messages provide
a more detailed view into the applications operation (not generally
helpful for end-users but helpful in debugging), and the debug category
is unused (in the codebase's application logic) but helpful for adding a
different category of messages (e.g. via logger.Debugf etc.) for one-off
developer-experience debugging.

Finally, Mobroute's routing algorithm, currently based on a time-dependent
approach to Dijkstra's algorithm has seen some further refinement. Lookup
tables have been extracted and generally the code is cleaner and built
in a more functional/testable way now. The core routing algorithm and
approaches to getting the most efficient routes is a work-in-progress
and will see further refinement with future updates.