~mil/mobroute

v0.9.0 20 days ago .tar.gz browse log

This release was focused largely on documentation updates
for the Go library & CLI. Additionally some minor performance
improvements & refactoring updates were made to cleanup the overall
codebase. Importantly, the Mobsql dependency was updated to v0.9.0 which
brings new parameters which can be used in Mobility Database GTFS data
ingest (via RTDatabase / or database CLI subcommand) to bring more
up-to-date GTFS data into the system. Note, the database schema has
changed and its recommended to clear cache between version updates (see
(5)). See below for a detailed list of changes for this release:

1. Library & CLI Documentation
The major update in this release is the rework of both the user-facing
CLI documentation and Go library documentation. For the Go library
side of things: all public types, structs, and functions in the top
level git.sr.ht/~mil/mobroute package now have proper docstrings. And
additionally example usages have been added to the `examples/` folder. For
the CLI, the userguide has been split from the CLI doc. Wherein the
former provides walkthrough style examples and the latter provides the
full documentation as you get from the `-h` flag for each subcommand on
the `mobroute` CLI.

Additionally, important to note: a new documentation website for the
larger Mobroute project is available and Mobroute's documentation is
published to this site at: https://mr.lrdu.org/mobroute

Go documentation is available at:
  - Go doc: https://pkg.go.dev/git.sr.ht/~mil/mobroute?tab=versions
  - Go examples: https://git.sr.ht/~mil/mobroute/tree/master/item/examples

And updated CLI documentation is available at:
  - CLI userguide: https://mr.lrdu.org/mobroute/doc/userguide_cli/
  - CLI doc: https://mr.lrdu.org/mobroute/doc/doc_cli/

2. Performance Improvement: Use OID instead of ConnID for Connections
Previously for referencing connections in the routing logic after CSA,
the generated 'ConnID' field was used. This caused issues when a large
number of connections (or origin GTFS stop_times) were present. Use of
ConnID has been replaced by using SQLite's builtin OID which points to
rows individually. There is a moderate performance improvement with
this update (which is particularly noticeable in the case of routing
with larger feeds with excessive sized stop_times).

3. Consistently refer to StopUID rather then Stop{IDU,SIDU,SSIDU}
The terminologies StopIDU, StopSIDU, and StopSSIDU were previously used
throughout the routing Go & SQL logic in order to reference stop unique
identifiers and their origin parent stations. This invented terminology
was confusing and rather now we simply use the new terminology StopUID
(short for stop unique identifier) which uniquely maps a stop ID and feed
ID combination. And in the case a parent stop is used, StopParentUID is
referenced. This cleanup also allows us to properly refer to 'StopUID'
in the RTSchedule function.

4. Removal of Oneshot Library Functionality
The library formerly offered two usage variants: Oneshot & Runtime
usage functionality. Wherein the former allowed for 'one-stage' runtime
initialization, load, compute, and {routing,stops,schedule} functionality
batched into a single function call. While seemingly convenient, this
added complexity as there are configurable params at each stage (for
runtime initialization, load, compute, and routing) and encapsulating
all these params into a single function just complicated maintenance. As
such, oneshot functionality has been completely removed and runtime /
stepwise functionality is the defacto recommended and only supported
usage pattern. See the updated Go docs and examples for more information
on library usage.

5. Update to Mobsql v0.9.0 & Passthrough for Update / Direct Agency URL Support
As per usual the minor version tagged with this Mobroute release (v0.9.0)
is correlated with the minor release for Mobsql (v0.9.0). The included
update to v0.9.0 on Mobsql brings an update to the `RTDatabase` function &
`database` CLI subcommand on Mobroute to allow two new params for the
`loadmdbgtfs` op. The `MDBGTFSUpdate` parameter allows for ignoring
caching for the Mobility Database CSV & GTFS data & enforces explicitly
updating the Mobility Database to ensure the latest GTFS data is pulled
per MDB. And the `MDBGTFSDirectURLs` parameter allows the user to specify
that *direct* agency URLs should be used in GTFS data ingest (wherein by
default cloud CI-generated bucket mirrored Mobility Database URLs are
used - which may occasionally get out of date). These two new params
allow end-users to more reliably get up-to-date GTFS data loaded.

5. Schema Updates
A few small schema updates have been applied for naming & code structure
consistency. It's recommended to clear your database & cache (rm
`~/.cache/mobroute`) before updating from v0.8.* to v0.9.0. Apologies
for the manual user-intervention; long-term I am considering investing
in a more standard SQL migration process.

v0.8.4 2 months ago .tar.gz browse log

Minor patch (v0.8.4) release published as a fixup to v0.8.3 which was
indaventerantly force overwritten.  This patch contains retract directive
for v0.8.3 which should not be used.

v0.8.3 2 months ago .tar.gz browse log

Minor patch release includes changes:
  - Allow glob searching feed id / stop id for RTStops
  - Add optimizations to routeparams input; implement feedsclean optimization
  - Handle case for NaN in dist haversine formatting calculation
  - Store optimized feeds array in output for diagnostics

v0.8.2 2 months ago .tar.gz browse log

Minor patch release to address a few small perf issues:
  - Make LoadStopToParent test ordering independent / sort in unit test
  - Add support for glob in RTStops
  - Added missing index for stops related to stopwalks query
  - Cap _vcaltoservice to static 2024-2028 year range
  - Add index for _ctcaltoservice on (feed_id, service_id, service_date)
  - Add nLimit param to RTStops
  - Bump mobsql to 0.8.3 (fixes indexes for stopwalks / agency)

v0.8.1 2 months ago .tar.gz browse log

Minor patch release with 2 small updates:
  - Bumps mobsql dep to v0.8.2 (which addresses HH:MM:SS / H:MM:SS bug)
  - Log connection path / stop details when CSA alg broken error occurs

v0.8.0 2 months ago .tar.gz browse log

This release of Mobroute implements some large scale performance
optimizations for computed tables (generated transfers, calendar to
services, and connections) in addition to adding support for customizable
logging. Load & compute speed is dramatically improved for adding new
GTFS feeds. The Mobsql dependency for this release is bumped to 0.8.1. See
below for a detailed list of changes:

1. Transfers Computed Table RTree Perf Optimization
The SQL view logic (_vtransfersgen) which is responsible for creating the
computed/generated transfers table has been wholesale reworked to optimize
performance. The view logic for computed transfers is essentially a
cross-comparison between all stops (per GTFS feed) to find stops with
a haversine distance with less then a 3km threshold to consider these
transfers valid for inter-stop transfers.

Calculating this view had previously been implemented using a naive
self join (using a rough distance 3 km heuristic) and then computing the
haversine distance between matched stops. This logic has been reworked
using SQLite's RTree extension (and utilizing the new init/deinit
computed table functionality in Mobsql). RTree allows for hierarchical
organization of the bounding boxes for stops and has much better
performance characteristics then the former self join implementation
logic. The resulting set of stops distances for transfers is identical
to the old logic as verified by unit tests; and the net performance gain
is conservatively a 2-3x speedup. Note speed scale was formerly quadratic
so you may see even greater performance improvements.

2. Calendar to Services Computed Table Perf Optimization
The SQL view logic (_vcaltoservice) responsible for creating the
calendar to services mapping computed table has been wholesale reworked
to optimize performance. This view allows mapping a single YYYYMMDD day
for a GTFS feed to its associated valid service ids for that day (which
is an integral dependency for routing connections extraction prior to
running the core routing algorithm).

This view had previously been implemented with a simple and naive
group by per feed id / service id / day which exhibited very bad
performance characteristics as the size of the GTFS calendar and
calendar_dates tables grew. The newly reworked logic uses plain joins
(no group by) and has vastly improved performance characteristics as
the size of the calendar & calendar_dates tables grow. Along with (1),
these 2 performance optimizations (of _vtransfersgen and _vcaltoservice)
contribute to a significantly decreased compute speed time for the grand
majority of feeds.

3. Generated Connection Computed Table Perf Optimization
While not as significant as the performance optimizations mentioned
for (1) and (2); a smaller performance optimization was applied to
the SQL view logic (_vconn) responsible for creating the connections
computed table. The update applied is that connections are no longer
ordered by departure time in the view itself (since this order *needs*
to be applied at the time of extraction / during a routing library call
in anycase since timezone logic is applied at this step and will vary
per service ID). A moderate increase in performance is gained by not
ordering by departure time in computed table calculation as such.

4. Ability to Set Customizable Logger in Runtime Initialization
In initializing a Mobroute runtime, the user may now specify a custom
logger (log.Logger) to override the default logger (which prints logs
to STDERR). This allows downstream consumers & applications to customize
where to send logs for load & routing calls and provides better debugging
transparency.

v0.7.0 4 months ago .tar.gz browse log

This release of Mobroute was focused on tooling updates to add new stops,
schedule, and database library functions, addition of custom GTFS loading
support, and removal of GTFS feed autoload / ingestion functionality
for runtime library functions. See below for detailed notes on changes:

1. Addition of RTStops Library Function
A new library function RTStops has been added which allows the enduser to
query stops for a given feed based on an input coordinate and maximum
kilometer threshold cutoff. If the input coordinate and maximum km
threshold are unspecified, this function can be used to dump a list of
all stops for the given feed IDs. Similar functionality was present in
early versions of Mobroute however was cut to facilitate large scale
refactoring. Adding this functionality back enables transparency into
the underlying GTFS data loaded for each feed; and further the stop IDs
returned may be directly used with the new relatedly added RTSchedule
library function (see next section).

See the new CLI subcommand `mobroute stops` for testing details; further
documentation for RTStops in library usage is in the works.

2. Addition of RTSchedule Library Function
A new library function RTSchedule has been added which allows the enduser
to query departure times for a given feed ID & stop. This allows the
user to get a quick overview of departures and increases the general
utility of the library as a whole. The schedule library function reuses
timetable logic from the routing algorithm / CSA and essentially the
schedule library function queries 'connections' in CSA's terminology
directly. Similar functionality was present in early versions of Mobroute
but was cut to facilitate large scale refactoring. Now that the CSA
schedule SQL logic has been stabilized, adding this functionality
in addition to the RTStops library function back should enable more
general-purpose usecases for the Mobroute library besides just routing.

See the new CLI subcommand `mobroute schedule` for testing details;
further documentation for RTStops in library usage is in the works.

3. Addition of RTDatabase & RTDatabaseq Library Functions
Two new library functions to interface with Mobsql (Mobroute's underlying
GTFS SQLite loading & ETL interface) have been added. The new RTDatabase
function allows for manipulating (loading, purging, updating) GTFS data
in the SQLite database and RTDatabaseq allows for querying said GTFS
data in the SQLite database respectively.

These functions are essentially a 'raw' passthrough to Mobsql
functionality but was important to add as (1) this way a 'single' database
connection can be used for both database load & routing/stops/schedule
calls and (2) by adding this passthrough, endusers/consumers of Mobroute's
library only have to interface with the Mobroute and not Mobsql directly.

See the new CLI subcommands `mobroute database` and `mobroute databaseq`
for testing details; further documentation for RTDatabase and RTDatabaseq
in library usage is in the works.

4. Custom GTFS Loading Support (via RTDatabase)
Custom GTFS archives load support has been added via the newly added
support in Mobsql. This functionality is exposed through the new
RTDatabase library function and  may be used with the 'loadcustomgtfs'
op allowing the user to pass a negative feed ID to be associated with
a user-specified GTFS zip archive (file:// or http:// path) & loaded to
the database and for routing, stops, and schedule functionality. See the
CLI documentation for the `mobroute database` subcommand for more details.

5. Library Usage Pattern Update: Runtime GTFS Autoload Removal
In prior releases runtime library usage using the RTRoute function and
similar would automatically 'load & compute' the necessary data for
the request *upon use*. In this release, the pattern for runtime library
functions (RTRoute, RTStops, RTSchedule, etc.) usage has been updated such
that GTFS archives are no longer 'automatically' loaded. Rather the new
runtime library pattern usage is now: 1) initialize the Mobroute Runtime,
2) use the new RTDatabase function to load the relevant GTFS data, 3)
use the RTDatabase function to compute the routing-optimized tables
from the GTFS data, and then 4) run the RTRoute or GTFS-dependent other
function with the loaded feed ID(s). You can see the Go documentation
for more details on each of these steps.

If you are just interested in one-off / oneshot library routing, stops,
database, or schedule requests (with automatic/implicit GTFS loading /
ingestion), you may still use the Oneshot* functions for all-in-one
functionality. E.g.  the Oneshot functions still invoke the old pattern
of 'automatic' GTFS ingestion and the CLI remains the same with this
pattern and serves as the defacto reference implementation for oneshot
library usage.

v0.6.0 6 months ago .tar.gz browse log

This release of Mobroute was focused on tooling updates in order to: make
the library & CLI more friendly to end users; and to aide in the initial
general public release (on F-Droid) for Transito, which is Mobroute's
associated mobile app and reference implementation for runtime library
usage. See below for a detailed list of changes.

1. Library: Public Interface Cleanup - Oneshot & Runtime Functions
The public interface for the Mobroute library has been wholesale reworked
with an eye toward external consumer usage. The largest change is that
the library's API now presents both 'oneshot' and 'runtime' function
variants. The defacto reference implementation for usage of Mobroute's
library 'oneshot' functions is Mobroute's CLI located in the source
`cli/` folder. And the defacto reference implementation for Mobroute's
library 'runtime' functions is the Transito mobile application which
pools a single Mobroute runtime (and database connection) for the entire
app usage.

The Runtime library functions (as utilized by Transito), allow a
*single database connection* & runtime to be persisted between multiple
library function calls. The user first creates a MobrouteRuntime (via
RTInitialize) and then the user is able to perform multiple routing
requests using the same runtime and database connection using the RTRoute
function. While RTRoute is currently the only runtime function available
apart from RTInitialize (to initialize the runtime); this pattern of
using a single runtime/database connection sets things architecturally
in a good place for future updates & planned functionality (such as
querying stoptimes, nearby stops, and similar). See the Transito app for
an example of Mobroute library's runtime functions usage. By convention
all runtime library functions are and will be prefixed with 'RT'.

Meanwhile, Oneshot library functions (as utilized by the Mobroute CLI),
offer functions callable in a single (or 'oneshot') request. Unlike the
runtime library functions, no prior setup is needed for oneshot functions
and this has the benefit that the entire request (for routing requests
and similar) is serializable as a single JSON object. The database
connection is opened at the start of the function call and closed upon
returning with oneshot functions. Do note that internally the new oneshot
library functions (OneshotRoute & OneshotMobsql) use the runtime API
under-the-hood and the oneshot API is largely a convenience for end-users
who want to fire off a routing request with a single call and is the
simpler option for integration for one-off use-cases. See the CLI for
an example of Mobroute library's' oneshot function usage. By convention
all oneshot library functions are and will be prefixed with 'Oneshot'.

2. Library: Initial Documentation
Initial documentation for the Mobroute library has been added in the form
of godoc strings and the overall presentation of the documentation has
been improved if you view within godoc, pkg.go.dev, and similar. More
extensive documentation is slated for a later release (see roadmap),
however for basic usage of the new runtime & oneshot library functions,
the initial documentation (and reference implementations of the Mobroute
CLI & Transito app) added in this release should suffice.

3. CLI: Functionality & Documentation
A number of large changes have been made to the Mobroute CLI. Among
these changes are: (1) the CLI has been reworked to use the newly
updated Oneshot Mobroute library functions, (2) the CLI documentation and
functionality for both the route and passthrough Mobsql functionality has
been reworked to be more friendly to endusers wrt. JSON object formatting
and docstrings in -h help messages (3) YAML functionality has been removed
and all requests are expected to be formatted as JSON, (4) Feed IDs are
used directly instead of the old 'filter format' specification which just
added complexity, and finally (5) error handling is vastly improved due
to both code restructuring & the addition of route request validation
(see next section).

4. Routing: Route Request Validations
Validations have been added for route requests by way of utilizing
struct tags via the github.com/go-playground/validator library which has
been added as a dependency. This library allows for validations such
as checking the passed transfer categories, output formats, feed IDs,
tunable walk parameter, and similar simply by struct tag annotation. Both
CLI & library calls for routing are validated upon usage. As a result,
the user can now expect much friendlier & more structured error messages
in the case of failed route request due to invalid route parameters.

5. Routing: Embedding of Route Request in GeoJSON Response
The GeoJSON output formatter has been modified to include the original
route request serialized as a JSON string embedded in the output
route LineString property 'metadata' field. This was done largely for
convenience & debugging in that the http://geojson.lrdu.org service which
provides links for the map view output format now automatically passes
through the 'original request'. This has the benefit that the map URLs
(which are easily shareable) are now much easier to debug & replicate in
that all that is needed to 'recreate' a map URL route is simply to run
the mobroute CLI with the JSON buffer included in the map link as in:
`mobroute route -rp '{}'` wherein `{}` is the JSON request.

6. Tooling: MDBID & Source References Renamed to FeedID
All references to 'MDBID' and 'source' in the codebase have been replaced
with the term 'FeedID'. The previously used MDBID and source terms were
used interchangeably and created ambiguity as to what the difference
between these two terms were. Replacing all references should make it
clearer both from the database & in the Go code what exactly is being
referred to. As well the term 'FeedID' should be more self-explanatory
to end users then either of the former terms. This update matches a
similar naming update for FeedID in Mobsql 0.6.

7. Tooling: Go SQLite Driver Portability / Simplification
Since initial versions of Mobroute, there has been a hard dependency on
the SQLite driver library mattn/go-sqlite3. A feature specific to this
library called 'Go SQlite3 Extensions' has been historically utilized to
call Go code directly from SQLite queries in the core of the routing
API. This provided a good amount of convenience in that haversine
distance, timezone lookup, and similar functionality didn't have to
be directly implemented in SQL. However, in this release the reliance
on all 'Go SQLite3 Extensions' functionality has been deliberately
removed. There are a number of reasons this was done; the primary
driving reason has to do with debugging. Without using these bindings,
all queries executed & printed to the console can be simply replicated
just by dumping the exact query in the sqlite CLI with the parameterized
args. This offers a much better developer UX for debugging for both code &
live debugging scenarios. Additionally, without relying on the Go/SQLite
interop extension pattern specific to this library, Mobroute's SQLite
driver could be very simply be switched out in the future for another
Go SQLite driver. This has benefit in both testing scenarios (e.g. in
the case of a bug with the mattn/go-sqlite3 library) and also opens the
door to potential for future optimization wrt. the choice of which Go
SQLite driver is used.

v0.5.0 8 months ago .tar.gz browse log

This release of Mobroute was focused on improving overall compatibility
of routing functionality to work with a larger number of GTFS feeds. Many
edgecase scenarios have been corrected; and given the correct provided
routing parameters & specification-conforming GTFS data, users can now
expect routing to work reliably in the vast majority of cases. In
addition, a large number of unit test cases have been added to
subcomponents of the routing system. Details on changes are below:

1. Transfer Categories System Rework & Performance Improvements
The way transfers are handled, particularly for implicit transfers and
generated transfers, has been vastly reworked.

- Three distinct transfer categories can now be toggled: feed,
  implicit, and generated transfers. These three categories can be
  used in any combination. The defaults are to use feed and implicit
  transfers. The user should manually enable generated transfers when
  desired (e.g. if a feed does not provide its own transfers or when
  using >1 feed).
- The implementation of implicit transfers (e.g. transfers from one
  stop to the same stop) now applies at the SQL loader level (rather
  then at the algorithm level) and the concept of 'transfer categories'
  is completely abstracted from the routing / CSA algorithm itself.
- Performance-related: a heuristic is used to avoid a full selfjoin
  in determining potential computed/generated transfers. Additionally
  the size of the generated transfers table was decreased by avoiding
  duplicate entries. Both of these changes vastly decreases the amount
  of time to create the generated transfers table & improves overall
  transfers performance.

2. Parent Stop Lookup Accuracy & Performance Improvements
The logic which determines looking up parent stops has been reworked
entirely and now utilizes the GTFS location_type field.

- The GTFS location_type field is properly respected and each case
  (for root stations, exit/entry/nodes, platforms, and boarding areas)
  is handled separately in implementation logic. The new logic is much
  more accurate then the previous logic which contained a number of bugs
  effecting GTFS feeds such as the Prague feed (MDBID 767).
- The recursive CTE logic previously utilizing the parent_station field
  was removed, as location_type field lookup is all that is needed for
  accurate lookup.
- Performance is vastly improved for parent stop lookups since without
  the recursive CTE and with utilizing location_type, lookup is a 'flat'
  operation. Additionally parent station lookup no longer utilizes a
  computed table (_cvstopparent) but rather utilizes the direct SQL view
  (_vstopparent) in downstream logic (since it's efficient enough to do
  so); and this thus also entirely removes a good amount of wall time
  in the initial compute cycle prior to routing.

3. Multiday Route Support
Multiple day route support has been added such that connections that are
obtained from multiple different adjacent schedule days can be used in
combination for a single calculated target route.

- Beginning a route at or near midnight in the target locale now can be
  expected to work and the system will pull multiple connections from
  different schedule days (interleaving properly) into a single unified
  schedule from which to determine the target route.
- Previously, routes beginning around midnight would defacto fail
  (depending on each feed's availability of timetable data exceeding the
  >24hr boundary). Now instead, the routing system works in all cases
  where routing would previously fail in this usecase.
- Returned connections are formatted in RFC3339 format to allow seeing
  which day each leg component time is from (in addition to timezone).

4. Timezone Handling Support
Timezone handling logic has been implemented per GTFS specification and
each feed's timezone is respected (independently of the routing request's
input timezone).

- Connections extraction logic (in the connections loader) handles
  normalizing all connections times to UTC times (independent of timezone)
  prior to ever being entered into the CSA algorithm.
- Routes are formatted with times in RFC3339 format normalized
  to whichever timezone or UTC delta the input routing request used.
- Multiple feeds with agency providers using different timezones may
  be utilized in a single routing request.

5. Unit Tests: SQL Views Tests, SQL Loaders Tests, and CI
The overall testsuite particularly for the SQL-based components of the
system (e.g. everything before hitting the CSA algorithm) has been vastly
improved.

- Unit tests have been introduced for transfers, stopwalks, and
  connections functionality. SQL 'loader' tests handle extensively testing
  edgecases for each of these components. Unit tests for prior mentioned
  timezone handling, multiday logic, transfers logic, and parent stop
  lookup logic is implemented. See the `dbquery_test` package.
- In addition to SQL 'loader' tests, more componentized SQL view tests are
  implemented to test the behavior of SQL views used as subcomponents
  of the SQL loaders (e.g. _vstopparent, _vcaltoservice). See the
  `dbstubquery_test` package.
- The GTFS-based routing integration tests (within `apiroute_test`)
  that generate results at https://ci.lrdu.org/tests_results/
  have been updated to additionally validate route duration &
  number of transfers to make it simpler to debug routing system
  updates. Additionally, results for each commit are now published to
  https://ci.lrdu.org/tests_results_refs/
- The CI that is responsible for generating test results has been
  updated to send emails with build results to the new
  ~mil/mobroute-ci@lists.sr.ht mailing list.

v0.4.0 11 months ago .tar.gz browse log

This release of Mobroute was focused around usability updates to make
the project more approachable to end-users; and additionally added some
infrastructure to improve the testsuite. Details on changes below:

1. Output Formatters Rework; And addition of GeoJSON & Map URL Formatters
A new field "output_formats" has been added to the request structure
for route requests. This field enables the user to customize the format
of the output returned the calculated routes. Previously the "legs" and
"diagnostics" formats were always shown by default. Now it is customizable
if you wish to suppress either of these fields. Additionally, several new
formatters have been added. Importantly, a new GeoJSON formatter allows
exporting the route rendered in GeoJSON format which is capable of being
viewed in many external applications. And additionally, the new mapurl
formatter provides a URL based on the http://geojson.lrdu.org service
which renders arbitrary GeoJSON in the browser.

The full list of supported output formatters (and description of each)
now is as follows:
- legs: Shows the route "legs" in format as it was previously
- diagnostics: Merges the previous 'diagnostic_perf' & 'diagnostics_route'
- request: Shows the raw request used for the route
- geojson: Show the route as GeoJSON points (with steps information merged)
- mapurl: Shows the route as a URL on geojson.lrdu.org domain with GeoJSON
          passed in hash

2. Migration of Views / Computed Tables to Mobroute Proper
The actual (SQL) views & computed tables  used in Mobroute's routing
logic, previously housed in Mobsql, has been migrated to Mobroute's
codebase itself. This change was enabled by an API update in Mobsql's
library that allows end users / consumers of Mobsql's API to define
arbitrary SQL views & computed tables. This update makes the overall
codebases of Mobroute and Mobsql alot cleaner in that now Mobsql is purely
a GTFS-to-SQLite ETL tool and all Mobroute application-specific-logic
lives in Mobroute's repo and is parameterized in the call to initializing
Mobsql's runtime. See the `db/dbschemaextra` folder for more details.

3. Calendar Service Mapping View Fix
Historical routes (e.g. routes calculated for dates in the past) are now
properly calculated. Prior to changes in this cycle, if you attempted
to calculate a historical route (e.g. for any time in the past) the
route request would fail as the underlying _vcaltoservice view only
looked at dates in the present and 30 days in the future. This view
has been corrected to properly respect both the underlying calendar.txt
and calendar_dates.txt GTFS tables and as such now routes are properly
calculated for dates in the past & >30 days in the future. This was
an important change to make as this enables tests to properly function
independent of 'current' GTFS data.

4. Routing API Testing: GTFS Testing Architecture
Testing architecture for the routing API itself has been added in this
release.  Prior, the main testsuites in Mobroute revolved around testing
the CSA algorithm (which is abstracted from the routing API - the latter
being a higher level interface). The whole routing API covers quite a
bit: from Mobility Database CSV ingestion, to GTFS import via Mobsql,
to SQL view extractions, to running the CSA algorithm, and to finally
formatting the response. With this release cycle, the whole API for
routing is tested as an integrated unit. See the new `apiroute_test`
package for details. Also 'realistic' GTFS tests (sourced from actual
Mobility Database GTFS feeds snapshotted) were added in this package. The
'realistic' GTFS tests run in CI and the results are outputted in HTML
format and uploaded to:

http://ci.lrdu.org/tests_results/

5. Documentation Rework
Documentation for Mobroute has been reworked as a whole to be more
approachable to end-users. This involved splitting prior documentation
into multiple pages and a rework of the README and project description
overall. The documentation overall also matches the style & fits
cohesively with the related Transito project.
1 / 2

Branches

remove-oneshot-lib
schedules-optimization
_vcaltoservice-optimizations
_cvtransfersgen-rtree
_cvtransfersgen-rtable
sqlite-extension-csv
doc-roadmap-revision-08032024
api-refactor-oneshot-wip
api-refactor-snapshot
cvtransfersgen-parent-optimization
csa-neg-starttime-support
timezone-handling-wip
multiday-connections
agency-id-formatting
simplify-csa-implicit-transfers
bucketing-poc-rebase
transfers-loader-tests
csa-transfers-logic-tests
simplify-transfers-logic
connections-sql-perf
formatters-refactor
stopwalks-loader-tests
stops-stoptimes-api
dbloader-tests-improvements
2024-roadmap-revision-m5-m6-m7
refactor-remove-db-connection-struct
realistictests-rework
roadmap-2024-rework
formatters-refactoring
stop-details-formatting
basic-ntransfers-limiter
csa-transfers-refactor-details-struct
max-transfer-seconds-rework