Release 0.10.0
NEWS
With 2621 commits (and almost three years of development) since 0.9.0
this release is not only the biggest (103% more commits) but the longest
in terms of calendar time so far.
Read about the changes in the Changelog file.
Release 0.9.0
NEWS
With 1287 commits since 0.8.4 this release is the biggest one in the
history of Viua so far.
And not only in terms of commit count, but also functionality.
CLOSURES
Closures were reworkerd.
The `enclose*` family of instructions was renamed to `capture*`, and
closures got their own assembly directive - `.closure:`.
Functions declared as closures are not directly callable and
must be instantiated first using `closure` instruction.
CONCURRENCY
Concurrency-related aspects of the VM also got some love.
Blocking operations (`join` and `receive`) can now specify a timeout for
how long they should block a process.
The timeout may be specified in seconds, milliseconds, or
as the `infinity` token.
DEFAULT VALUES
Some instructions now can be written in a shortened form and
the assembler will inset default values for omitted tokens.
These include for example `join`, `receive`, `istore`, and `fstore`.
DEFAULT COMPILE-TIME KEYWORD
The new compile-time `default` keyword can be used wherever it is legal
to omit a token.
The assembler will change the `default` keyword into the default value
for that place.
It is useful when usage of the default value should be stated explicitly.
IOTA COMPILE-TIME KEYWORD
The new compile-time `iota` keyword generated an ever-increasing integer
starting from 1.
It can be used to automatically assign register indexes:
.name: %iota answer
istore %answer local 42
print %answer local
It is especially useful when code is changed, as the assembler will
reindex the registers automatically and free the programmer from this
task.
WATCHDOG PROCESSES CANNIBALISE TIME OF CRASHED PROCESSES
Previously, when a process crashed, the VM spawned a watchdog and
run it to completion.
This is now fixed, and crashed processes become their own watchdogs on
failure - so watchdogs are now on the same level as all other processes,
and may be preempted to prevent starvation of "normal" processes.
Before the crashed process becomes a watchdog its stack is unwound.
VOID TARGET REGISTER
The new `void` keyword can be used as a target register.
Using void register as the target register will drop the value that
would by normally produced by the instruction.
Some examples:
; drop the result of function call (if any)
call void foo/0
; delete the value
move void %1 local
EXPLICIT REGISTER SET SPECIFIERS
The `tmpri` and `tmpro` instructions are no longer needed as the VM
supports explicit register set specifiers for register operands.
This means that to move a value from local register set to the static
register set the following instruction can be used:
; move <target> <source>
move %1 static %1 local
...instead of this sequence:
ress local
tmpri %1
ress static
tmpro %1
Explicit register set specifiers make code shorted and more efficient.
ATOMS
Atoms are unique values whose only property is that they may be checked
for equality.
Useful as tags.
Supported in Viua by `atom` and `atomeq` instructions.
TEXT AND UTF-8
Starting with this release, Viua uses UTF-8 as its internal character
set for text values.
What is more, a special text type was added to the VM's list of
primitive types.
Text values must always be valid UTF-8.
A `text*` family of instructions was introduced to distinguish text
values from a "string of bytes" values produced by `strstore`
instruction.
As an additional feature, all values can be casted to text using the
`text` instruction:
istore %1 local 42
text %2 local %1 local ; register 2 will contain text "42"
DEFERRED CALLS
A very useful feature.
Deferred calls may be registered to be called when the frame they were
registered in is popped off the stack (during unwinding, normal returns,
or tail calls).
They are useful as a debugging aid, and can be used to implement
resource management schemes.
More information about deferred calls can be found on the
weekly.viuavm.org blog.
ADDITIONAL NOTES
This release also introduces one very useful and important feature:
processes can now contain many stacks, and switch execution between
them.
Such an additionl will make implementation of several new
functionalities much easier and more intuitive.
For example:
- interrupts: they must be run in the context of a specified process,
but are in no relation to what the process is currently executing so
they are a prime candidate to be run on a different stack
- message filtering: a function could be provided to filter messages
available in a process' mailbox, and as this function's execution is
is only a tool to achieve some other goal, it would be a good idea to
run the function on another stack
- stackful coroutines: coroutines that can yield from frames at
arbitrary depth of the stack, not only from the top-most frame
and possibly more.
A style guide has been introduced with the help of the `clang-format`
tool.
VM got a basic static analyser that is able to catch errors at compile
time.
Release 0.8.4
NEWS
With 75 commits since 0.8.3 this relase brings few, but important improvements to
the core of the virtual machine.
SMP
Viua now support simultaneous multiprocessing for virtual processes.
The maximum number of virtual process that can be run in parallel depends on the settings the machine
is started with, and is limited by capabilities of the underlying hardware (for example - the VM will
not run 4 processes in parallel if only two CPU cores are available, assuming each core can run only
one thread).
INSTANT DETACHING
Processes can be spawned as immediately detached, by using 0 as the target register index.
The VM will interpret this as "spawn this process and execute it, and I am not interested in
communicating with it".
The VM will give parent process neither the possibility of joining spawned process, nor will it
return a PID for the process.
LAUNCH-TIME CONFIGURABLE SCHEDULER NUMBERS
Viua can be instructed to spawn a certain number of VP and FFI schedulers at launch-time, the limits
are no longer hard-set at compile time.
The number of spawned schedulers can not be changed at runtime.
Two environment variables control the number of schedulers spawned:
- VIUA_VP_SCHEDULERS: for number of VP schedulers
- VIUA_FFI_SCHEDULERS: for number of FFI schedulers
The CPU frontend provides the `--info` option; among other things, it provides information about
scheduler numbers the VM would spawn in current evironment.
Use `--json` option to get `--info` output in JSON format.
Release 0.8.3
NEWS
With 141 commits since 0.8.2 this release is mostly a gradual
Release 0.8.3 is mostly a gradual improvement over 0.8.2, the 141 commits pushed
to the repository introduce some new but not revolutionary features, and
a bunch of minor fixes.
The big things release brings are vector packing, multiple FFI schedulers, and
possibility of embedding metadata in Viua VM bytecode files.
NEW COMMENT SYNTAX
The assembler supports Lua-style comments beginning with `--` and
running till the end of line.
Semicolon comments are not deprecated and can still be used.
EXTERNAL FUNCTION AND BLOCK SIGNATURES INCLUDED IN BYTECODE DISASSEMBLY
Before 0.8.3 the disassembler did not emit signatures for functions and
blocks linked from external to the analysed translation units.
This proved to be extremely inconvenient and was fixed in this release.
Viua VM bytecode files now be freely disassembled and reassembled without
worrying about missing signatures during reassembling.
EMBEDDING METADATA IN BYTECODE FILES
Starting with this release Viua VM bytecode format supports embedding metadata in
bytecode files by employing `.info:` assembler directive.
Example line:
.info: license "GNU GPL v3"
Metadata is stored in a simple map where keys are restricted ASCII identifiers, and
values are strings.
String is currently the only embeddable value type.
COMPILE-TIME JUMP TARGET VERIFICATION
Jump targets are now verified at compile-time and assembler refuses to generate bytecode when
it detects jump errors in input source code.
Some absolute jumps cannot currently be verified by assembler and their verification is
perfoemed at runtime (e.g. absolute forward jumps).
COMPILE-TIME BLOCK CHECKING
Assembler checks if signatures for blocks used in a translation unit are present, and
refuses to compile code when input source code contains references to undefined blocks.
VECTOR PACKING
`vec` instruction can now be used to pack objects during vector creation.
Until this release vectors had to be created by spawning a vector object, and
then pushing objects one by one to the newly created container.
vec 1
vpush 1 (istore 2 40)
vpush 1 (istore 2 41)
vpush 1 (istore 2 42)
This proved to be cumbersome, so `vec` instruction was augmented to support packing.
Now, vec instruction receives three operands instead of one (the second and
the third operands are optional, and default to zero).
istore 2 40
istore 3 41
istore 4 42
vec 1 2 3
Second operand to the vec instruction is the index of the first register to be packed,
the third is the number of registers to be packed.
Above code creates a vector in register 1, and packs three objects starting from register 2.
Packed objects are moved inside the vector, leaving input registers empty.
By placing objects into registers carefully vector packing can be used to speed up
multiple-value returns from functions.
REMOVE RACE CONDITION WHEN REGISTERING EXCEPTIONS THROWN BY FFI CALLS
This release removes a race condition from the code that could cause the machine to lose
exceptions that were thrown by FFI calls.
The race condition was triggering a bug when an exception was registered in a process by the
FFI scheduler while VP scheduler was inspecting the process's state after executing it.
If the exception was registered between checks for terminated-status, and stopped-status the
exception has been quietly dropped and the process removed from the pool;
the precise order of events that triggered the bug was:
- exception has been thrown in FFI call, but the FFI scheduler did not yet register it in the
process that requested the call
- VP scheduler did not mark the process as terminated (and so the exception handling routines
have not been invoked)
- FFI scheduler registered an exception (which pushed the process into a terminated-stopped
state)
- VP scheduler checked if the process stopped and removed it from the list of running
processes
MULTIPLE FFI SCHEDULERS
As of 0.8.3 Viua VM is now able to spawn and utilise multiple FFI schedulers (their number
configurable at compile time, by default the VM spawns two).
This means that several FFI calls can be serviced in parallel, so the machine blocks less and
is able to execute FFI-intensive programs faster.
By default Viua spawns three C++ threads now, which causes overscheduling on systems with single
or dual-core CPUs.
VM configuration can be adjusted if this is undesirable.
Release 0.8.2
NEWS
With 171 commits since 0.8.1, this release brings improvements
to machine's core code, and some long overdue fixes.
The changes in this utterly uninteresting release are mostly internal and
will not (*should* not) affect user code much.
EXTRACTED VIRTUAL PROCESS SCHEDULER OUT OF CPU
Virtual process scheduler was extracted out of the CPU code.
This means that the CPU is now only the source of static information about
running program - what modules are loaded, where a function begins, what
methods a class responds to, etc.
It also holds the queue of FFI call requests (it has not changed since the last
release).
The *dynamic*, process-related information that are prone to change during lifetime
of a program (e.g. frames on call stack of process, exceptions mid-flight, messeges
in transfer) are now handled by "virtual process scheduler", which is a special class
concerned only with managing VM processes.
For now, the VM spawns only one VPS (virtual process scheduler) but in future releases
it will be changed, and machine will spawn a number of VPSs - each in its own host
thread.
Introduction of multiple active VPSs will mark the move from just concurrency, to
true parallelism of virtual processes inside the machine (i.e. they will not just
run one-after-another round-robin style, but some of them might actually be running
at *the same* time, only under different scheduler).
FEATURE OF THE DAY: std::unique_ptr<>
Machine now employs `std::unique_ptr<>` in some places in code to manage
lifetime of dynamically allocated objects.
The `std::unique_ptr<>` is now heavily used only in the `VirtualProcessScheduler`
code to manage stack frames, exceptions, and global and static register sets, but
in later releases it will gradualy replace naked pointers in other parts of the VM.
LESS CRASHING, MORE STACK TRACES
In previous releaes, when one virtual process crashed, and the crash was not handled
by the watchdog process, it brought down the whole VM, and produced a stack trace.
This behaviour is now altered: when a process crashes and there is no watchdog to
catch the escaped exception the VM will print a stack trace for the crashed process
but will continue running.
This change is intended to enhance reliability, errors will always happend, they just
should be isolated from the "healthy" parts of the system and serviced, instead of
bringing the whole system down with them.
What if the `main/` function crashes?
If there is a watchdog process to handle the failure, the exception will be serviced and
the VM will continue execution, to finish with exit code 0.
If there is no watchdog process, machine will continue running and
when all other processes finally terminate it will close with exit code 1.
WATCHDOG PROCESS SPAWNED PER-VPS, NOT PER VM
Each VPS manages its own watchdog; thus, virtual-process-crashing errors are localised
to the next nearest point after the process, instead of being propagated further (to
the CPU) and clogging up the metaphorical pipes.
Currently, there is only one active VPS so this move does not change much, but this
will pay off when machine starts spawning multiple schedulers.
FUTURE
According to release schedule, next release should introduce multiple FFI schedulers.
This will provide speed improvements due to greater parallelism on machines with more
cores to utilise (as each scheduler will run in parallel on its own thread).
Machines with one and two cores will not notice any change; and there are no plans to
"overschedule", i.e. to spawn more threads than there are real cores available to
utilise (if the hardware provides only two cores, only one FFI scheduler will be spawned).
Release 0.8.1
NEWS
With 238 commits since 0.8.0, this release brings in another set of improvements
the code of the machine; stronger compile-time checks, improved entry function
generation in assembler, different flavours of main function and
fixed FFI scheduling.
STRONGER COMPILE-TIME CHECKS
Assembler is now able to better verify function calls; a whole new verification
stage was introduced into the assembler's code which is able to detect arity
mismatches between function calls and declarations, frames with gaps (i.e. when
not all parameter slots in the frame are filled), and double-passing parameters.
Also, all errors are now enabled by default - this change was suggested by Harald
Eilertsen (https://github.com/snake66), who also provided valuable insight on the
topic of multithreading in C++, which is highly relevant to another part of this
release.
FIXED FFI SCHEDULING
Viua VM spawns two threads when it launches: one runs virtual processes of the
machine, and the other runs functions executed directly on the host CPU (the
foreign functions).
Release 0.8.0 shipped with a race-condition which sometimes could make the
machine hang after the FFI call was scheduled.
IMPROVED EXPIRED POINTER HANDLING
When an expired pointer is encountered, its type may be safely inspected and
its string representation may be safely generated.
In previous releases this would throw an exception.
This is particularly important when an expired pointer is present in stack
trace after a virtual process crash as it may be printed without any
problems.
FATAL VM EXCEPTION WHEN WATCHDOG PROCESS EXITS
Watchdog process cannot finish execution normally.
If the machine detects it did, it throws a fatal exception and
shuts itself down.
IMPROVED SAMPLE CODE
Sample code must have been updated for current release due to stricter
compiler checks.
As a bonus, old non-testing code has been cleaned up and
rewritten in up-to-date VM assembly.
FUTURE
Next release will be mostly a refactoring one, so should arrive in less than
a month if everything goes smooth and as planned (fingers crossed).
It will bring a new virtual process scheduler, extracted from the main CPU
code; the CPU will become more static - only providing information about
entry points locations, type hierarchy status and routing calls between
virtual and FFI schedulers.
Release 0.8.0
NEWS
This is another release introducing substantial changes to machine's code and
capabilities.
CLOSURES
Viua 0.8.0 sports much better support for closures than previous releases by
giving programmers way to choose which parts of the environment (which values) are
captured and how they are captured (by copy, by move, or by reference).
The cost for this feature is lost backwards compatibility.
VIRTUAL PROCESSES
Threads are renamed to processes.
A unit of execution in Viua VM is a "virtual process".
Each virtual process is separated from every other one.
Processes communicate only by means of message packages.
That does not sound very much like a definition of a "thread".
INTER-FUNCTION TAILCALLS
Viua 0.8.0 supports inter-function tailcalls.
Looping can be now be elegantly implemented using recursion instead of
assembler-like `branch` and `jump` instructions.
IMMEDIATE COPY ON PASS-BY-COPY
Starting with this version, machine immediately copies parameters passed by copy.
FOREIGN CALL OFFLOADING
Before this version FFI calls were blocking whole machine, i.e. if one process called
a foreign function, no other process could run until foreign function returned.
This was caused by the fact that machine was single-threaded.
Viua 0.8.0 spawns two threads: one for running native Viua code, and a second one
for running foreign code.
The process that called a foreign function is suspended until the foreign function returns
but all other VM processes are not.
STANDARD LIBRARY IMPROVEMENTS
Some additions and improvements were made to standard library modules `std::misc`,
`std::vector` and `std::functional`.
Release 0.7.0
NEWS
This release introduces some substantial changes to the machine, and
that is the reason for jumping from 0.6.1 straight to 0.7.0.
Some backwards incompatible changes:
- `throw` no longer leaves thrown objects in their registers,
thrown objects are moved out of their stack frame,
- `free` instruction was renamed to `delete`,
Some more backwards incompatible changes, this time due to introduction
of pointers.
Using pointers if cheaper than using references (by a large margin) so a few
standard library functions were rewritten to accept pointers where they used to
accept references.
Pointers are cheaper because they do not escape machine's basic memory management
model.
Then, a fix.
If a detached thread generates uncaught exception the machine will print out
trace of the correct stack.
Previously always the stack of `main()` function was printed.
One feature implemented in this release is a "pass-by-move", a very efficient method
of passing parameters to functions.
Last, but not least, come the enhancements in multithreading.
First, return values can be extracted from threads when they are joined.
Second, threads can be suspended and woken up; this is important because the CPU can
easily skip suspended threads without looking at their state (other than the `suspended`
status).
Last feature introduced by the 0.7.0 is also one of the more important ones.
Machine provides a way for the programmers to specify a function to be run as
a "watchdog thread".
If a thread dies when a watchdog is active the machine passes a message to the watchdog
telling it what thread died, and why.
This enables the watchdog to restart the deceased thread if necessary.
If the watchdog itself dies it is restarted automatically by the machine.
In my opinion, this is a great win for the reliability of software written to run on the
machine.
Instead of worrying about every edge case the programmer can let the software crash,
log the error, restart the part of the program that failed, and carry on like if nothing
at all has happened.
As always, there are also numerours small improvements and bug fixes.