path: root/notes/versioning.txt

Versioning of symbols of modules
================================

Versioning optimization
-----------------------

Currently, versioning is done per identifier:

 \verdef 1.0
 \verdef 1.1
 \verdef 1.2
 \verdef 2.0
 \verdef 1.2.1

 since 1.0
 func f()
 since 1.2
 func f(int x)
 since 1.1
 func g(int x) -> int
 since 1.0
 func h() -> int

The versions are duplicated
- First, they appear in the \verdef
- Second, they appear in the "since ..." lines
Could it be done more efficiently?


Versions are usually not longer than a pointer, so maybe not?
Also, there will usually only be one version per symbol.

Solution 1: Version index directly as lookup key (won't work?)
--------------------------------------------------------------

On the other hand, it could make sense to include a "version index"
field in the version, that can be compared correctly in an easy way:

 \verdef 1.0            # index 0
 \verdef 1.1            # index 1
 \verdef 1.2            # index 2
 \verdef 2.0            # index 3
 \verdef 1.2.1          # index 4

Parsing with "version indices":
- parse \verdef and build tree
- when parsing since... definitions, look up in \verdef tree

Solution 2: Version index in wrapper
------------------------------------

 versions_root -> tree of
    - version wrapper with:
        - version index
        - version status (normal, deleted, deprecated, ...?)
        - identdecl

Maybe something like this:

 #define VDT_NORMAL      0
 #define VDT_DELETED     1
 #define VDT_DEPRECATED  2
 #define VDT_BACKPORTED  3

 /**
  * Header for versioned declarations. Fields are only valid for declared
  * identifiers (must be exported), not for referenced external identifiers
  * (if we even should allow versioned external references...)
  */
 struct VersionedDeclHeader {
    unsigned int vertype : 3;
    unsigned int verindex : 29;
 };
 
 union VersionedDecl {
    struct VersionedDeclHeader hdr;
    struct {
        struct VersionedDeclHeader hdr;
        struct IdentDecl decl;
    } normal;
    struct {
        struct VersionedDeclHeader hdr;
        struct VersionedDecl *from_ver;
    } backport;
 };

After all \verdefs are parsed, then we can build a lookup table
from index to the version definition itself.

 unsigned int num_versions;
 struct VerDef **versions;

Non-sequential versions (e.g. maintenance branches)
---------------------------------------------------
Problem:

    0       1       2       3       4       5
    1.0     1.1     1.2     2.0     1.2.1   2.1

    4 >/= 3 (because it does not satisfy the 2.0 API)
    3 >/= 4 (because 4 (1.2.1) might contain improvements to the 1.x API)

Definitions of versions:
("a" and "b" are indices)

   version(b) > version(a)    <=>   b>a AND (ver_str(b) > ver_str(a))
   version(b) < version(a)    <=>   b<a AND (ver_str(b) < ver_str(a))
   version(b) == version(a)   <=>   b==a    <=>   ver_str(b) == ver_str(a)
   version(b) </> version(a)  for non-sequential versions

Use "previous indices":


index:      0       1       2       3       4       5       6       7
ver_str:    1.0     1.1     1.2     2.0     1.2.1   2.1     2.2     1.2.2
prev:       -       0       1       2       2       3       5       4

Version definitions
-------------------
Version definitions need at least this info:

 struct VerDef {
     struct TreeNode version;
     unsigned int verindex : 29;
     
 };


Avoiding too many versions
--------------------------
There should only be a \verdef when the API changes (exception: major
versions). This could be enforced.

It would be nice to be able to hide *fully* deleted versions.
Perhaps we should have a "\verdef ... clean" to not inherit old stuff
(including version history) from older versions.
That way, older definitions (AND old \verdefs) can be removed in the
following major version.

For example:
 # in version 0.x:
 \verdef 0.1 abc3242...
 \verdef 0.2 3df2424...
 # in version 1.x:
 \verdef 0.1 abc3242...
 \verdef 0.2 3df2424...
 \verdef 1.0 clean d327983...
 \verdef 1.1 5233532...
 # in version 2.x:
 # 0.x \verdef deleted!
 \verdef 1.0 clean d327983...
 \verdef 1.1 5233532...
 \verdef 2.0 clean 82fe819...

Old version could perhaps be provided through a wrapper/emulation library:

 # in version 2.x:
 \verdef 1.0 clean d327983...
 \verdef 1.1 5233532...
 \verdef 2.0 clean 82fe819...
 # in emulation library for version 0.x
 # (this should use the test suite for 0.x)
 \module somelib
 \type emulation
 \depends somelib 1.0
 \verdef 0.1 abc3242...
 \verdef 0.2 3df2424...
 
What should the maximum number of versions be? Perhaps 10 000?

SLUL ABI and emulation
----------------------
The ABI of SLUL itself might need to change at some point. In fact, we may
want to have multiple ABIs from the start (C compatible and "improved" ABI).

This could be done by adding an extra string of letters just before the
version.
On DLL and other non-symver platforms:
    somefunc__a_1_0_0    <-- ABI A
    somefunc__b_1_0_0    <-- ABI B
On SO-with-symver platforms:
    somefunc@SOMELIB__A_1_0_0   <-- ABI A
    somefunc@SOMELIB__B_1_0_0   <-- ABI B

Some ABIs may be able to co-exist in the same library (through wrapper calls
for the older one). With other ABI combations, that may not be possible (for
example, if the layout of data structures changes).

SLUL ABI changes should be invisible to, and not controlled by, the source
code being compiled. It should be a compiler option.


Keyword for version definitions
-------------------------------

 \api_ver (best?)
 \verdef
 \interface
 \interface_ver
 \interface_version

Adding versioning to symbols
----------------------------
It should be allowed to make unversioned symbols versioned.
In order to be API-safe, we need to require that:

 1. the added version is the oldest version
 2. the oldest version is not a "clean" version

=> The API hashes need to suffix the oldest version to any unversioned
   symbols, but only if the oldest version is not a "clean" version.

Undoing mistakes in APIs / Rolling back a version
-------------------------------------------------
Normally, SLUL does not allow symbols to be removed. However, it would
be nice to be able to completely undo changes in case of errors that are
detected quickly after release and/or errors that MUST be "removed from
history" (e.g. accidental inclusion of personal data or private tokens,
or copyright violations).

This could be done by marking a version as deleted, and not basing later
versions upon it.

For example:
    \slul 0.0.0
    ...
    \api_def 0.1.0 TWlzc2luZyBvbmUgc2luZ2xlIGJ5dGUx
    \api_def 1.0.0 deleted
    \api_def 1.0.1 VGVzdAABAgMwMTIzNDU2Nzg5YWJjZGVm

Only the last version/versions can be deleted, because a version cannot
be based on a deleted version.

Tightening constraints in versions
----------------------------------

Input constraints:
- These cannot be made stricter, but warnings could be added.
Output constraints:
- These could be made stricter (but must be versioned still)

Example syntax:

    since 1.0
    func do_stuff(int x) -> int
        cond x >= 0
        bestpractice x >= 1 since 1.1
        cond return < 100
        cond return < 90 since 1.2

Major versions in names?
------------------------

For modules that break the API in major versions (e.g. that strictly follow
semantic versioning), it could be useful to allow the major version to go
directly into the name somehow.

Basic solution with merging name and version:

    # in the module itself:
    \name something1
    \version 1.0.1

    # in dependencies:
    \depends something1.0.1

The downside is that it is easy to mistype. And it could also be confusing.

Another way could be to search for both "something" and "something1" when
searching for dependencies, and then check whether that module uses semantic
versioning (this could perhaps be indicated by an additional keyword after
the version, e.g. "\version 1.0.1 semver").
(And for semvar at 0.x, the compiler could implicitly add "unstable" as well).

Alpha/beta/RC versions
----------------------

Currently, SLUL uses the tilde character for this, e.g. "1.0~rc1" syntax.
That is also what Debian does.

But SemVer has added support for pre-releases as well, and there the
dash character is used for this purpose. (Even SemVer 1.0 has this).