path: root/notes/interface_files_deps.txt

Interface dependencies in interface files?
==========================================

TODO: Clean this up.

Should dependencies have to go into the interface files?

Yes: In order to keep interface files as single files, they would have
to go into the interface files (e.g. as `use` lines or something similar).

But that leads to another problem: Isn't it desireable for consistency to have
the same method to include files for both the interface and the implementation?
Also, it important that users don't confuse interface dependencies with
implementation dependencies.

Can these two problems be solved at the same time?

The best solution so far appears to be this syntax, i.e. API-hashes in
a separate lockfile and versions as blocks that can contain a comment
about how the API has changed in that version.

    ```
    version 0.1.0
    # Initial version
        use_types some_module 1.0
    end
    version 0.1.1
    # Adds new enum values for x
    end
    version 0.1.2
    # Adds support for usage together with y in some_module
        use_types some_module 1.3
    end
    ```

Maybe use some other keyword than `version`, to clarify that it is not about
implementation versions?

- `api`
- `interface`  (best?)
- `since` (sounds strange in case of the first version)


Have `use` lines for interface and `dependencies.index` for implementation
--------------------------------------------------------------------------

* This is inconsistent obviously.
* Could have a different keyword for interface dependendencies.
    - Perhaps `interface_depends` like old SLUL?
    - Or something without an underscore:
        - `interfacedepends ...`
        - `interface depends ...`
        - `with type <type> from ...`
        - `with type <module>.<type> ...`
        - `with Type from ...` (good. but long:)
        - `with Type from <module> <version> since <sincever> <hash>`
        - `use type <type> from ...`
        - `using types from ...`
        - `external <type> from ...`
        - `here <type> = <module> <version>`
        - `expose <type> from <module> <version>`
        - `export <type> from <module> <version>`
        - `export <module> <version>`
        - `expect <module> >= <version>`
        - `extract <type> from <module> <version>`
        - `require depends <module> <version>`
        - `mustdepend <module> <version> since <sinceversion>`
           (a bit misleading if deps can be minimized)
        - `musthave ...`
        - `transitive <module> <version> since <sinceversion>`
        - `mutual dep ...`
        - `shared dep ...`
        - or some abbreviated word: `idep`, `mdep`, `sdep`
    - Also, perhaps the API hash should be included. And it should go first
      for alignment:
        - `expose <hash> <module> <version> since <sinceversion>`
        - `typesfrom <hash> <module> <version> since <sinceversion>`
           (good, but long)
        - `use ...`
        - or use a multi-line format (but with 43-character API-hashes,
          there's not a lot of space left. Even with base94 it would be
          40 characters).
        - if the API-hash can be reduced to 160 bits, then 27 characters is
          enough (`log(64**27,2)==162`). With a short keyword like `use`,
          it should be possible to fit within 80 characters wide lines.
        - maybe it can be shortened further... see below.
* How to tokenize the version? (it's not a float)
    - in old SLUL, the version appears in the module header,
      so it had a separate tokenizer anyway.
    - having the parser "tell" the tokenizer whether floats or versions
      are expected can work, but is a bit ugly.
    - combine/prefix with `:`? e.g. `somemodule:1.0 since :0.3`
    - or tokenize together with the preceeding keyword
      (e.g. `version 1.2.3` or `since 1.2.3`)
    - if using API-hashes, then the name and version are just for the user
      (and to check that the user is not being tricked). So they could go
      into double quotes, as a string:
        ```
        interfacedepends AAAAAAAAAAAA "longmodulename 123.456.789" since 1.2
        ```
* How to tokenize the API-hash?
* include the API hash in versions?
    - It's important to use the right module.
    - And the interface.slul file should be freestanding.
    - BUT the compiler will do API checks at compile-time, so there's no
      security reason to have a long hash here. Maybe it could be really short,
      like 12 characters? `log(64**12,2)==72`
        - the package registry could enforce that no API-version of any module
          has a the same hash as any other version (whether in the same module
          or not).
        - uniqueness could also be enforced at client side at the developer
          even across different registries
        - uniqueness could also be enforced at load time, to prevent loading
          two different API-hashes with the same 72-bit prefix.
        - security-critical libraries could choose to use the full 43-character
          hash (without a trailing `=`).
    - This can go wrong if the API-hash is absent in the interface file:
        - The user might compile with an incorrect dependency, and end up with
          a different API-hash (and this would affect API-hashes of dependent
          modules), leading to a confusing error when trying to load some
          application using the library.
        - A library author might decide the change to an incompatible
          dependency with the exact same name and version. This change would
          ONLY go into the dependencies.index file. So it would break API
          without any changes at all to the interface file!
          (but shouldn't API hashes of the since-versions go into the
          interface file? or some kind of API-lockfile?)
    - And regardless, during development, one could allow the hash to be absent:
        ```
        interfacedepends "longmodulename 123.456.789"
        ```
    - Alternative solution: Put API-hashes in a separate API-lockfile.
      And the API-hashes can include API-hashes of interface dependencies as
      well.
* Several components are needed:
    - Module to depend on
    - And a mapping from since versions to dependency versions:
        - Module version to depend on
        - Since version
    - The mapping could be autodetected, but I think it is better to have
      explicit versions that can serve as documentation.
        - Maybe the <sinceversion> could be autodected, though?
        - Also, it is probably a good idea to auto-minimize interface
          dependencies.
        - Existing symbols should have API-hashes of some kind (either per
          symbol or per API-version), and those should include the interface
          dependencies and their versions:
            - So modification is prevented.
            - So safe autodetection is possible.
            - But error messages will not be as helpful
            - But it might not be as intuitive?
    - Could have repeated interface_depends lines, like old SLUL:
        ```
        interface_depends <module> <depversion1> since <sinceversion1>
        interface_depends <module> <depversion2> since <sinceversion2>
        ```
    - Or could have a list/block with multiple versions:
        ```
        interface_depends <module>
            <depversion1> since <sinceversion1>
            <depversion2> since <sinceversion2>
        end
        ```
    - Or could have a single list/block with all modules:
        ```
        interface_depends
            <module> <depversion1> since <sinceversion1>
            <module> <depversion2> since <sinceversion2>
        end
        ```
    - Or could have a depends line per symbol:
        ```
        since <sinceversion>
        types from <module> <version>
        func do_stuff
            int x
        end
        ```
    - Or depends line + types per symbol:
        ```
        since <sinceversion>
        type <types...> from <module> <version>
        func do_stuff
            int x
        end
        ```
    - Or sections:
        ```

        since <sinceversion>
        types from <module> <version>

        ...
        ```
    - Or sections with types:
        ```

        since <sinceversion>
        type <types...> from <module> <version>

        ...
        ```
    - Or versions with "use_types" lines (best?):
      (I think use_types is more intuitive than add_types)
        ```
        version 0.1.0
        # Initial version
            use_types some_module 1.0
        end
        version 0.1.1
        # Adds new enum values for x
        end
        version 0.1.2
        # Adds support for usage together with y in some_module
            use_types some_module 1.3
        end
        ```
    - Wildcard types (but how should this work with trees? if there is only
      a single lookup per occurence of a type, and searches are only done
      at {underscore, start of uppercase sequence, end}, then it might work?):
        ```
        type Gtk* from gtk3 3.0
        ```

Have a `main.slul` file
-----------------------

* This would contain `use` lines to specify dependencies on other modules.
* In addition, it could contain functions/constants
    - but this might be a bad thing.
    - mixing code and dependencies could lead to merge conflicts.
* In libraries, any `use` lines in `interface.slul` would be implicitly
  includes in the implementation too (as if they were in `main.slul`).


Have an `implementation.slul` file
----------------------------------

* This would only contain `use` lines.
* What should it be called?


Have separate "interface modules"
---------------------------------

* That is apparently what Guile does:
    https://ekaitz.elenq.tech/fasterMes5.html
* It could allow implementing more than one interface in a module.
* It might actually simplify the bootstrap compiler
* But it also means more work when creating a library.
* This is unrelated to whether the interface goes into one or multiple files.


Use sections/markers to combine multiple files into one
-------------------------------------------------------

This is not only useful for library interfaces, it could also be used to
create single-file applications. Basically, it would be structured something
like this:

    MARKER <FILENAME1>
    CONTENTS1
    ....

    MARKER <FILENAME2>
    CONTENTS2

I think there might already be some existing (human-readable) file format for
this, which allows multiple files to be placed in a single file.
* There's of course multipart MIME, but it's not that readable, and it's
  perhaps too flexible/complex? And it also requires MIME-type assignments
  (which can be personal prs.* in the beginning, though)
* Cookie-jar format, but extended with filenames:
    % file1
    % file2

If files "purposes" are based on file extensions, then it could look like
this:

    .deps:

    stdlib 1.0
    somelib 0.5

    .slul:

    class Xyz
    ...

To trigger this multi-file mode, the first non-comment line would have to be
a file marker line.

Actually, this could be generalized to class files also:

    .deps:

    stdlib 1.0
    somelib 0.5

    Xyz.slul:
    ...

Other syntaxes:

    [.deps]
    [Xyz.slul]

    === .deps
    === Xyz.slul

    --- .deps
    --- Xyz.slul

The multi-file syntax should only be allowed for interface files and for
single-file modules (otherwise it could be confusing). For interface files,
the file name could be `interface.slul` (or `<modulename>.slul`). For
single-file modules, the filename could be simply `<modulename>.slul`.