Introduction

Symbiont is a purpose-built language for expressive data manipulation. It is named so, because it relies on a mutualistic symbiotic relationship with a host language.

As an expressive scripting language, Symbiont is able to keep and update state of data, as well as consume from and publish to external data sources. The extent of these capabilities heavily rely on the host language and implementation with that language.

By design, the language is intended to be decoupled from the host language. The main host language is PHP. However, the syntax is purposely designed around Vaughan Pratt’s Top Down Operator Precedence. The implementation details are referenced from Douglas Crockford’s paper on a parser for Simplified JavaScript that is written in Simplified Javascript.

In the wild

Symbiont is used as an example language for the talk On top of the Elephpant. (Nov. 5th 2020)

Feel free to join. No prior knowledge of programming language design is expected.

Syntax example

A brief example of the syntax:

$catalog: @products;
$product: $catalog.ensure('tnt');

$product
    .forEach($_.media_gallery_entries)
    .where($_.media_type = 'image')
    .keepIf($_.file matches pcre2:/portrait\.(jpg|png)$/);

The example consists of:

The following operations are performed:

  1. Variable $catalog points to storage products.
  2. Variable $product points to storage products entry tnt. If entry tnt could not be found, it is created according to a predetermined structure, specific to the products storage.
  3. For each child of the media_gallery_entries data node of $product having a media type image, a filter is applied. Only entries with a file that end in portrait.jpg or portrait.png are kept.
  4. End of program clean-up is triggered. This ensures the modifications on $product are persisted to the products storage.

The previous example is written verbosely to highlight individual parts of the syntax. The following code should work exactly the same:

@products('tnt')
    .forEach($_.media_gallery_entries)
    .where($_.media_type = 'image')
    .keepIf($_.file matches pcre2:/portrait\.(jpg|png)$/);

Applying the code

The following shows a data structure, before and after applying the Symbiont program:

[
   {
      "sku": "tnt",
      "media_gallery_entries": [
         {
            "media_type": "image",
            "file": "tnt_product_display_portrait.jpg"
         },
         {
            "media_type": "image",
            "file": "tnt_product_display_landscape.jpg"
         },
         {
            "media_type": "video",
            "file": "explosion_default_yield.mp4"
         }
      ]
   }
]

After having applied the program, the structure has been updated to the following:

[
   {
      "sku": "tnt",
      "media_gallery_entries": [
         {
            "media_type": "image",
            "file": "tnt_product_display_portrait.jpg"
         },
         {
            "media_type": "video",
            "file": "explosion_default_yield.mp4"
         }
      ]
   }
]

The following happened:

  1. The product with SKU tnt got retrieved.
  2. An operation was prepared for each entry of media_gallery_entries.
  3. The operation was narrowed down to only apply to nodes having media_type set to image.
  4. Nodes were kept if their file property matched against the given regular expression. In this case, the file had to end in portrait and either use the .jpg or .png extension.
  5. Changes were automatically persisted.

This effectively removed the “landscape” oriented product image, keeping the portrait image and video file.

How to use Symbiont

Currently, Symbiont can be run on a local installation of PHP >= PHP 7.4, or alternatively inside a Docker container.

Environment variables

The following environment variables influence the execution of Symbiont.

Variable Possible values Description
SYMBIONT_VERBOSE Any non-zero length string. Shows the PHP stack trace on top of the Symbiont exception output.
SYMBIONT_MODE tokenize, parse, graph Defaults to parse. Interprets the program up until the matching step and outputs the result.

Symbiont modes

For each SYMBIONT_MODE, the following is the result:

Mode Output Combined with SYMBIONT_VERBOSE
tokenize Tokens, line by line, with their value. The file path, start of the token and end of the token are prefixed to each line.
parse AST as a JSON object. No additional effects.
graph AST as a Graphviz digraph. No additional effects.

Local PHP installation

When PHP is installed locally, run your script as follows:

bin/symbiont /path/to/script.sym

Alternatively, one can start their script with a Shebang pointing to Symbiont.

#!/path/to/symbiont

Since Symbiont is programmed to ignore all comments and whitespace while parsing, this does not interfere with the script.

Docker container

★ Once Symbiont reaches a better stability, images will be pushed to Docker Hub.

To build a fresh Docker image running Symbiont, run:

docker build -t symbiont:latest .

Then, to run a local symbiont script:

docker run --rm -v $PWD:/app symbiont:latest examples/types/numbers.sym

Development

To rebuild a fresh image after each changed line of code, would slow down development.

While one can set up a prepared running container and mount the local project, for ease of use, there is a dev script that uses an existing image with all the platform requirements installed.

./dev tests/types/numbers.sym

Watching Symbiont files

Within the project, a file watcher is configured for the fileExtension called Symbiont. It uses the dev command to verify changes in *.sym files.

In order for the watcher to work, a file type Symbiont with file name pattern *.sym has to be created.

Follow this guide to create new file type associations within PhpStorm.

Once the watcher works, it checks if it can parse the current file on change. If the parser works as expected, nothing happens. When an error occurs, the console opens with the error output of the parser.

Environment variable aliases

The dev script listens both to the documented environment variables, and a version without the SYMBIONT_ prefix. E.g.:

MODE=tokenize ./dev examples/types/numbers.sym

Has the same effect as:

SYMBIONT_MODE=tokenize ./dev examples/types/numbers.sym

If a pre-existing environment variable conflicts, use the version with prefix. Prefixed versions supersede the non-prefixed versions.

E.g.:

MODE=tokenize SYMBIONT_MODE=parse ./dev examples/types/numbers.sym

In the above case, the used mode will be parse.

Testing

To run unit tests against the library files, use the Symbiont configured phpunit.xml.dist run configuration within PhpStorm.

If Composer packages are out of date, first run the composer update --dev run configuration within PhpStorm.