Using RagConnect -- an example

The full example is available at https://git-st.inf.tu-dresden.de/jastadd/ragconnect-minimal.

Preparation and Specification

The following examples are inspired by the real test case read1write2 The idea is to have two nonterminals, where input information is received on one of them, and - after transformation - is sent out by both.

Let the following grammar be used:

A ::= <Input:String> /<OutputOnA:String>/ B* ;
B ::= /<OutputOnB:String>/ ;

To declare receiving and sending tokens, a dedicated DSL is used:

// endpoint definitions
receive A.Input ;
send A.OutputOnA ;
send B.OutputOnB using Transformation ;

// mapping definitions
Transformation maps String s to String {:
  return s + "postfix";
:}

This defines A.Input to receive updates, and the other two tokens to send their value, whenever it changes. Additionally, a transformation will be applied on B.OutputOnB before sending out its value.

Such mapping definitions can be defined for receiving tokens as well. In this case, they are applied before the value is set. If no mapping definition is given, or if the required type (depending on the communication protocol, see later) does not match, a "default mapping definition" is used to avoid boilerplate code converting from or to primitive types.

Furthermore, let the following attribute definitions be given:

syn String A.getOutputOnA() = "a" + getInput();

syn String B.getOutputOnB() = "b" + input();
inh String B.input();
eq A.getB().input() = getInput();

In other words, OutputOnA depends on Input of the same node, and OutputOnB depends on Input of its parent node. Currently, those dependencies can be explicitly written down, or incremental evaluation can be used.

Dependency tracking: Manually specified

This specification happens also in the DSL (dependencies have to be named to uniquely identify them):

// dependency definitions
A.OutputOnA canDependOn A.Input as dependencyA ;
B.OutputOnB canDependOn A.Input as dependencyB ;

Dependency tracking: Automatically derived

To automatically track dependencies, the two additional parameters --incremental and --trace=flush have to be provided to both RagConnect and (in the later stage) JastAdd. This will generate a different implementation of RagConnect relying on enabled incremental evaluation of JastAdd. The value for incremental has only been tested for incremental=param. The value for trace can include other values besides flush.

An experimental, optimized version can be selected using --experimental-jastadd-329 reducing the risk of conflicts between concurrent attribute evaluations. However, this requires a version of JastAdd that resolved the issue 329.

Using generated code

After specifying everything, code will be generated if setup properly. Let's create an AST in some driver code:

A a = new A();
// set some default value for input
a.setInput("");
B b1 = new B();
B b2 = new B();
a.addB(b1);
a.addB(b2);

If necessary, we have to set the dependencies as described earlier.

// a.OutputOnA -> a.Input
a.addDependencyA(a);
// b1.OutputOnB -> a.Input
b1.addDependencyB(a);
// b2.OutputOnB -> a.Input
b2.addDependencyB(a);

Finally, we can actually connect the tokens. Depending on the enabled protocols, different URI schemes are allowed. In this example, we use the default protocol: MQTT.

a.connectInput("mqtt://localhost/topic/for/input");
a.connectOutputOnA("mqtt://localhost/a/out", true);
b1.connectOutputOnB("mqtt://localhost/b1/out", true);
b2.connectOutputOnB("mqtt://localhost/b2/out", false);

The first parameter of those connect-methods is always an URI-like String, to identify the protocol to use, the server operating the protocol, and a path to identify the concrete token. In case of MQTT, the server is the host running an MQTT broker, and the path is equal to the topic to publish or subscribe to. Please note, that the first leading slash (/) is removed for MQTT topics, e.g., for A.Input the topic is actually topic/for/input.

For sending endpoints, there is a second boolean parameter to specify whether the current value shall be sent immediately after connecting.

Remarks for using manual dependency tracking

When constructing the AST and connecting it, one should always set dependencies before connecting, especially if updates already arriving for receiving endpoints. Otherwise, updates might not be propagated after setting dependencies, if values are equal after applying transformations of mapping definitions.

As an example, when using the following grammar and definitions for RagConnect ...

A ::= <Input:int> /<Output:String>/ ;
receive A.Input using Round ;
send A.Output ;

A.Output canDependOn A.Input as dependency1 ;

Round maps float f to int {:
  return Math.round(f);
:}

... connecting first could mean to store the first rounded value and not propagating this update, since no dependencies are set, and not propagating further updates leading to the same rounded value even after setting the dependencies.

An advanced example

Non-terminal children can also be selected as endpoints (not only tokens).

Normal Non-Terminal Children

Receiving normal non-terminal children and optionals means to replace them with a new node deserialized from the received message. Sending them involves serializing a node, and sending this representation in a message.

Suppose, the following (shortened) grammar is used (inspired from the testcase tree)

Root ::= SenderRoot ReceiverRoot ;
SenderRoot ::= <Input:int> /Alfa/ ;
ReceiverRoot ::= Alfa ;
Alfa ::= // some content ...

Now, the complete node of type Alfa can be sent, and received again using the following connect specification:

send tree SenderRoot.Alfa ;
receive tree ReceiverRoot.Alfa ;

Currently, receiving and sending trees requires the explicit demarcation from tokens using the keyword tree.

To process non-terminals, default mappings are provided for every non-terminal type of the used grammar. They use the JSON serialization offered by the RelAST compiler, i.e., interpret the message as a String, deserialize the content reading the message as JSON, or vice versa. Additional dependencies are required to use this feature, as detailed in the compiler section.

Receiving List Children

When receiving list children, there are a few more options to match the connection to given requirements.

Suppose we use a similar grammar as above, i.e.:

SenderRoot ::= /AlfaList:Alfa*/ /SingleAlfa:Alfa/;
ReceiverRoot ::= Alfa* ;

Several options are possible:

list

A message for a list endpoint can be interpreted as a complete list (a sequence of nodes of type Alfa) by using the list keyword instead of tree:

receive list ReceiverRoot.Alfa ;

list + with add

Upon receiving the message, the deserialized list can also be appended to the existing list instead of replace the latter. This can be achieved using the keyword with add in addition to the keyword list:

receive list with add ReceiverRoot.Alfa ;

tree (indexed)

A message for a list endpoint can also be interpreted as an element of this list.

receive tree ReceiverRoot.Alfa ;

Upon connection, the index of the deserialized element to set, has to be passed (1 in the example below). The list must have enough elements once a message is received.

receiverRoot.connectAlfa("<some-url>", 1);

tree (wildcard)

Similar to the tree (indexed) case above, messages are interpreted as an element of the list, but the connection can also be made using a "wildcard topic" and without an index. Then, once a message is received from a new concrete topic, the deserialized element will be appended to the list and this topic is associated with the index of the newly added element. Any further message from that topic will replace the element at the associated index. In the short example below, MQTT is used to with a wildcard topic, as # matches every sub-topic.

receiverRoot.connectAlfa("mqtt://<broker>/some/topic/#");

// list is initially empty
assertEquals(receiverRoot.getAlfaList(), list());
// after receiving "1" on new topic "some/topic/one" (index 0)
assertEquals(receiverRoot.getAlfaList(), list("1"));
// after receiving "other" on new topic "some/topic/two" (index 1)
assertEquals(receiverRoot.getAlfaList(), list("1", "other"));
// after receiving "new" on existing topic "some/topic/one" (index 0)
assertEquals(receiverRoot.getAlfaList(), list("new", "other"));

tree (indexed/wildcard) + with add

Combining tree and with add results in a connection, where messages are interpreted as elements of the list, and new elements are appended to the existing list. In that case, wildcard and non-wildcard connections behave in the same way, as no index has to be passed, and the element is always append at the end. Reusing the example from above, the following observations can be made.

receiverRoot.connectAlfa("mqtt://<broker>/some/topic/#");
// or
receiverRoot.connectAlfa("mqtt://<broker>/some/topic/one");
receiverRoot.connectAlfa("mqtt://<broker>/some/topic/two");

// list is initially empty
assertEquals(receiverRoot.getAlfaList(), list());
// after receiving "1" on topic "some/topic/one"
assertEquals(receiverRoot.getAlfaList(), list("1"));
// after receiving "other" on topic "some/topic/two"
assertEquals(receiverRoot.getAlfaList(), list("1", "other"));
// after receiving "new" on topic "some/topic/one"
assertEquals(receiverRoot.getAlfaList(), list("1", "other", "new"));