I've been thinking about a non-existent language for evaluating expressions on multiple data resources. It's key properties are:
- Limited capability
- Easily interpreted
- Global references to cross-application data models
- Logic owned by the backend
The fastest way to grok what I'm proposing is through a code snippet. Imagine this JSON object representing health potions in a game. Imagine the game syncs with a central server. In this example the predatory in-game salesman has configured the pricing such that health potions cost more if the player is below half health.
[
{"item": "heatlh-potion", "cost": 50, "available": "(>= $player.health 50)"}
{"item": "health-potion", "cost": 75, "available": "(< $player.health 50)"}
]
The application code might look like:
var availableItems = store.items.filter({ eval($0.available) })
When evaluated the available string will substitute the health property on the
registered player resource with the current value. Next it evaluates the lisp-y expression by
comparing that value to 50. In the end the value is either true or false and
this is passed to our normal Swift filter function.
Motivation
Why do something like this? Sure, it provides some flexibility, but is that really so useful? Yes. This approach translates nicely to a microservice architecture.
First, let me provide some historical context to our fictitious game. In our
game there are two resources $player and $store. Initial versions served
both resources from a monolith. The client always requested both
resources at once. When that was the case it was easy to ensure the
player always saw the right items. The Monolith would filter the set of items
before returning them to the client.
As time passed the resources were split into separate microservices, each exposing its own API. Perhaps clients called these directly. Perhaps an aggregator was introduced. (This is where you might invent something like GraphQL). No matter the mechanism, we're now providing subsets of data. You tell the backend what fields you want, and it'll give them you.
But things get messy as more and more fields depend on each other. How does the
client know which subset of fields to request? What if the client forgets to request
the $store while requesting the $player? That could lead to bugs!
To solve the problem we could go back to requesting everything at once, but we
all know what would happen. The app would slow down, battery life would drop, and
users would complain. Even if we were to correctly manage data dependencies,
there will be other obstacles. Rumor has it, some features are starting to
optimistically update state locally. How do you keep it all consistent!?
Solution
Here's the beauty of our JSON from above.
By moving evaluation of the available field to the client we've
decoupled the meaning of our data from its age.
We can refresh the $player resource every day and the store once a week and
there's no risk of users seeing invalid items.
[
{"item": "heatlh-potion", "cost": 50, "available": "(> $player.health 50)"}
{"item": "health-potion", "cost": 75, "available": "(<= $player.health 50)"}
]
There are other benefits.
- ETag caching becomes more effective since resources don't change as much.
- Clients use less bandwidth under these relaxed constraints
- Decoupled backend services no longer make calls to each other
- Expressions can be updated to include new dependencies with just a data change. These updates apply without requiring a new client version.
Open Questions
There are a ton. Does this language only include boolean expressions? Can it compute numbers? What about list or set operators? Stringy keys in maps? How does it handle other use-cases (removing a purchased item from the shop)? Can we nest evaluations? etc.
I don't have the answers. If you found this interesting let me know. If you read this and a use-case came to mind, let me know. Just send me a toot on mastodon: @hpincket@fosstodon.org