Operators
Operators are APIs that enable developers to customize their data applications. The operators have opinionated function signatures and flexible types. Some operators can be used independently, whereas others must be chained to accomplish the task.
Summary
The engine implements transforms
operators chained in an array inside a transforms section and named
operators following a hierarchical convention.
The transforms operators are:
The named operators are:
The engine also uses keyword operators to identify a multi-step operation:
Let's take a look at each operator in detail.
Transforms Operators
The transforms
section is an array that allows multiple operators to be chained and perform composite operations.
map
The map
operator takes an input record, applies a transformation, and forwards the modified record to the next operator.
For example, given an user record, you can mask their social security number with the following map
operation:
transforms:
- operator: map
run: |
fn mask_ssn(user: User) -> Result<User> {
let mut u = user.clone();
u.ssn = u.ssn.replace(|c: char| c.is_digit(10), "*");
Ok(u)
}
Check out map example in github for a working demo.