StreamX Key Rewrite feature
The StreamX Key Rewrite feature enables the transformation of the resource keys from the source system into the desired publication key format within StreamX Mesh. The feature is based on match-rewrite regular expression pattern pairs and uses the google/re2j package for regular expression processing.
Key Rewrite configuration
To configure the Key Rewrite feature, add the following environment variables in the StreamX Rest Ingestion YAML file:
ingestion:
rest-ingestion:
environment:
STREAMX_INBOXES_KEY-REGEX-REWRITE_PATTERNS_CONFIG1_MATCH-PATTERN: "MATCH_PATTERN"
STREAMX_INBOXES_KEY-REGEX-REWRITE_PATTERNS_CONFIG1_REWRITE-PATTERN: "REWRITE_PATTERN"
Where:
-
STREAMX_INBOXES_KEY-REGEX-REWRITE_PATTERNS
is a property prefix used for a key rewrite configuration., -
CONFIG1
is a key rewrite pattern identifier for match-rewrite pattern pair. -
"MATCH_PATTERN"
is a match pattern definition. -
"REWRITE_PATTERN"
is a rewrite pattern definition.
Key Rewrite pattern identifier
More than one pattern for transforming resource keys can be defined. To achieve this, add several entries under the 'environment' property, and assign each match-rewrite pattern pair a unique identifier. It is supported by chaining CONFIGX
, where X
pointing to testing matcher invocation order. See the following example:
ingestion:
rest-ingestion:
environment:
STREAMX_INBOXES_KEY-REGEX-REWRITE_PATTERNS_CONFIG1_MATCH-PATTERN: "MATCH_PATTERN_1"
STREAMX_INBOXES_KEY-REGEX-REWRITE_PATTERNS_CONFIG1_REWRITE-PATTERN: "REWRITE_PATTERN_1"
STREAMX_INBOXES_KEY-REGEX-REWRITE_PATTERNS_CONFIG2_MATCH-PATTERN: "MATCH_PATTERN_2"
STREAMX_INBOXES_KEY-REGEX-REWRITE_PATTERNS_CONFIG2_REWRITE-PATTERN: "REWRITE_PATTERN_2"
The order of processing match patterns is determined lexicographically by their identifiers. For the example given above:
CONFIG1
is processed before CONFIG2
.
A resource key can only be rewritten by one matcher. Once a match is found, further matchers are not processed.
Match-rewrite pair definitions
Match pattern
A match pattern is a regular expression that identifies publication keys for transformation. It can contain groups that correspond to specific parts of the key.
A group is denoted by (
and )
, it can be used for matching parts of strings in match pattern and capturing the matched parts.
Match-rewrite patterns use example
To transform the resource key from:
"example.com/${path}"
to:
"example.subdomain.com/${path}"
the following example can be used.
Match pattern example
The match pattern should capture the specific parts of the original resource key that are needed to transform and retain the necessary parts. The match pattern can look like this:
STREAMX_INBOXES_KEY-REGEX-REWRITE_PATTERNS_CONFIG1_MATCH-PATTERN: "^example.com/(.*)$"
Explanation of the match pattern:
-
^www.example.com/
: Asserts that the match must start at the beginning of the string. -
(.*)
: Captures everything after the domain, which corresponds to the path portion of the URL. The group can be referenced with$1
in the rewrite pattern.
Rewrite pattern example
The rewrite pattern reforms the matched keys according to the desired output:
STREAMX_INBOXES_KEY-REGEX-REWRITE_PATTERNS_CONFIG1_REWRITE-PATTERN: "example.subdomain.com/$1"
Explanation of the rewrite pattern:
-
example.subdomain.com/
: This part specifies a static string for the new domain, changing it fromexample.com/
. -
$1
: references the first captured group, which corresponds to the path part of the URL, ensuring that this part is retained in the transformed key.
For guidance on constructing regular expressions and how to use groups, refer to the RE2J documentation. |