HERE platform stream layer
- Last UpdatedFeb 27, 2025
- 1 minute read
The HERE platform stream layer data connector is implemented for source and sink connections.
This is the HERE platform stream layer data connector URI scheme:
olp+stream://platform.here.com/hrn:here:data::my-org:my-catalog/my-stream-layer
The source connector supports these query parameters:
consumer_group
: arbitrary stringoffset
:latest
orearliest
values
The source and sink connectors both support this optional query parameter:
creds_file
: path to the custom credentials file
This is an example of a source URI:
olp+stream://platform.here.com/hrn:here:data::my-org:my-catalog/my-stream-layer?consumer_group=MyGroup&offset=earliest&creds_file={CREDENTIALS_FILE_PATH}
Format auto-detection
The HERE platform stream layer connector supports format auto-detection. The connector recognizes the input and output data format automatically, which means that you can set SOURCE_FORMAT
and SINK_FORMAT
to auto
or leave the value empty.
The conditions for successful auto-detection are:
- The source and sink layers must have a schema assigned.
- Assigned schema name must contain
sensoris
,sdii_message_list
orsdii_message
.
Apache Kafka notice
Note that the HERE platform stream layer data connector uses an Apache Kafka backend. Kafka doesn't keep the order of incoming messages if parallel processing is enabled and only provides a total order of messages in a partition, not between different partitions in a topic.
If keeping the order of incoming messages is important for your use case, don't enable parallel processing. To ensure that parallel processing is disabled, set STREAM PARTITION PARALLELIZATION (PARTITIONS)
to 1
when you create a stream layer.