Data

Added

• Added a metadata solution to support the ability to get a list of volatile partitions that have changed since a provided timestamp:

  • By including a timestamp into your volatile metadata query, you can now request a list of volatile partitions that have changed since that timestamp. When processing or downloading volatile data, you only need to request the changes rather than download all volatile data, and thus improving your workflow efficiency and data transmission costs.
    • See related SDK release notes: "Added: getVolatilePartitions method in QueryAPI"
  • API deprecation notice:
    • Support of the existing functionality to get all volatile partitions that changed between version Vn and Vm will continue to exist for the next 6 months, after which point this support will be deprecated. Also note that as of this release, the result of that type of query is slightly different. The result now contains all the partitions that changed since the time Vn was committed.

Fixed

• A new Spark Connector has been included in the SDK to support the consumption of Parquet, Avro and Protobuf formatted data from an Index Layer. See the SDK release notes for more information about this Connector.

Marketplace

Added

• Made improvements to the Marketplace subscription usage report making the information meaningful and easy to understand for OLP users.
  Improvements include:

  • User friendly subscription heading and column titles
  • Subscription heading is now the subscribed data catalog name
  • Added data catalog HRN
  • Added layer HRN next to the layer name

Fixed

• Resolved the issue where when a Marketplace Provider grants data access to a Marketplace Consumer, there could be a delay before the Consumer sees the data catalog in their licensed data area, because the confirmation email is only sent after the consumer has accepted the subscription.