OPENSEARCH-3.7.0 release notes and known issues

Summary

Version 3.7.0 Release Notes Compatible with OpenSearch and OpenSearch Dashboards version 3.7.0 Features Add dynamic properties support for pattern-based field definitions without cluster state mapping updates ( #20816 ) Add pluggable data format engine with DataFormatAwareEngine for multi-format indexing ( #21181 ) Add Lucene engine implementation for pluggable data formats ( #21299 ) Add merge support for Parquet data format plugin via streaming k-way merge sort ( #21079 ) Add directory and IndexInput layers for WritableWarm tiered storage ( #21178 ) Add server-side implementation for tiering status APIs (GetTieringStatus and ListTieringStatus) ( #21220 ) Add server-side implementation for HotToWarm, WarmToHot, and CancelTiering APIs ( #21295 ) Add prefetch settings and stored fields prefetch for WritableWarm tiered storage ( #21285 ) Add slow logs, per-query metrics, and migration metrics for WritableWarm tiered storage ( #21332 ) Add module wiring and integration tests for WritableWarm tiered storage ( #21427 ) Add tiered object storage crate for warm node file routing ( #21204 ) Add event-driven scheduler and stage execution for analytics engine ( #21242 ) Add coordinator-side DataFusion reduce with streaming Arrow batches ( #21356 ) Add distributed aggregation with partial/final mode for analytics engine ( #21457 ) Add distributed join planning and execution for analytics engine ( #21639 ) Add PPL append command support with multi-child stage runtime for Union ( #21474 ) Add PPL dedup command support via ROW_NUMBER window function ( #21622 ) Add PPL eventstats and streamstats window function support ( #21734 ) Add PPL top and rare command support via window functions ( #21593 ) Add PPL parse command with regex mode via Rust UDFs ( #21573 ) Add PPL rex command with sed and extract modes ( #21550 ) Add PPL spath command with auto-extract mode via json_extract_all UDF ( #21664 ) Add 7 PPL JSON scalar functions to analytics engine route ( #21513 ) Add 23 PPL datetime scalar functions to analytics engine route ( #21556 ) Add 14 additional PPL datetime functions (Wave A) including strftime, date_format, maketime ( #21582 ) Add 30+ PPL math scalar functions to analytics engine ( #21520 ) Add PPL string scalar functions to analytics engine (18 functions) ( #21543 ) Add PPL conditional functions (coalesce, isempty, isblank, case, if, ifnull) to analytics engine ( #21643 ) Add PPL conversion scalar functions (num, auto, memk, rmcomma, dur2sec, ctime, mktime) to analytics engine ( #21628 ) Add PPL cryptographic functions (md5, sha1, sha2, crc32) to analytics engine ( #21611 ) Add PPL array constructor and 8 multivalue functions to analytics engine ( #21554 ) Add PPL bucketing scalars (span_bucket, width_bucket, minspan_bucket, range_bucket) ( #21621 ) Add PPL TAKE, FIRST, LAST, LIST, VALUES aggregate functions ( #21731 ) Add Lucene filter delegation from DataFusion for full-text search predicates ( #21555 ) Add performance delegation to Lucene for selective filter predicates ( #21701 ) Add native Arrow transport path with zero-copy transfer for stream transport ( #21253 ) Stream Arrow batches on data-node fragment execution path ( #21418 ) Add support for extra_fields outside _source indexing for improved vector ingestion throughput ( #20635 ) Add gRPC support for Min, Max, and Terms aggregations ( #21205 ) Add partition strategy setting for flexible shard-to-partition mapping in pull-based ingestion ( #21165 ) Add SplitToFieldsProcessor for distributing split values to target fields ( #21216 ) Add native memory based admission control for transport request throttling ( #21191 ) Add native memory search backpressure for off-heap query cancellation ( #21647 ) Add unified native allocator framework for Arrow allocations with elastic rebalancing ( #21703 ) Add on-demand jemalloc heap profiling support via JMX CLI tool ( #21599 ) Add search.max_buckets to workload group settings for per-tenant bucket limits ( #21721 ) Add additional sea

Improvements and security content

• Version 3.7.0 Release Notes Compatible with OpenSearch and OpenSearch Dashboards version 3.7.0 Features Add dynamic properties support for pattern-based field definitions without cluster state mapping updates ( #20816 ) Add pluggable data format engine with DataFormatAwareEngine for multi-format indexing ( #21181 ) Add Lucene engine implementation for pluggable data formats ( #21299 ) Add merge support for Parquet data format plugin via streaming k-way merge sort ( #21079 ) Add directory and IndexInput layers for WritableWarm tiered storage ( #21178 ) Add server-side implementation for tiering status APIs (GetTieringStatus and ListTieringStatus) ( #21220 ) Add server-side implementation for HotToWarm, WarmToHot, and CancelTiering APIs ( #21295 ) Add prefetch settings and stored fields prefetch for WritableWarm tiered storage ( #21285 ) Add slow logs, per-query metrics, and migration metrics for WritableWarm tiered storage ( #21332 ) Add module wiring and integration tests for WritableWarm tiered storage ( #21427 ) Add tiered object storage crate for warm node file routing ( #21204 ) Add event-driven scheduler and stage execution for analytics engine ( #21242 ) Add coordinator-side DataFusion reduce with streaming Arrow batches ( #21356 ) Add distributed aggregation with partial/final mode for analytics engine ( #21457 ) Add distributed join planning and execution for analytics engine ( #21639 ) Add PPL append command support with multi-child stage runtime for Union ( #21474 ) Add PPL dedup command support via ROW_NUMBER window function ( #21622 ) Add PPL eventstats and streamstats window function support ( #21734 ) Add PPL top and rare command support via window functions ( #21593 ) Add PPL parse command with regex mode via Rust UDFs ( #21573 ) Add PPL rex command with sed and extract modes ( #21550 ) Add PPL spath command with auto-extract mode via json_extract_all UDF ( #21664 ) Add 7 PPL JSON scalar functions to analytics engine route ( #21513 ) Add 23 PPL datetime scalar functions to analytics engine route ( #21556 ) Add 14 additional PPL datetime functions (Wave A) including strftime, date_format, maketime ( #21582 ) Add 30+ PPL math scalar functions to analytics engine ( #21520 ) Add PPL string scalar functions to analytics engine (18 functions) ( #21543 ) Add PPL conditional functions (coalesce, isempty, isblank, case, if, ifnull) to analytics engine ( #21643 ) Add PPL conversion scalar functions (num, auto, memk, rmcomma, dur2sec, ctime, mktime) to analytics engine ( #21628 ) Add PPL cryptographic functions (md5, sha1, sha2, crc32) to analytics engine ( #21611 ) Add PPL array constructor and 8 multivalue functions to analytics engine ( #21554 ) Add PPL bucketing scalars (span_bucket, width_bucket, minspan_bucket, range_bucket) ( #21621 ) Add PPL TAKE, FIRST, LAST, LIST, VALUES aggregate functions ( #21731 ) Add Lucene filter delegation from DataFusion for full-text search predicates ( #21555 ) Add performance delegation to Lucene for selective filter predicates ( #21701 ) Add native Arrow transport path with zero-copy transfer for stream transport ( #21253 ) Stream Arrow batches on data-node fragment execution path ( #21418 ) Add support for extra_fields outside _source indexing for improved vector ingestion throughput ( #20635 ) Add gRPC support for Min, Max, and Terms aggregations ( #21205 ) Add partition strategy setting for flexible shard-to-partition mapping in pull-based ingestion ( #21165 ) Add SplitToFieldsProcessor for distributing split values to target fields ( #21216 ) Add native memory based admission control for transport request throttling ( #21191 ) Add native memory search backpressure for off-heap query cancellation ( #21647 ) Add unified native allocator framework for Arrow allocations with elastic rebalancing ( #21703 ) Add on-demand jemalloc heap profiling support via JMX CLI tool ( #21599 ) Add search.max_buckets to workload group settings for per-tenant bucket limits ( #21721 ) Add additional sea

Affected products and versions

Recommended action

Review the official publisher document before deployment.

Official publisher evidence