srakahouses.blogg.se - Recompose liquid notes v1.5.2.1 win

#RECOMPOSE LIQUID NOTES V1.5.2.1 WIN CODE#
#RECOMPOSE LIQUID NOTES V1.5.2.1 WIN MAC#

Refactor engine cache serialization/deserialization logic.

Improve subgraph partitioning and consolidate TensorRT subgraphs when possible.

Workaround long CI test time caused by frequent initialization/de-initialization of TensorRT builder.

Provide option to share execution context memory between TensorRT subgraphs.

Quantization tool: improved ONNX shape inference for large models.Improved the thread job partition for QLinearConv, demonstrating up to ~20% perf gain for certain models.Specialized kernel to improve performance of quantized Resize by up to 2x speedup.New quantized kernels for weight symmetry to improve performance on ARM64 little core (GEMM and Conv).Memory usage reduction through use of containers from the abseil library, especially inlined vectors used to store tensor shapes and inlined hash maps.CPU usage savings for infrequent inference requests by reducing thread spinning.Memory optimizations to support compute-intensive real-time inferencing scenarios (e.g.Operator performance optimizations, including GatherElements.Improved C++ APIs that now utilize RAII for better memory management.onnxruntime-openvino package available on Pypi (from Intel).NOTE: netstandard1.1 support is being deprecated in this release and will be removed in the 1.13 release.Additional target frameworks: net6.0, net6.0-android, net6.0-ios, net6.0-macos.

#RECOMPOSE LIQUID NOTES V1.5.2.1 WIN MAC#

Mac M1 support in Python and Java packages.

Build switch to disable usage of abseil library to remove dependency.

Support for feeding external initializers (for large models) as byte arrays for model inferencing.

#RECOMPOSE LIQUID NOTES V1.5.2.1 WIN CODE#

For use with custom op development to reuse ORT code.

Support for invoking individual ops without the need to create a separate graph.

BeamSearch operator for encoder-decoder transformers models.

netstandard1.1 support is being deprecated in this release and will be removed in the next ORT 1.13 release.

The legacy API will be deprecated in next release (ORT 1.13). If you have an EP using the legacy compiler API, please migrate to the lightweight compile API as soon as possible.

For Execution Provider maintainers/owners: the lightweight compile API is now the default compiler API for all Execution Providers (this was previously only available for the mobile build).