Skip to content

v0.9.59.rc0

Pre-release
Pre-release
Compare
Choose a tag to compare
@basetenbot basetenbot released this 09 Jan 22:22
· 25 commits to main since this release
v0.9.59.rc0
5f835d0
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

What's Changed

  • Client side validation for non fp8 kv cache and fp8 context fmha by @joostinyi in #1302
  • Add encoder model support in trt-llm by @michaelfeil in #1294
  • Add kv_cache_host_memory_bytes as a configurable runtime setting by @joostinyi in #1303

Full Changelog: v0.9.58...v0.9.59.rc0

Contributors

michaelfeil and joostinyi
Assets 4
Loading