AttentionVCGitHubVC
Trending ReposTrending BuildersPortfolio
AttentionVC

Built by JustinZ and Jennie

AttentionVCA product by AttentionVC
Back to Trending
NVIDIA
NVIDIA/

TensorRT-LLM

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

Pythonblackwellcudallm-servingmoepytorch
Stars

15.1k

+1 /wk+2 /mo
Forks

3.0k

Issues

2.1k

Watchers

15.1k

Star History

Repository Info

LicenseNOASSERTION
CreatedAug 16, 2023
Last push2h ago
Homepagenvidia.github.io/TensorRT-LLM
Open on GitHub