2020-2-capstone-design1 / HCS_project

Go to a project
Toggle navigation Toggle navigation pinning
김지훈
92dff93a 1 parent e2b95d2f

tensorRT ipynb

Inline Side-by-side
Showing 1 changed file with 299 additions and 0 deletions
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Untitled0.ipynb",
"provenance": [],
"private_outputs": true,
"collapsed_sections": []
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "8k6IDrTDGZ5W"
},
"source": [
"## Pytorch 추론시간 단축을 위한 정밀도 감소 - TensorRT / torch2trt library\n",
"\n",
"### 환경설정\n",
"\n",
"- [Nvidia developer portal](https://developer.nvidia.com/nvidia-tensorrt-7x-download) 회원가입\n",
"- TensorRT 7.0.0.11 for Ubuntu 1804 and CUDA 10.0 DEB local repo package 다운로드 후 colab 업로드\n",
"\n",
"### TensorRT 7.0 을 colab 에 설치하는 방법"
]
},
{
"cell_type": "code",
"metadata": {
"id": "ApojKcRcGf2j"
},
"source": [
"!nvcc --version"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "NZD-bw7YIlHL"
},
"source": [
"### colab CUDA 10.1 삭제"
]
},
{
"cell_type": "code",
"metadata": {
"id": "jJQrjrcfIm8u"
},
"source": [
"# remove CUDA 10.1\n",
"\n",
"!sudo apt-get --purge remove cuda nvidia* libnvidia-*\n",
"!sudo dpkg -l | grep cuda- | awk '{print $2}' | xargs -n1 dpkg --purge\n",
"!sudo apt-get remove cuda-*\n",
"!sudo apt autoremove\n",
"!sudo apt-get update"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "tRhvY4fZIygS"
},
"source": [
"### CUDA 10.0 설치 - 입력 세번 필요 (Y, 31, 1)"
]
},
{
"cell_type": "code",
"metadata": {
"id": "73tZNq2WI41u"
},
"source": [
"# Installing CUDA 10.0\n",
"\n",
"!wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb\n",
"!sudo dpkg -i cuda-repo-ubuntu1804_10.0.130-1_amd64.deb\n",
"!sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub\n",
"!sudo apt-get update\n",
"!wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb\n",
"!sudo apt install -y ./nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb\n",
"!sudo apt-get update\n",
"\n",
"# Install NVIDIA driver\n",
"!sudo apt-get -y installnvidia-driver-418\n",
"\n",
"# Install development and runtime libraries (~4GB)\n",
"!sudo apt-get install -y \\\n",
" cuda-10-0 \\\n",
" libcudnn7=7.6.2.24-1+cuda10.0 \\\n",
" libcudnn7-dev=7.6.2.24-1+cuda10.0 --allow-change-held-packages"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "5HAS_SQ7LBgj"
},
"source": [
"!nvcc --version"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "emv-dfsgLEE9"
},
"source": [
"### TensorRT 설치 - colab .deb 파일 경로 확인"
]
},
{
"cell_type": "code",
"metadata": {
"id": "YI96JETCLJ3I"
},
"source": [
"# install tensorrt\n",
"!sudo dpkg -i \"/content/drive/My Drive/capstone1/CAN/nv-tensorrt-repo-ubuntu1804-cuda10.0-trt7.0.0.11-ga-20191216_1-1_amd64.deb\"\n",
"!sudo apt-key add /var/nv-tensorrt-repo-cuda10.0-trt7.0.0.11-ga-20191216/7fa2af80.pub\n",
"\n",
"!sudo apt-get update\n",
"\n",
"!sudo apt-get install libnvinfer7=7.0.0-1+cuda10.0 libnvonnxparsers7=7.0.0-1+cuda10.0 libnvparsers7=7.0.0-1+cuda10.0 libnvinfer-plugin7=7.0.0-1+cuda10.0 libnvinfer-dev=7.0.0-1+cuda10.0 libnvonnxparsers-dev=7.0.0-1+cuda10.0 libnvparsers-dev=7.0.0-1+cuda10.0 libnvinfer-plugin-dev=7.0.0-1+cuda10.0 python-libnvinfer=7.0.0-1+cuda10.0 python3-libnvinfer=7.0.0-1+cuda10.0\n",
"\n",
"!sudo apt-mark hold libnvinfer7 libnvonnxparsers7 libnvparsers7 libnvinfer-plugin7 libnvinfer-dev libnvonnxparsers-dev libnvparsers-dev libnvinfer-plugin-dev python-libnvinfer python3-libnvinfer\n",
"\n",
"!sudo apt-get install tensorrt"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "DcpWP5snL2yy"
},
"source": [
"### TensorRT 설치 확인"
]
},
{
"cell_type": "code",
"metadata": {
"id": "uOW85nnxLdSB"
},
"source": [
"!dpkg -l | grep TensorRT"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "k7xHrhZuNb0U"
},
"source": [
"### 런타임 다시시작하기"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8EQ__AkjMH9G"
},
"source": [
"### [torch2trt](https://github.com/NVIDIA-AI-IOT/torch2trt) 라이브러리 설치"
]
},
{
"cell_type": "code",
"metadata": {
"id": "k7IztAyvMGlP"
},
"source": [
"cd /content/drive/My\\ Drive/capstone1/CAN"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "qR0V215wMHXF"
},
"source": [
"!pip3 install pycuda\n",
"!git clone https://github.com/NVIDIA-AI-IOT/torch2trt"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "fG9UtL6sNAgM"
},
"source": [
"cd /content/drive/My\\ Drive/capstone1/CAN/torch2trt"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "iMEc08ilNDW8"
},
"source": [
"!python setup.py install"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "LpK7xRFdNVbr"
},
"source": [
"import tensorrt\n",
"import torch2trt"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "M1DDuNL4Npn-"
},
"source": [
"### 추론시간 비교하기"
]
},
{
"cell_type": "code",
"metadata": {
"id": "9WddajL5Pj6E"
},
"source": [
"cd /content/drive/My\\ Drive/capstone1/CAN"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "PgKGWmSqNr4z"
},
"source": [
"import utils\n",
"import numpy as np\n",
"import importlib\n",
"importlib.reload(utils)\n",
"\n",
"utils.run_benchmark('./weights/fed_avg_50_0.9688.pth')"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "qh7_UaFsRykh"
},
"source": [
"- torch: 464 msc / 96.87 % / 0.05931\n",
"- trt: 354 msc / 96.87 % / 0.05931\n",
"- trt float 16: 323 msc / 96.87 % / 0.05931\n",
"- trt int8 strict: 401 msc / 96.86 % / 06405\n",
"\n",
"\n",
"#### int8 양자화 실패한 이유 추측\n",
"- input 이 0.0 ~ 1.0 이기 때문에 계산이 전부 float 연산\n",
"- nvidia tensorRT 가 정확도가 너무 낮아질 경우 양자화 무시\n",
"- 사용법 미숙, 모델이 너무 간단함\n"
]
}
]
}
\ No newline at end of file