-
-
Save ajeetraina/010edbdcdf576e06281d7da88e46345f to your computer and use it in GitHub Desktop.
| $ mkdir -p ~/.docker/cli-plugins && docker pull docker/docker-model-cli-desktop-module:v0.1.24 && docker create --name extractor docker/docker-model-cli-desktop-module:v0.1.24 nop && docker export extractor > _extract.tmp.tar && tar -xOf _extract.tmp.tar cli-plugins/model/linux/docker-model > ~/.docker/cli-plugins/docker-model && chmod +x ~/.docker/cli-plugins/docker-model && rm _extract.tmp.tar && docker rm extractor && docker rmi docker/docker-model-cli-desktop-module:v0.1.24 | |
| $ docker model run ai/llama3.2 | |
| _MODEL_RUNNER_TREAT_DESKTOP_AS_MOBY=1 |
export _MODEL_RUNNER_TREAT_DESKTOP_AS_MOBY=1
ajeetraina@ajeetraina-desktop:~/Downloads$ mkdir -p ~/.docker/cli-plugins && docker pull docker/docker-model-cli-desktop-module:v0.1.24 && docker create --name extractor docker/docker-model-cli-desktop-module:v0.1.24 nop && docker export extractor > _extract.tmp.tar && tar -xOf _extract.tmp.tar cli-plugins/model/linux/docker-model > ~/.docker/cli-plugins/docker-model && chmod +x /.docker/cli-plugins/docker-model && rm _extract.tmp.tar && docker rm extractor && docker rmi docker/docker-model-cli-desktop-module:v0.1.24/Downloads$
v0.1.24: Pulling from docker/docker-model-cli-desktop-module
764efb7c35e9: Pull complete
e9ae7d158f63: Pull complete
Digest: sha256:5b3afa19bdc674582c63b807c49d78fe409a405daec6aa6830af0ce6b3dde897
Status: Downloaded newer image for docker/docker-model-cli-desktop-module:v0.1.24
docker.io/docker/docker-model-cli-desktop-module:v0.1.24
90ea3747e08bdb68b7bfb222df922e33feac8cd057e9f63287ba95687af22a39
extractor
Untagged: docker/docker-model-cli-desktop-module:v0.1.24
Deleted: sha256:5b3afa19bdc674582c63b807c49d78fe409a405daec6aa6830af0ce6b3dde897
ajeetraina@ajeetraina-desktop:
DD_CPU_GPU_CV 2893mW/3079mW VDD_SOC 1645mW/1646mW
05-19-2025 18:19:04 RAM 3029/7620MB (lfb 1x4MB) SWAP 555/3810MB (cached 0MB) CPU [9%@1728,100%@1728,100%@1728,100%@1728,100%@1728,100%@1728] EMC_FREQ 1%@2133 GR3D_FREQ 0%@[305] NVDEC off NVJPG off NVJPG1 off VIC off OFA off APE 200 [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] VDD_IN 7650mW/7859mW VDD_CPU_GPU_CV 2854mW/3023mW VDD_SOC 1605mW/1636mW
05-19-2025 18:19:05 RAM 3029/7620MB (lfb 1x4MB) SWAP 555/3810MB (cached 0MB) CPU [13%@1728,100%@1728,100%@1728,100%@1728,100%@1728,100%@1728] EMC_FREQ 1%@2133 GR3D_FREQ 0%@[305] NVDEC off NVJPG off NVJPG1 off VIC off OFA off APE 200 [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] VDD_IN 7689mW/7825mW VDD_CPU_GPU_CV 2898mW/2998mW VDD_SOC 1647mW/1638mW
05-19-2025 18:19:06 RAM 3030/7620MB (lfb 1x4MB) SWAP 555/3810MB (cached 0MB) CPU [38%@1728,100%@1728,100%@1728,100%@1728,100%@1728,100%@1728] EMC_FREQ 1%@2133 GR3D_FREQ 0%@[305] NVDEC off NVJPG off NVJPG1 off VIC off OFA off APE 200 [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] VDD_IN 7767mW/7816mW VDD_CPU_GPU_CV 2976mW/2994mW VDD_SOC 1647mW/1640mW
05-19-2025 18:19:07 RAM 3030/7620MB (lfb 1x4MB) SWAP 555/3810MB (cached 0MB) CPU [18%@1728,100%@1728,100%@1728,100%@1728,100%@1728,100%@1728] EMC_FREQ 1%@2133 GR3D_FREQ 0%@[305] NVDEC off NVJPG off NVJPG1 off VIC off OFA off APE 200 [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] VDD_IN 7689mW/7797mW VDD_CPU_GPU_CV 2898mW/2980mW VDD_SOC 1647mW/1641mW
docker model run ai/llama3.2
Interactive chat mode started. Type '/bye' to exit.
Hello
Hello! It's nice to meet you. Is there something I can help you with or would you like to chat?
write a poem on docker and ai
In silicon halls of code and might,
A world of wonder, born of night,
Docker's veil, a container's guise,
A universe of worlds, in tiny sizes.
With AI's spark, the future's gleam,
A fusion born, of human dream,
The container's gate, now swings wide,
As artificial minds, do side by side.
The orchestrator, a conductor's hand,
Guides the machines, to a master plan,
Docker's bridges, connect the dots,
As AI's insights, the future plots.
The AI's eye, a lens to see,
The code's secrets, in a virtual sea,
Docker's tools, a navigator's aid,
To chart the course, through the digital trade.
The AI's brain, a cognitive core,
Analyzes data, and learns more,
Docker's layers, a puzzle to fit,
As AI's patterns, unlock the digital hit.
Note
These instructions should work for any Debian-derived distribution.
Configure the production repository:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list |
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' |
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
Optionally, configure the repository to use experimental packages:
sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list
Update the packages list from the repository:
sudo apt-get update
Install the NVIDIA Container Toolkit packages:
sudo apt-get install -y nvidia-container-toolkit
mkdir -p ~/.docker/cli-plugins && docker pull docker/docker-model-cli-desktop-module:v0.1.24 && docker create --name extractor docker/docker-model-cli-desktop-module:v0.1.24 nop && docker export extractor > _extract.tmp.tar && tar -xOf _extract.tmp.tar cli-plugins/model/linux/docker-model > ~/.docker/cli-plugins/docker-model && chmod +x
/.docker/cli-plugins/docker-model && rm _extract.tmp.tar && docker rm extractor && docker rmi docker/docker-model-cli-desktop-module:v0.1.24/Desktop$ docker model run ai/llama3.2v0.1.24: Pulling from docker/docker-model-cli-desktop-module
764efb7c35e9: Pull complete
e9ae7d158f63: Pull complete
Digest: sha256:5b3afa19bdc674582c63b807c49d78fe409a405daec6aa6830af0ce6b3dde897
Status: Downloaded newer image for docker/docker-model-cli-desktop-module:v0.1.24
docker.io/docker/docker-model-cli-desktop-module:v0.1.24
0cf5bae0a48687e6da49d67223d5943b56fbf84761d9202fc643d13ca5a8b103
extractor
Untagged: docker/docker-model-cli-desktop-module:v0.1.24
Deleted: sha256:5b3afa19bdc674582c63b807c49d78fe409a405daec6aa6830af0ce6b3dde897
ajeetraina@ajeetraina-desktop:
2f074dc76c5d: Downloading [=> ] 2f074dc76c5d: Downloading [=====> ] 2f074dc76c5d: Downloading [=======> ] 2f074dc76c5d: Downloading [==========> ] 2f074dc76c5d: Downloading [==============> ] 2f074dc76c5d: Downloading [==================> ] 2f074dc76c5d: Downloading [=====================> ] 2f074dc76c5d: Downloading [=========================> ] 2f074dc76c5d: Downloading [=============================> ] 2f074dc76c5d: Downloading [================================> ] 2f074dc76c5d: Downloading [====================================> ] 2f074dc76c5d: Downloading [=======================================> ] 2f074dc76c5d: Downloading [=========================================> ] 62e701f2cf14: Downloading [==================================================>] 2f074dc76c5d: Downloading [===========================================> ] 2f074dc76c5d: Downloading [===============================================> ] ac5862982139: Downloading [===========================> ] 2f074dc76c5d: Downloading [==================================================>] e6b767e32275: Downloading [===========================> ] 38c3841881c2: Downloading [================> ] 38c3841881c2: Downloading [==================================================>] e6b767e32275: Pull complete Digest: sha256:87459e7dba1a503c08379ba7605bd45a85d3d995a613cc8846be874b1a3ae163
Status: Downloaded newer image for docker/model-runner:latest
Successfully pulled docker/model-runner:latest
Creating model storage volume docker-model-runner-models...
Starting model runner container docker-model-runner...
Unable to find model 'ai/llama3.2' locally. Pulling from the server.
Downloaded: 0.00 MB
Model pulled successfully
Interactive chat mode started. Type '/bye' to exit.