zengqingfu1442

Setup

On every machine in the cluster install openmpi and mlx-lm:

conda install conda-forge::openmpi
pip install -U mlx-lm

Next download the pipeline parallel run script. Download it to the same path on every machine:

zengqingfu1442

Harvester GPU Provisioning

0. Install Harvester, then SSH into the server.

1. Edit /boot/grub/grub.cfg as follows:

 set default=0
 set timeout=10
 

zengqingfu1442

Harvester SNAT Problem

Problem Description

Harvester leverages the Kubernetes service to provide the load balancer for the service in the Harvester virtual machines. The backend servers of the load balancer are the <VM IP>:<service port>.

cw-harv:/home/rancher # kubectl get svc
NAME                        TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)        AGE
default-nginx-lb-db9bdca5 LoadBalancer 10.43.113.238 80:32586/TCP 4h32m

zengqingfu1442

import collections
import uuid
from typing import Dict

import numpy as np
import tritonclient
import tritonclient.grpc as grpcclient
import tritonclient.utils.shared_memory as shm

ShmHandle = collections.namedtuple(

zengqingfu1442

Docker Hub 镜像加速器

国内从 Docker Hub 拉取镜像有时会遇到困难，此时可以配置镜像加速器。Docker 官方和国内很多云服务商都提供了国内加速器服务。

Dockerized 实践 https://github.com/y0ngb1n/dockerized

配置加速地址

Ubuntu 16.04+、Debian 8+、CentOS 7+

zengqingfu1442

FROM ubuntu as ubuntu

RUN apt-get update
RUN apt-get install -y dpkg-dev wget gnupg2 curl

ARG ubuntu_packages="wget htop default-jre-headless apt-transport-https nvidia-container-toolkit cuda-drivers libopengl0 linux-image-extra-virtual omnisci"

WORKDIR /opt/packages/deb

# Nvidia-Docker

zengqingfu1442

# Run this in the project repo from the command-line
# http://stackoverflow.com/a/4593065/99923

git log --shortstat --author "Xeoncross" --since "2 weeks ago" --until "1 week ago" | grep "files changed" | awk '{files+=$1; inserted+=$4; deleted+=$6} END {print "files changed", files, "lines inserted:", inserted, "lines deleted:", deleted}'

zengqingfu1442

Project:

• project/: A directory named with the project's name which stores the actual Python package
  • __init__py
  • app.py
  • settings.py
  • urls.py
  • models/
    • __init__.py
    • baes.py
• handlers/

zengqingfu1442

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import sys, time, threading, abc
from optparse import OptionParser

def parse_options():
    parser = OptionParser()
    parser.add_option("-t", action="store", type="int", dest="threadNum", default=1,
                      help="thread count [1]")

zengqingfu1442

git config --global https.proxy http://127.0.0.1:1080

git config --global https.proxy https://127.0.0.1:1080

git config --global --unset http.proxy

git config --global --unset https.proxy


npm config delete proxy