Naufal Suryanto naufalso

PhD | AI Researcher

naufalso / get_memory_size.py

Created January 19, 2025 03:39 — forked from philschmid/get_memory_size.py

Get needed GPU per precision for a Hugging Face Model Id

	from typing import Dict, Union
	from huggingface_hub import get_safetensors_metadata
	import argparse
	import sys

	# Example:
	# python get_gpu_memory.py Qwen/Qwen2.5-7B-Instruct

	# Dictionary mapping dtype strings to their byte sizes
	bytes_per_dtype: Dict[str, float] = {