A quick script to measure the speed of all your downloaded models in Ollama.

ollama_speed_test.py

import ollama
import csv

PROMPT = """
Read the following text and rephrase it in a more contemporary and casual manner:

The different accidents of life are not so changeable as the feelings
of human nature. I had worked hard for nearly two years, for the sole
purpose of infusing life into an inanimate body. For this I had
deprived myself of rest and health. I had desired it with an ardour
that far exceeded moderation; but now that I had finished, the beauty
of the dream vanished, and breathless horror and disgust filled my
heart. Unable to endure the aspect of the being I had created, I
rushed out of the room and continued a long time traversing my
bed-chamber, unable to compose my mind to sleep. At length lassitude
succeeded to the tumult I had before endured, and I threw myself on the
bed in my clothes, endeavouring to seek a few moments of forgetfulness.
But it was in vain; I slept, indeed, but I was disturbed by the wildest
dreams. I thought I saw Elizabeth, in the bloom of health, walking in
the streets of Ingolstadt. Delighted and surprised, I embraced her,
but as I imprinted the first kiss on her lips, they became livid with
the hue of death; her features appeared to change, and I thought that I
held the corpse of my dead mother in my arms; a shroud enveloped her
form, and I saw the grave-worms crawling in the folds of the flannel.
I started from my sleep with horror; a cold dew covered my forehead, my
teeth chattered, and every limb became convulsed; when, by the dim and
yellow light of the moon, as it forced its way through the window
shutters, I beheld the wretch—the miserable monster whom I had
created. He held up the curtain of the bed; and his eyes, if eyes they
may be called, were fixed on me. His jaws opened, and he muttered some
inarticulate sounds, while a grin wrinkled his cheeks. He might have
spoken, but I did not hear; one hand was stretched out, seemingly to
detain me, but I escaped and rushed downstairs. I took refuge in the
courtyard belonging to the house which I inhabited, where I remained
during the rest of the night, walking up and down in the greatest
agitation, listening attentively, catching and fearing each sound as if
it were to announce the approach of the demoniacal corpse to which I
had so miserably given life.
"""

with open('speed_test_results.csv', mode='w', newline='') as csvfile:
    csv_writer = csv.writer(csvfile)
    csv_writer.writerow(['Model', 'Family', 'Parameter Size', 'Quantization Level', 'Total Duration (s)', 'Load Duration (s)', 'Prompt Eval Count', 'Prompt Eval Duration (s)', 'Eval Count', 'Eval Duration (s)', 'Tokens per Second'])

    for model in ollama.list().models:
        model_name = model.model
        family = model.details.family
        parameter_size = model.details.parameter_size
        quantization_level = model.details.quantization_level

        # Skip embedding models
        if 'embed' in model_name.lower():
            continue

        print(model_name)

        # Generate response
        response = ollama.generate(
            model=model.model,
            prompt=PROMPT,
        )

        total_duration = response.total_duration / 1e9 # Convert nanoseconds to seconds
        load_duration = response.load_duration / 1e9 

        prompt_eval_count = response.prompt_eval_count
        prompt_eval_duration = response.prompt_eval_duration / 1e9
        eval_count = response.eval_count
        eval_duration = response.eval_duration / 1e9 

        tokens_per_second = (prompt_eval_count + eval_count) / (prompt_eval_duration + eval_duration)

        csv_writer.writerow([
            model_name,
            family,
            parameter_size,
            quantization_level,
            total_duration,
            load_duration,
            prompt_eval_count,
            prompt_eval_duration,
            eval_count,
            eval_duration,
            tokens_per_second
        ])

        print(response.response)
        print("- - - - -\n\n")