Marius Ghita mhitza

See geerlingguy/ollama-benchmark#21 (comment)

Llama 3.2 3b

command and output

$ wget https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct-Q4_K_M.gguf
  
$ ~/llama.cpp/source/build/bin/llama-bench -m ~/models/Llama-3.2-3B-Instruct-Q4_K_M.gguf -n 128 -p 512,4096 -pg 4096,128 -ngl 99 -r 2

As the Deck is mainly marketed as a portable gaming device, there are a few kinks with the desktop experience. In this living blog post I'm collecting the notes, scripts, and steps I've went through to replicate my existing workflows on the Deck Desktop.

SteamOS is the first immutable distro I'm using, which makes things a bit more challenging, after using Fedora - non-GNOME spins - for more than 14 years where I've become accustomed to the distribution packages, and patched up the system to what I wanted by sudo viming across the filesystem.

In the last couple of days I've typed away at some Ansible automation, and one specific group of tasks I dealt with was the creation of a target symlink and parent directories. I wasn't able to do this in one go, and thus arrived at the following result:

- name: "absolute home {{ config_path }}"
  set_fact:
    absolute_home_config_path: "{{ user_home_dir + '/.' + config_path }}"

- name: "{{ absolute_home_config_path|dirname }} present"

chmod +x strip-js-comments.mjs
npm install --save @babel/generator
npm install --save @babel/parser
./strip-js-comments.mjs assets/js/script.js

Q2 is closing in for the first release of [Rocky Linux][1], but until that rolls around I think it might be a good time to give CentOS Stream a try.

While I'm probably one of those users who will jump ship once Rocky Linux hits the shelves, so to speak, 2021 might not necessarily be the moment I make the switch. I'm not going to rant about RedHat's current track record; I will say that I somehow managed to trip on a couple of differences between RedHat 8 and CentOS 8. Which should even exist in the first place. Hopefully, the same issues aren't going to resurface with Rocky Linux. With the amount of Ansible roles I depend on daily, I'll wait things out a bit to stabilize first.

This document was based on my local [GNU Guile][1]-3.0.5 setup. I'm not sure if it works as is with an older version of GNU Guile.

Because of a [dependency in Fedora][2] I had to compile GNU Guile from source release. As such, in your local setup the paths will differ. This is only relevant when defining the GUILE shell variable, and referencing the tags file in the vimrc.

Note that when building GNU Guile from source be sure that you have the readline-devel (or distro equivalent package) installed. That way the ./configure step will pick that up, and the ice-9 readline module will be usable.

	{-# LANGUAGE BlockArguments, Strict, LambdaCase, NoMonomorphismRestriction, MultiWayIf #-}
	import Control.Monad.State.Strict
	import Data.List
	import Data.Char
	import Data.Ord hiding (Down)
	import Data.Bifunctor
	import Data.Function
	import Data.Functor.Identity
	import Debug.Trace

	{-# LANGUAGE BlockArguments, BangPatterns, LambdaCase, NoMonomorphismRestriction #-}
	import Control.Monad.State.Strict
	import Data.List
	import Data.Bifunctor
	import Data.Maybe

	forEach xs st f = snd $ foldM (\st x -> runState (f x) st) st xs
	alterF = modify . first
	alterS = modify . second
	getF = gets fst

	{-# LANGUAGE BlockArguments, Strict #-}
	import Control.Monad.State
	import Data.List
	import Data.Char
	import Data.Bifunctor

	takeUntilIncluding p [] = []
	takeUntilIncluding p (x:xs)
	\| p x == True = x : takeUntilIncluding p xs
	\| otherwise = [x]

	{-# LANGUAGE BlockArguments, Strict #-}

	import Control.Monad.State
	import Data.List
	import Data.Bifunctor (first)
	import Data.Char

	forEach xs state' f = foldM (\st v -> runState (f v) st) state' xs