TomAugspurger’s gists

TomAugspurger / io-test.py

Last active February 19, 2025 15:50

kvikio / moto slow benchmark

	"""
	Sample demonstrating slowness of cudf.read_parquet with kvikio from S3.


	# Instructions

	Download the dev wheels (skip if you have it built locally; rquires vpn access)

	curl -LO https://downloads.rapids.ai/ci/cudf/pull-request/17957/793d0b2/cudf_wheel_python_cudf_cu12_py312_x86_64.tar.gz
	curl -LO https://downloads.rapids.ai/ci/cudf/pull-request/17957/793d0b2/cudf_wheel_cpp_libcudf_cu12_x86_64.tar.gz

TomAugspurger / teams.sh

Last active March 10, 2025 15:10

	#!/usr/bin/env bash

	set -e

	PATH=/opt/homebrew/bin:$PATH

	# The exact UUID doesn't matter, but this should match what you select in the UI.
	UUID=edef356f-55b1-4132-b311-e5e8c10c6a33
	# This path is MacOS specific.
	OUTPUT=$HOME/Library/Containers/com.microsoft.teams2/Data/Library/Application\ Support/Microsoft/MSTeams/Backgrounds/Uploads

TomAugspurger / example.ipynb

Created January 31, 2025 20:46

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

TomAugspurger / test_zarr.py

Created January 31, 2025 20:39

	"""
	dali + Zarr (GPU) example.

	This script adapts the GPU example from
	https://docs.nvidia.com/deeplearning/dali/user-guide/docs/examples/general/data_loading/external_input.html
	to use Zarr for storage.

	To run it, you'll currently need to use my fork of zarr-python:

	pip install git+https://github.com/TomAugspurger/zarr-python/@tom/fix/gpu

TomAugspurger / gputest.py

Created January 28, 2025 12:12

	"""
	Test reading (and writing) Zarr data with host and GPU.

	$ python gputest.py gpu read
	gpu read 2.96
	$ python gputest.py cpu read
	cpu read 1.24
	"""

	import argparse

TomAugspurger / track.html

Created May 28, 2024 14:49


	<!DOCTYPE html>
	<html lang="en">

	<head>
	<meta charset="utf-8" />
	<meta http-equiv="X-UA-Compatible" content="IE=edge" />
	<meta name="viewport" content="width=device-width, initial-scale=1" />
	<title>memray - flamegraph report (memory leaks)</title>
	<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/css/bootstrap.min.css">

TomAugspurger / goes-ice.ipynb

Created November 27, 2023 18:23

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

TomAugspurger / ids.txt

Created October 23, 2023 15:36

Variable query times

TomAugspurger / item_collection.json

Created October 18, 2023 17:19

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

TomAugspurger / add_geoparquet.py

Created June 6, 2023 02:00

	import dask.dataframe as dd
	import adlfs
	import geopandas
	import dask_geopandas
	import pyarrow.fs
	import pyproj
	import json
	import dask.distributed
	from typing import Dict, Any, Union, Optional

Tom Augspurger TomAugspurger