Last active
January 27, 2021 11:12
-
-
Save 0xdabbad00/f8c0054f3fb4eeb9a3e95a01cdca2c36 to your computer and use it in GitHub Desktop.
Walking botocore
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Clone botocore and run `git rev-list --all > commit_list.txt` to get a chrological list of the hashes. | |
Then I manually installed https://github.com/nok/git-walk but had to modify it to use my list instead of generating it's own each time because it was going into a loop. | |
So `read_commit_ids` looks like this: | |
``` | |
def read_commit_ids(): | |
#cmd = 'git rev-list --all' | |
#log = subp.check_output(cmd.split()).strip() | |
#log = [line.strip() for line in log.split('\n')] | |
with open("commit_list.txt") as f: | |
log = f.read().splitlines() | |
return log | |
``` | |
Then create `get_count.sh` with (for counting actions): | |
``` | |
#!/bin/bash | |
# This uses https://github.com/nok/git-walk installed manually | |
until [ $(git show -s --format=%H) = "463d8b7836e4db1c62c6b3bbff1a5e25dffe6f78" ]; do | |
echo "`git show -s --format=%ci`,`cat botocore/data/*/*/service*.json | jq '.metadata.endpointPrefix as $service|.operations|keys[]|$service+":"+.' 2>/dev/null | sort | uniq | wc -l | sed 's/ *//'`" | |
git walk prev | |
done | |
``` | |
For counting services I used: | |
``` | |
#!/bin/bash | |
# This uses https://github.com/nok/git-walk installed manually | |
until [ $(git show -s --format=%H) = "463d8b7836e4db1c62c6b3bbff1a5e25dffe6f78" ]; do | |
echo "`git show -s --format=%ci`,`ls -d botocore/data/*/ | wc -l | sed 's/ *//'`" | |
git walk prev | |
done | |
``` | |
Run this and output to tmp file. Then I opened this in Visual Code and did a search and replace on the date to get it in a format with just '2018-01-01'. | |
Then I created a file `max_by_month.py` with: | |
``` | |
#!/usr/bin/env python | |
with open("counts.csv") as f: | |
lines = f.read().splitlines() | |
dates = [] | |
counts = [] | |
month = '' | |
month_max = 0 | |
for l in lines: | |
parts = l.split(',') | |
date_parts = parts[0].split('-') | |
line_month = date_parts[0]+'-'+date_parts[1] | |
if line_month != month: | |
print('{}-01,{}'.format(month, month_max)) | |
month_max = 0 | |
month = line_month | |
if month_max < parts[1]: | |
month_max = parts[1] | |
``` | |
Then opened in excel and created the chart. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment