Elly Belfort-Mattos elly-belfort
ntantri
/ s3_executor.py
Last active
December 3, 2023 18:09
An executor library which uses the select_runner.py to run queries on AWS S3 files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/usr/bin/python | |
| # -*- coding: utf-8 -*- | |
| import argparse | |
| import sys | |
| import boto3 | |
| from select_runner import * |
ntantri
/ select_runner.py
Last active
April 22, 2021 04:45
AWS S3 Select Runner - used to execute queries on S3 files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import boto3 | |
| s3 = boto3.client('s3') | |
| def write_to_file(r, output_file): | |
| """ | |
| This function would write the output to the file. |