Kevin Yang KevinYang21
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #### 2018-03-05: | |
| #### 2018-03-06: | |
| #### 2018-03-07: | |
| #### 2018-03-08: | |
| #### 2018-03-09: |
KevinYang21
/ owner_run_as_user_dag_id.task_id
Created
August 24, 2018 23:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| amit_pahwa svc_di_data_eng minerva_compute.wf__gold__jitney_events__daily_all__namespace__logging_universal | |
| amit_pahwa svc_di_data_eng minerva_compute.wf__gold__jitney_events__daily_all__namespace__logging_search | |
| amit_pahwa svc_di_data_eng minerva_compute.wf__gold__jitney_events__daily_all__namespace__logging_performance | |
| pai_liu svc_di_trip_platform trip_platform_metrics.wf_jitney_daily_all_logging_itinerary | |
| pai_liu svc_di_trip_platform trip_platform_metrics.wf_jitney_daily_all_logging_systems | |
| pai_liu svc_di_trip_platform trip_platform_metrics.wf_jitney_daily_all_logging_performance | |
| xirui_liu svc_di_china_and_india china_story.jitneyevents__logging_saved | |
| teng_wang svc_di_growth pg_remarketing_events.wf_jitney_events_daily_all | |
| aaron_keys svc_di_data_infra airolap_batch_general.wf_jitney_events_daily_all | |
| shawn_chen svc_di_growth pg_remarketing_dashboard.wf_jitney_events_daily_all |
KevinYang21
/ list_of_corrupted_success
Last active
August 24, 2018 21:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| core_payments ps__gold_deposits_mobile_daily_all 2018-08-24 00:00:23 2018-08-24 05:54:56 | |
| core_payments ps__gold_payments_web_daily_all 2018-08-24 00:00:23 2018-08-24 05:55:04 | |
| core_payments ps__gold_grouptravel_daily_all 2018-08-24 00:00:23 2018-08-24 05:55:07 | |
| select wf_jitney_events_daily_all_namespace_logging_selectsupply 2018-08-24 00:01:01 2018-08-24 05:55:20 | |
| host_paid_growth_data_integrity wf_jitney_logging_hostgrowthconversionevent 2018-08-24 00:01:13 2018-08-24 05:55:30 | |
| china_tns jitney_events_daily_all 2018-08-24 00:01:26 2018-08-24 05:55:43 | |
| china_story jitneyevents__logging_saved 2018-08-24 00:01:36 2018-08-24 05:55:53 | |
| core_context wf_jitney_daily_all_webpayments 2018-08-24 00:01:29 2018-08-24 05:56:03 | |
| core_context wf_jitney_daily_all_payments 2018-08-24 00:01:29 2018-08-24 05:56:18 | |
| visitors_tool wf_jitney_ns_rookery 2018-08-24 00:01:59 2018-08-24 05:56:46 |
KevinYang21
/ dag_ids
Last active
August 24, 2018 19:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| svc_di_data_eng minerva_compute.wf__gold__jitney_events__daily_all__namespace__logging_universal | |
| svc_di_data_eng minerva_compute.wf__gold__jitney_events__daily_all__namespace__logging_search | |
| svc_di_data_eng minerva_compute.wf__gold__jitney_events__daily_all__namespace__logging_performance | |
| svc_di_trip_platform trip_platform_metrics.wf_jitney_daily_all_logging_itinerary | |
| svc_di_trip_platform trip_platform_metrics.wf_jitney_daily_all_logging_systems | |
| svc_di_trip_platform trip_platform_metrics.wf_jitney_daily_all_logging_performance | |
| svc_di_china_and_india china_story.jitneyevents__logging_saved | |
| svc_di_growth pg_remarketing_events.wf_jitney_events_daily_all | |
| svc_di_data_infra airolap_batch_general.wf_jitney_events_daily_all | |
| svc_di_growth pg_remarketing_dashboard.wf_jitney_events_daily_all |
KevinYang21
/ clear
Last active
August 24, 2018 08:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| airflow clear -t wf__gold__jitney_events__daily_all__namespace__logging_universal -s 2018-08-23 -e 2018-08-23 -sd /home/kevin_yang/data/airflow/minerva_pipelines/minerva_pipelines.py -d -c minerva_compute;airflow clear -t wf__gold__jitney_events__daily_all__namespace__logging_search -s 2018-08-23 -e 2018-08-23 -sd /home/kevin_yang/data/airflow/minerva_pipelines/minerva_pipelines.py -d -c minerva_compute;airflow clear -t wf__gold__jitney_events__daily_all__namespace__logging_performance -s 2018-08-23 -e 2018-08-23 -sd /home/kevin_yang/data/airflow/minerva_pipelines/minerva_pipelines.py -d -c minerva_compute;airflow clear -t wf_jitney_daily_all_logging_itinerary -s 2018-08-23 -e 2018-08-23 -sd /home/kevin_yang/data/airflow/trip_platform/trip_platform_metrics.py -d -c trip_platform_metrics;airflow clear -t wf_jitney_daily_all_logging_systems -s 2018-08-23 -e 2018-08-23 -sd /home/kevin_yang/data/airflow/trip_platform/trip_platform_metrics.py -d -c trip_platform_metrics;airflow clear -t wf_jitney_daily_all_logging |
KevinYang21
/ list
Created
August 24, 2018 07:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| minerva_compute.wf__gold__jitney_events__daily_all__namespace__logging_universal partition_name: jitney_events.daily_all/ds={{ ds }}/namespace=logging_universal | |
| minerva_compute.wf__gold__jitney_events__daily_all__namespace__logging_search partition_name: jitney_events.daily_all/namespace=logging_search/ds={{ ds }} | |
| minerva_compute.wf__gold__jitney_events__daily_all__namespace__logging_performance partition_name: jitney_events.daily_all/ds={{ ds }}/namespace=logging_performance | |
| trip_platform_metrics.wf_jitney_daily_all_logging_itinerary partition_name: jitney_events.daily_all/namespace=logging_itinerary/ds={{ ds }} | |
| trip_platform_metrics.wf_jitney_daily_all_logging_systems partition_name: jitney_events.daily_all/namespace=logging_systems/ds={{ ds }} | |
| trip_platform_metrics.wf_jitney_daily_all_logging_performance partition_name: jitney_events.daily_all/namespace=logging_performance/ds={{ ds }} | |
| china_story.jitneyevents__logging_saved partition_name: jitney_events.daily_all/ds={{ds}}/namespace=logging_saved | |
| pg_remark |
KevinYang21
/ print_path.py
Created
August 16, 2018 00:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # you may not use this file except in compliance with the License. | |
| # You may obtain a copy of the License at | |
| # | |
| # http://www.apache.org/licenses/LICENSE-2.0 | |
| # | |
| # Unless required by applicable law or agreed to in writing, software | |
| # distributed under the License is distributed on an "AS IS" BASIS, | |
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| # See the License for the specific language governing permissions and | |
| # limitations under the License. |
KevinYang21
/ template.sh
Last active
July 15, 2018 07:43
Template command to sync old devbox and new devbox
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # To use the template, replace <instance_id> with the real devbox instance ids and run them on your local machine. | |
| USERNAME=`whoami` # your SSH username | |
| NEW_DEVBOX=<instance_id>.inst.aws.us-east-1.prod.musta.ch # hostname of the new devbox | |
| OLD_DEVBOX=<instance_id>.inst.aws.us-east-1.prod.musta.ch # hostname of the old devbox | |
| ssh -A $NEW_DEVBOX "sudo SSH_AUTH_SOCK="\$"SSH_AUTH_SOCK rsync -avzhe 'ssh -o StrictHostKeyChecking=no' —rsync-path='sudo rsync' $USERNAME@$OLD_DEVBOX:/home/$USERNAME/ /home/$USERNAME/" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # -*- coding: utf-8 -*- | |
| # | |
| # Licensed to the Apache Software Foundation (ASF) under one | |
| # or more contributor license agreements. See the NOTICE file | |
| # distributed with this work for additional information | |
| # regarding copyright ownership. The ASF licenses this file | |
| # to you under the Apache License, Version 2.0 (the | |
| # "License"); you may not use this file except in compliance | |
| # with the License. You may obtain a copy of the License at | |
| # |