General Use

• On the left side, you can see the DAGs that are On or Off. You can filter at the top using:
• Click on Schedule to see times a DAG has run
• DAGs that are On but have no schedule (“None”) are manually run

To trigger a DAG to run:

1. Click play sign under the Actions column
   
   NOTE: If a schedule exists and a DAG was turned off/disabled, running the DAG once will reenable the schedule!

To view a DAG log:

1. Click on DAG name
2. Click on code name
3. Click Log

Specific DAGs

Push to Prod

Push to Prod is now an automated process as of November 2023. For details on the automated process and timing, see SA+TVA Ops Overview Monthly Processing & QA Issues process document: Push to PROD: New Automated Processes 11/20/23 section.

Manual Push to Prod Process

For all entries in the sales.production_ready table, this DAG will ensure their data is visible in PROD Dashboard (https://dashboard.clarivoy.com)

1. Locate daily_sa_refresh_prod_dashboard
2. Click play sign under the Actions column
   
3. Click Trigger
4. Watch #salesattribution channel in Slack. Look for:
   
5. If this DAG ever fails, escalate to Engineering

Sales and Leads Processing

To manually force a file to fully process outside of the normal schedule,

1. Run either:
   etl_leads_preprocessing_processing
   etl_sales_preprocessing_processing
   1. Search for DAG name in Search box
   2. Click play sign under Links column
   3. Click Trigger
2. Errors related to Sales and Leads file processing will be broadcast in the #sales-leads-processing Slack channel. You must watch this channel and fix any errors that are broadcast as files will not be processed if they error here; The file in question will stay "stuck" in Insyncing Data Drop until the error is fixed.

File Reprocessing

If we get an error in #sales-leads-processing that a file did not process, this means that the file did not process properly.

1. Correct the file error (Can use the same name)
2. Rerun either of these 2 preprocessing dags:
   etl_leads_preprocessing_processing
   etl_sales_preprocessing_processing
3. There will be no alerts if the run is successful. To verify successful run, look for the file in IDD to have moved to Data Archive.
4. File will then be picked up by the processing DAG the next time it is scheduled to run.

Checking Error Logs

Errors related to Sales and Leads file processing will be broadcast in the #sales-leads-processing Slack channel. You must watch this channel and fix any errors that are broadcast as files will not be processed if they error here; The file in question will stay "stuck" in Insyncing Data Drop until the error is fixed.

FOR troubleshooting #sales-leads-processing errors, see the Troubleshooting #sales-leads-processing Errors section in the SA+TVA Monthly Tasks Processing process document.

See Viewing Log Files on the Jobs Server

Log files are:

• etl_sales_preprocessing.log
• etl_leads_preprocessing.log