Default Backdata Provided for new Subscription IDs

For all new IDs entered in the DMUI within the past day (created_ts is within the previous day in clarivoy.prod_third_party_data_config table) where vendor_name='cdk' and status=active:

• Gets active start date for the property from claripedia.property_active_dates table
  • If the active start date was < 31 days ago and > 90 days ago, the API will get data back to the start date. 
• If no active start date exists or the active start date > 90 days ago, the API will get data back to 30 days ago.

For more details, see CDK API: Daily DAG process.

Backdata Pull Process

This process is used if we did not get all of the backdata we need during Onboarding, or if we need to rerun certain dates after CDK fixes data issues on their end.

1. Find the dealer's subscription ID (API key).
   1. Use this Looker board: Third Party Data Config
   2. OR check the DMUI Property setup → 3rd Party page → CDK
2. Enter the request on this Google Sheet:  manual_cdk_run
   1. Enter Subscription ID in the SubscriptionID column
   2. Enter the StartDate and EndDate using format YYYY-MM-DD.  Make your StartDate about 10-15 days prior to what you actually need to ensure we get all data needed (see explanation below).
      

      About startDate and endDate

      Per CDK: startDate/endDate both look at ContractDate AND DEALEVENT3DATE.  DEALEVENT3DATE is the date when the deal is finalized (which in the API payload response = dealevent “F”).

      This is DIFFERENT from the date we use for sales_date which is deal.accountingDate, so EXPAND YOUR SEARCH RANGE by about 15 days earlier than you actually need to account for deals with a ContractDate prior to their accounting dept finalizing the sale.

       
   3. Enter ApiPoint which should be:
      1. history = For history, MAX 6 MONTHS at a time. EndDate must be 32 days ago or earlier.
         Split into separate queries if you need more than 6 months.
         1. EndDate must be 32 days ago or earlier.  
         2. StartDate must be 6 months prior to EndDate or sooner. 
      2. closed = For closed, MAX 31 days back from today (StartDate must be 31 days ago or sooner).
         1. StartDate can be up to 31 days ago.
         2. EndDate can be today.
3. Run the Airflow DAG cdk_manual_daterange_new
4. Check the server log file cdk_manual_daterange_new.log to verify the query was successful.
   Errors are broadcast in Slack channel #cdk-api-dag-alerts.
5. Go back to the Google Sheet and clear out the subscription ID that you just ran, so we know it was completed (and is not still outstanding)
6. If this is for an Onboarding task, set the task status to Waiting on Data.
   1. 🚀 RL automations will set the task field Verify Data date to 2 days later.
   2. 🚀 2 days later, the task status will flip to Verify Data