Once the Processing Status on the SA Processing board hits “CSL QA” or “QA Issue” it’s the CSL’s turn to check that labels are correct and that any customer requests from the last month (such as new classifications or labels) were addressed.

Step 1: Source Classification

1. Run the unclassified report and go through it and make sure all your properties are classified correctly. Note: you must have this file uploaded by 3 pm est daily. https://www.dropbox.com/home/Insyncing%20Data%20Drop/Source%20Classification/Classify%20These%20Files

Step 2: Call Tracking Classification

1. Run the unclassified report and go through it and make sure all your properties are classified correctly. Note: you must have this file uploaded by 3 pm est daily. https://www.dropbox.com/home/Insyncing%20Data%20Drop/Call%20Tracking%20Classification/Classify%20These%20Files

Step 3:  Check that All Leads were Loaded

1. Go to the Leads QA 2 board in Looker
2. Select client and property name
3. Set dates to last 3 complete months
4. Look at the counts on the Monthly Leads Report (with Property) and compare the latest month to prior months. 

What are you looking for?

• Confirm that all channels are populated
• Confirm that numbers are consistent with prior month
• Are any Lead Providers missing or added?
• Did the counts increase or decrease dramatically? Note that some degree of variation is normal, especially since it will usually track with sales increases or decreases.

If there are issues:

• Tag Ops and/or CSL in the item's conversation updates

Step 4: Attribution Overview board in Looker

1. Go to the Attribution Overview looker board
2. Select client and property name
3. Set dates to last 3 complete months
4. If looking at multiple properties, sort Monthly Property Report by Sales Month first, then Property Name
5. Look at matching levels for the latest month and compare to prior months:
   1. Review Attribution % (don’t look at Sum of Total Attributed Sales - raw #) 
   2. Review Offline Attribution %  [CRM, Chat, CT, Email Opens, Direct Mail]
      1. If problem, we may have loaded files wrong, mapped incorrectly, etc. - Look at Leads QA2 for the property - make sure the # of leads are in line
   3. Offline Matching less than 35% is a flag.
      1. Note in the conversation updates the reason for stopping. Note any other anomalies (such as low lead counts, low or high sales counts, etc.)
      2. Follow steps for investigation as outlined in Process for Investigating Low Offline Attribution
   4. Review Online Attribution %   [LFI (Clarivoy Leads), Lat/Long, Lat/Long 3rd Party]
      1. If problem, open a ticket for Engineering
   5. Review LiveRamp Attribution %

What are you looking for?

• Review the Monthly Property Report module and check across all columns for any major deltas (which may lead to issues)
• Look over data volume by Lead supplier/type for major changes (which may indicate issues with ingestion)

If there are issues:

• Tag Ops and/or CSL in the item's conversation updates

Step 5: Review Client data in Stage Dashboard

1. Attribution Overview
   1.  Look at how channels and sources are trending
2. Executive Summary
   1. Check the Executive Summary and make sure the data is correct.
3. Sales Overview
   1. Check Sales Overview make sure you have no unclassified sales.
4. KPI over Time
   1. Go to the dashboard, “KPI Over Time” page
   2. Select and deselect channels to look at how they change over time
   3. Click to sources and see how they are trending
5. Campaigns
   1. Go to the dashboard, “Campaigns” page
   2. Look through the “Identified Campaign” list and note how each campaign has been classified into Channel and Source
   3. What are you looking for?
      1. Do the campaigns look like they’re associated with the correct sources and channels? Sometimes this level of data uncovers classification issues that were not evident at higher levels. For example, a campaign label that says “CarGurus SEM” may be classified as Organic Search. 
   4. If there are issues:
      1. The approach to fix classification issues depends on what type of data it is. First you need to determine the type of data. Call Tracking data can be easily recognized because the campaign label will always include the phone number. If it’s not Call Tracking, then make a Manuscript ticket for the ETL team to find out if it’s CRM data or web traffic.
      2. Once you know what the data type is, you can look at the raw data to see if the increase is valid. CRM attribution and call tracking attribution can be checked in the respective raw files. Web traffic attribution can be checked in Google Analytics (if we have access) and the Web Traffic report to see tagging.
      3. If you find that a source is mis-labeled, follow the steps in the Taxonomy chapter to address.
6. ROI
   1. Look over the ROI page 
   2. Check for data splits for source consolidation errors - possible issues: 
   3. two sources for the same thing in one channel >> ticket for source consolidation 
   4. two sources for the same thing in different channels>>  this issue must be discovered: check source and call tracking - if no issues are reconcilable there, then check the taxonomy data and if the issue is web taxonomy >> create taxonomy ticket on the Support > Tickets Monday board 
      
       
   5. What are you looking for?
      1. Are the channels trending with the overall sales trend?
      2. Did any channels increase or decrease significantly from last month?
      3. Within channels, did any sources increase or decrease significantly from last month?
      4. If so, do you already know why? (e.g., client canceled direct mail)
   6. If there are issues:
      1. Click to the Source level:  If you see large changes, especially if they don’t track with other channels, click to the Source view to see which source is driving the change. Does that source belong there? Could that source be mislabeled? Sometimes this indicates that a customer has re-assigned a call tracking number but did not update the label. For example, a number labeled as “Facebook” might suddenly be re-used on the customer’s main home page, so calls will spike and are unrelated to Facebook. If you don’t recognize why the source has increased or decreased:
         1. Make a ticket for the ETL team to find out where the attribution increase is coming from. Is it CRM, call tracking, web traffic?
         2. Once you know what the data type is, you can look at the raw data to see if the increase is valid. CRM attribution and call tracking attribution can be checked in the respective raw files. Web traffic attribution can be checked in Google Analytics (if we have access) and the Web Traffic report to see tagging.
         3. If you find that a source is mis-labeled, follow the steps in the Taxonomy & Data Classification Process Document to address.
      2. Make sure you look at the channel marked “Other” to see what sources are classified there. This is often where you will find attribution for sources where the name was vague (for example, “Google,” which could be organic, paid or even display), which can then be clarified with the customer when they see the data. But sometimes you will find sources that are clear enough that they should be re-assigned to other channels.
      3. Check attribution for other rooftops in the group:  Since for dealership groups we run matching holistically across the group, you may find attribution showing up in one rooftop for a vendor that this rooftop doesn’t use. For example:
         1. The customer may be working with Autotrader for only one store, and an Autotrader lead submitted to that store ended up buying from another store. So there would be attribution to Autotrader at the second store even though they’re not using Autotrader.
         2. The customer recently doubled their paid search spend for one store (or even the parent domain) and this resulted in an increase in paid search attribution for other stores as well.
         3. You can see the nature and degree of this cross-family attribution by looking at the “Sales Overview” page and using the “Touchpoints” part of the Filter tab on the left side. By toggling the store’s domain off, you can see the sales that were influenced by other properties. Note: third party auto vendors will always be listed separately in this Touchpoint list, regardless of whether they provided a lead to this store or elsewhere.
            
            
            In the screenshot above, the domain for this dashboard (BaxterSubaru.com) has been toggled off in the filter, so all the attribution showing here is for Subaru sales influenced by other Baxter stores. Direct Mail, for example, shows up here even though the Subaru store does not use Direct Mail.

Step 6: CSL QA Complete

Once you complete QA, determine whether the data is ready for the client to see. 

• If everything is fine but matching is not yet complete, we usually want to hold it because the data may look low and be unsatisfying to the customer. Change the status to Waiting on Classifications if it needs more time for LR/NR” and hold it there. Make sure to check it every day so that when matching is completed.
• If you find errors in QA that are just misclassifications, we usually want to share it with the client anyway, while we work in parallel on the relabeling. If it’s large enough that the client will notice, you may want to tell the client proactively as a known issue that you’re fixing. Most of the time the issues are small, worth fixing but not worth mentioning to the client. By all means, though, use your judgment. If you find a source has been mislabeled and it’s creating a dramatic spike and it’s quickly fixed in either CRM labeling or Call Tracking, then you may want to fix it before pushing the data to Prod. Basically, ask yourself “if the client sees the dashboard as is, will it give them a reason to doubt Clarivoy’s competence or accuracy?” If the answer is yes, then hold the data until it’s fixed. Change the status to “QA Issue,, See Comments for QA issues and any updates.

The process of pushing data from Stage to Prod can take as long as 48 hours. Once it’s finished, the CS Specialist will change the status in Monday to Approved for Prod. The Ops Team will then push the client data into Prod and change the status to Prod In Progress. After verifying Prod has successfully run, Ops Team will change the status to Verify Prod.