Popular topics: Web Traffic Tracking GuideCDK: Setting Up DMS Sales Reports

Data Integration: Scraper Method

Table of Contents

Links to Other Wall-E Process Documents Scraper Setup Process Request Login Login Setup Customer Vendor Logins Processes Verify Permissions Wall-E Setup Combined Data for Multiple Properties Verify Data

Links to Other Wall-E Process Documents

 

Scraper Setup Process

Request Login

:bust_in_silhouette: CSL or CS Specialist

If dealer selects Scraper method, we need to collect a login and setup the login. Collecting the login is part of Onboarding.

The Scraper method is currently only offered for dealers using DealerSocket, DriveCentric, ELEAD, VinSolutions CRM (does not include VS CT), or Dealer.com Call Tracking. If the dealer uses another CRM and selects Scraper, CS Specialist will alert them through GuideCX that they must instead upload a file.

 

 

  1. Dealer receives email requesting login information, with permissions specified based on which CRM they have
  2. Dealer provides login by responding in Rocketlane task conversation.
  3. CSL - If the dealer provides the login to you in an email, post the login credentials in the task comments.
  4. CSL or CS Specialist - Once login is received in the task comments, set task status to Setup Needed.
    1. 🚀 Rocketlane automations will reassign the task to Ops who will pick it up from here.

 

Login Setup

Customer Vendor Logins Processes

Saving Customer Vendor Logins

⚠️ Always use Google Chrome with the Keeper browser extension

 

Whenever we receive a vendor login from a dealership, it should be set up and saved in Keeper vault in the shared folder Customer Logins.  Access to this folder is strictly limited to the Ops Team (DataOps Team in Keeper) who performs login setups, password changes, report setups, and monthly data downloads.

:bust_in_silhouette: Ops Onboarding Specialist

  1. Log in using the temp password provided, or for some vendors, using the activation link that was emailed to datateam.
    1. If there are login issues, tag the dealer in the task chat explaining and asking for their help.  Add the dealer back as a task Assignee and set the task status back to In Progress.
  2. Save the login in Keeper in the shared folder Customer Logins. 
    1. In Keeper, in the left navbar, click the + icon
    2. Create New Record
      1. Location = Customer Logins (shared folder)
      2. Record Type = Login
      3. Title = Use the format Vendor Name - Dealer or Group Name (the same format as the SA + TVA Ops Monthly Tasks board).
        Example: DealerSocket - ABC Motors
      4. Click Next
  3. Keeper Login Details Tab
    1. Password = click the Dice icon to generate a new password.  Login and change the password to this new secure password.
    2. Website Address = Make sure you update the URL to the login URL for the specific vendor.  Often times, Keeper will populate the “new account setup URL” and not the “login URL”.  
      Example:  When setting up a CDK CRM (Elead) login, Keeper may autopopulate the URL as:  https://retail.cdk.com/welcome
      You need to change it to the actual CDK login URL as found on the Vendor Details board or in the Rocketlane task notes:  https://retail.cdk.com
    3. Setup Security Questions: If the login requires you to setup security questions and answers, edit the Keeper entry and place the Questions and Answers in the “Notes” field.
  4. If 2FA is offered by the vendor, continue below,
    If 2FA is not offered, save the Keeper entry: in the top right corner, click the yellow  Save  button.

Setting up 2FA

  1. Setup 2FA: If the vendor offers an option for MFA (Multi-Factor Authentication; also known as 2FA), enable it.
      If you are given multiple options for MFA (email, text message, authenticator app), you should choose Authenticator App if available.

2FA via Authenticator App (Preferred)

Obtain the MFA Setup code from the vendor site.
* If this login will be used on the Scraper, you MUST setup 2FA.

  1. When you are offered the QR Code, find the option that says “Can't scan” and click it.  You will then be provided with a string of numbers and letters.  This is called the Secret.
  2. Save the Secret:  
    1. Click Add Custom Field
    2. Choose Hidden Field
    3. Above the field value box, next to where it says “Hidden Field”, click Edit Label
    4. For the field name, enter Secret
    5. Click Save
    6. In the field value box, paste the Secret code you were given by the website.
  3. Add 2FA:
    1. Click Add Two-Factor Code
    2. Next to Manual Entry, click Setup
    3. In the Secret Key box, paste the Secret code you were given by the website.
    4. No need to fill in any of the other fields.
    5. Click Add
    6. Keeper will then give you 2FA codes for this entry.  Confirm the 2FA code on the vendor website.
  4. ln the top right corner, click the yellow  Save  button.
 
 

2FA via Email

  1. Use the email address on our user profile (typically datateam+clientname@clarivoy.com - replace dealername with the dealer name, no spaces or punctuation).
  2. Update the Keeper entry “Notes” field to note “2FA via email”.
  3. ln the top right corner, click the yellow  Save  button.
 
 

2FA via Phone

  1. Use the Data Team Google Voice Phone number:  ‪(401) 267-2981‬
    Text messages sent to this number are automatically forwarded to the datateam inbox as an email.
    1. If the vendor does not allow you to use this phone number (DriveCentric is one example), they have blocked use of VOIP phone numbers.  You will need to use your personal cell phone number.  In the Keeper entry “Notes” field, note whose phone number was used.
  2. Update the Keeper entry “Notes” field to note whose phone number is being used for 2FA.  Example: “2FA via Charlie's cell”
  3. ln the top right corner, click the yellow  Save  button.
 
 

 

Verifying Access and Permissions

  1. Verify Access to All Stores - Verify we have access for all applicable stores within a group.  The store selector dropdown box is usually in the top right corner of the CRM.
  2. Verify Permissions - See the task notes (integration instructions that were sent to the dealer) for information on what permissions are required.
    1. If no specific permissions are required, we require access to: 
      1. View all leads for all users
      2. Create Reports
      3. Download/Export data
    2. If the permissions are incorrect, tag the dealer in the Rocketlane task and ask them to update the permissions.  Add the dealer back as a task Assignee and set the task status back to In Progress.

 

Verify Permissions

:bust_in_silhouette: Ops Onboarding Specialist

  1. Verify Permissions - Use the below instructions.  If the permissions are incorrect, tag the dealer in the Rocketlane task and ask them to update the permissions.  Add the dealer back as a task Assignee and set the task status back to In Progress.
    1. DealerSocket: verify we have access to Email Logs, perform a quick Search and verify that the "Export" button exists.
    2. DriveCentric: verify we have access to Mining, then Deals.  Select one of the existing mining reports, click the three dots in the top right corner of the report tile and verify that a “Download” option exists.
    3. Elead: perform a quick test search and verify that "Download Merge Results" appears as an option (this verifies we have permissions to export data).
    4. VinSolutions: Onboarding Specialist will verify we have access to create a report. (Do not create the report; just verify that the "Create Report" option exists under Insights/Reporting.
    5. Dealer.com: Verify we have access to Phone logs for each store (separately; permissions are set at the store level) and that we have access to export the Phone logs.

 

Wall-E Setup

:bust_in_silhouette: Ops Specialist or Ops Coordinator

  1. Login to https://wall-e-onboarding.clarivoy.com/
  2. Scraper processes for adding new dealers can be found here: Wall-E Onboarding App
  3. Make sure to map the property aliases in the claripedia.property_aliases table for all stores inside the login.

    Combined Data for Multiple Properties

    Any Leads data that cannot be mapped specifically or accurately to a specific rooftop gets loaded to the parent property.
    See Loading Data to a Parent Property for details on this process.

     
  4. Wall-E will pull data the following day, including (if Pull Backdata was checked) 90 days of backdata.
  5. Set the task status to Waiting on Data
    1. 🚀 RL will set the "Verify Data Date" task field to 2 days later
  6. 🚀 2 days later, RL will change the RL task status to Verify Data

 

Verify Data

:bust_in_silhouette: Ops Team

When the task status is Verify Data, verify the data.

  1. Use the Looker Sales Dashboard link in the update to confirm data is flowing and looks good. Highlight the link using your mouse, right click and chose "Go to https://......" to open the link.
  2. For Sales only:  If it is prior to the Start Date, you will not see data in the Sales Standardized chart. Check the Sales Raw chart instead.
  3. Verify we are receiving leads and/or sales and set the task status to Completed
    1. ✴️ PRO - Rocketlane Zapier Integration creates a new item in the SA+TVA Ops Monthly Tasks board.
      1. For details on this process, see: Data Tasks Setting to Completed in Rocketlane.
      2. If the Monthly Task board item cannot be created by Zapier, an error message should appear in the task Chat explaining why.  Typically this is because the required fields are not filled in on the task's Additional Info tab (Method, Task Vendor, Data Type).


Keep exploring

Related Articles

Ops Team: Sale Count Discrepancy Investigations Loading Data to a Parent Property Third Party Data Sharing Processes

Can't find what you're looking for?

Our team is here to help.

Contact Support