To connect to the following websites in your web browser, you will need to first connect to the Jumphost/Airflow server:

• Airflow
• Wall-E Onboarding Configuration

Installation

These instructions are for MacOS.  Instructions for Windows may vary.

1. Download iTerm2
2. Install the software
3. Launch iTerm2

Connecting to JumpHost

1. You may need to Set up an SSH Key in order to connect.
2. In iTerm2, type:  ssh -D 1080 jumphost1.clarivoy.com
3. For instructions on setting up a profile for quick access, see Setting up iTerm Profiles.

Connecting to the Jobs Server

Most Airflow DAGs store log files on the Jobs server that can be used for researching technical issues.

1. You may need to Set up an SSH Key in order to connect.
2. In iTerm2, type: ssh jobs-runner.clarivoy.com
3. For instructions on setting up a profile for quick access, see Setting up iTerm Profiles

Viewing Log Files on the Jobs Server

1. In iTerm2, Once connected to the Job server, type:

cd ../clarivoyjobs/logs
ls

This will present you with a list of all available log files on the server.  The file named [job].log will be the most recent, with [job].log.1 being the next most recent (further back), followed by [job].log.2, etc. with [job] representing the name of the log file.

2. To view a specific log file, type:

 tail -100000 [job].log

with [job] representing the name of the log file.  This will allow you to view the most recent 100,000 rows in the log file.

If the log file is larger and you need to view more lines, adjust the 100000 in the command.

Once the log has been displayed, you can use CTRL+F to search the log file for the specific API key, subscription ID (CDK), property name, or property ID that you need.

Common Jobs Server Log Files

• etl_leads_preprocessing.log 
• etl_sales_preprocessing.log

Setting Up iTerm Profiles

These instructions are for the MacOS version.

1. iTerm2 menu
2. Settings
3. Profiles tab
4. Click the + button in the bottom right corner to add a Profile.
5. Fill in these details for the new profile:
   1. General tab
      1. Name
      2. Command = Command
      3. Command string = your connect command for the server
         e.g. ssh jobs-runner.clarivoy.com
      4. Send Text at start = the commands you want iTerm to execute upon conneting to that server.
         e.g. to view CallRevu logs on the Jobs server:  cd ../clarivoyjobs/logs; tail -10000 callrevu_phone_api_pull.log
   2. Colors tab - This is not necessary, but I like to use different colors for each Profile to help easily identify what I am looking at and to differentiate between the tabs.
      1. Defaults: Background = set a background color for the tab's terminal display (the background color behind the text)
      2. Scroll to the bottom and under Tab = I set a similar color for the tab and check “Use custom tab color”
   3. Window tab - This is important so the tab created when you open the profile gets a unique name that helps identify it.
      1. New tabs: Check “Use custom title for new tabs using this profile” and enter a name for the tab (I use the same name as the profile name)
6. You can choose to set a default profile that iTerm will open each time you start the app.  I set my default as the Airflow/Jumphost server as that is the profile I use most often,
   1. On the left pane, select the profile you want as your default
   2. At the bottom, click “Other actions”
   3. Click “Set as Default”
   4. Now iTerm will open this tab each time you start the app.
7. There is no Save button.  Just close the dialog box to save.

Profile example

Here is an example of a profile I named CallRevu which will connect to the Jobs server and display the last 10,000 rows in the CallRevu Phone API log:

General

 

 

Colors

 

 

Window

 

 

 

Using iTerm Profiles

1. in iTerm2 for MacOS, press Command + Shift + B to view the Profiles pane on the right side of the iTerm window.
2. Double click on the profile you wish to open.  It will open in a new tab (if you have “New tab” selected at the bottom of the Profiles tab; this is the default).

Common Terminal Commands

ls

• ls = lists all log files

tail

• tail -10000 <logname>  =  view 10000 lines of the log
• tail -10000 etl_leads_preprocessing.log  =  view 10000 lines of the leads preprocessing log
• tail -10000 etl_sales_preprocessing.log  =  view 10000 lines of the sales preprocessing log

less

• less etl_leads_preprocessing.log  =  view beginning of leads preprocessing log, page by page
• less etl_sales_preprocessing.log  =  view beginning of leads preprocessing log, page by page
• When running less:
  • SHIFT+G = go to bottom of logs
  • CTRL+F = find - search for “error”
  • Q = quit