Overview

Teaching: 20 min
Exercises: 0 min

Questions

• What is Tensorboard and How to use?

Objectives

• Open Tensorboard in M2

Tensorboard

• TensorBoard is a visualization web app to get a better understanding of various parameters of the neural network model and its training metrics.
• Such visualizations are quite useful when you are doing experiments with neural network models and want to keep a close watch on the related metrics.
• It is open-source and is a part of the Tensorflow version 2+

Some of the useful things you can do with TensorBoard includes:

• Visualize metrics like accuracy and loss.
• Visualize model graph.
• Visualize histograms for weights and biases to understand how they change during training.
• Visualize data like text, image, and audio.
• Visualize embeddings in lower dimension space.

Let setup the procedure to open Tensorboard with M2 HPC

Open Terminal in Open OD or regular terminal, make sure that you are in the same node with Jupyter Lab instance.

Install Tensorboard

pip install tensorboard

Starting Tensorboard

In the previous episode, we already created the log files for CNN training under the same working folder (logs_CNN). Now, let’s load up the log files in the same directory:

# First activate the working conda environment:
source activate ~/tensorflow_2.9

# Second, change directory to where you have the log_CNN saved:
cd Workshop/SMU_DL

# Run the tensorboard with log data in the logs_CNN directory
tensorboard --logdir logs_CNN --host 0.0.0.0

The following information appears:

$ tensorboard --logdir logs_CNN --host 0.0.0.0
2022-03-21 14:41:55.277397: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/users/tuev/Applications/lib:/users/tuev/Applications/lib
2022-03-21 14:41:55.277464: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
/users/tuev/.conda/envs/ML_SKLN/lib/python3.6/site-packages/tensorboard_data_server/bin/server: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by /users/tuev/.conda/envs/ML_SKLN/lib/python3.6/site-packages/tensorboard_data_server/bin/server)
TensorBoard 2.8.0 at http://0.0.0.0:6017/ (Press CTRL+C to quit)

From the above information, let’s remember the node number (p014) and the port number (6017) and these numbers are different for different users.

p014:6017

Setting up Socket Proxy

For Macs OS/Linux

• Setup the SSH tunnel with port forwarding to M2.
• Open a terminal and run the following command with corresponding port number (6017 in this case)(this terminal must be kept opened for the socket proxy to be active):

$ ssh -D 6017 -C -q -N USERNAME@m2.smu.edu

• User need to authenticate themselve with password and DUO

For Window OS

• Install Moba XTerm (https://mobaxterm.mobatek.net)
• Open Moba XTerm and login M2:
• Click on Tunneling

and select New SSH Tunnel:

• From the openning windows, select the following option:
  • Dynamic port forwarding (SOCKS proxy)
  • Forward port 6017
  • SSH Server: m2.smu.edu
  • Username: your own username
  • SSH Port 22

• Press Save and the following window appears:

• Select the play button to start the service, The MobaXTerm will ask for your credential (including Duo)
• Once accepting the Duo push, your Moba XTerm screen would look like this:

Leave the Window on and open Firefox:

Setting up Firefox:

• Firefox should be enable to view website through the proxy.
• Once Firefox opened:
  • Windows: select “ALT+T+S” to open the setting tab.
  • Macs OS: Preferences\General

• Scroll all the way down, you will see Network Settings. Click on Settings …

• Fill in the following information:
• Manual proxy configuration
• SOCKS v5
• Port 6017

• Enable checkboxes for Proxy DNS and Enable DNS over HTTPS:

• Click OK and open a new Tab in Firefox:
• Type in the address bar the node name and port number:

p014:6017

The Tensorboard window appears:

Key Points

• Tensorboard