02. MLOps: Model Monitoring - Weights and Bias

ML Model Monitoring

Generally we will be running experiments by tweaking hyper-parameters, trying different models to test their performance, see the connection between your model and the input data, and perform advanced tests. Having all these logged at a single place will help in getting better and faster insights.

The easiest way to ensure things work smoothly is to use ML model monitoring tools.

Dedicated tools can also be used to collaborate with your team, share your work with other people. It’s a shared space for teams to collaborate, participate in model creation and further monitoring. It’s easier to exchange ideas, thoughts and observations, and spot errors when you have real-time insight into what’s happening with your models.

There are many libraries available to monitor machine learning models. The prominent ones are:

• Comet

• MLFlow

• Neptune

• TensorBoard

• Weights and Bias

and many more...

At week 1 we used TensorBoard for monitoring

However this week we will be using Weights and Bias.

In this post, I will be going through the following topics:

• How to configure basic logging with W&B?

• How to compute metrics and log them in W&B?

• How to add plots in W&B?

• How to add data samples to W&B?

Weights and Bias Configuration

Run the command:

wandb login

You will be prompted with the following:

Follow the authorization link: https://wandb.ai/authorize and copy paste the api key.

Configuring

Create a project at W&B and then use the same name here. So that all the experiments will be logged into that project.

Because of we were using TensorBoard, it's easy to integrate with wandb.

import tensorflow as tf
import wandb
wandb.init(config=tf.flags.FLAGS, sync_tensorboard=True)

Now all the logs will be tracked in W&B.

References

• WandB with Tensorflow

• WandB Documentation

• Raviraja Ganta's MLOps Basic