When you are getting started in Weka, you may feel overwhelmed.

There are so many datasets, so many filters and so many algorithms to choose from.

There is too much choice. There are too many things you could be doing.

Too much Choice
Photo by emilio labrador, some rights reserved.

Structured process is key.

I have talked about process and the need for tasks like spot checking algorithms to overcome the overwhelm and start learning useful things about your problem. In this post I want to give you a simplified version of this process that you can use to practice applied machine learning.

Problem Solving Template

This template a streamlined process that focuses on learning about the problem, a good solution, and doing so very quickly.

It is organized into the six-steps of applied machine learning. Each step is broken down into specific questions for you to answer by using the Weka Explorer

The six-steps of the process and their objectives are as follows:

1. Problem Definition
2. Data Analysis
3. Data Preparation
4. Evaluate Algorithms
5. Improve Results
6. Present Results

In the following sections I will summarize the key questions to answers for each step of the process. You might like to print out these questions or copy them into a document to create your own template document.

1. Problem Definition

The objective of the problem definition is to understand and clearly describe the problem that is being solved.

Problem Description

1. What is an informal description of the problem?
2. What is a formal description of the problem?
3. What assumptions do you have about the problem?

Provided Data

1. What constraints were imposed to select the data?
2. Define each attribute in the provided dataset.

2. Data Analysis

The objective of data analysis is to understand the information available that will be used to develop a model.

Attribute Histograms Showing Class Values

1. What data types are the attributes?
2. Are there missing or corrupted values?
3. Review the distributions of the attributes, what do you notice?
4. Review the distributions of the class values, what do you notice?
5. Review the attribute distributions with class values in the histograms, what do you notice?
6. Review pairwise scatter plots of attributes, what do you notice?

3. Data Preparation

The objective of data preparation is to discover and expose the structure in the dataset.

1. Normalize the dataset
2. Standardize the dataset
3. Square the dataset
4. Discretize attributes (if integer)
5. Remove and/or replace missing values (if present)
6. Create transforms of the dataset to test assumptions raised in the Problem Definition

4. Evaluate Algorithms

The objective of evaluating algorithms is to develop a test harness and baseline accuracy from which to improve.

Algorithm ranking when analyzing results in the Weka Experimenter

1. Explore different classification algorithms
2. Design and run a spot-check experiment
3. Review and interpret the algorithm rankings
4. Review and interpret the algorithm accuracy
5. Repeat process as needed

5. Improve Results

The objective of improving the results is to leverage results to develop more accurate models.

Algorithm Tuning

1. Explore different algorithm configurations
2. Design and run a algorithm tuning experiment
3. Review and interpret the algorithm rankings
4. Review and interpret the algorithm accuracy
5. Repeat process as needed

Ensemble Methods

1. Explore different ensemble methods
2. Design and run a algorithm ensemble experiment
3. Review and interpret the ensemble rankings
4. Review and interpret the ensemble accuracy
5. Repeat process as needed
6. Can you improve results with other meta algorithms, such as thresholding?
7. Can you improve results by using other algorithms in the same family as algorithms that are performing well?

6. Present Results

The objective of presenting the results is to describe problem and solution so that it can be understood by third parties.

Complete the following section to summarize the problem and solution.

1. What is the Problem?
2. What is the Solution?
3. What were the Findings?
4. What are the Limitations?
5. What are the Conclusions?

How To Use

There are a number of interesting datasets in the “data” directory of the Weka installation. There are also many datasets on the UCI machine learning repository that you can download and work on.

Select a problem and work through it using this template. You will be surprised at how much you learn and how much a structured process like this can help to keep you focused.

Summary

In this post you learned about a structured template for working the process of applied machine learning. This template can be printed and used step-by-step to work through a problem in the Weka Machine Learning Workbench.

Answering the specific questions in each step of the template will quickly build up a deeper understanding of the problem and your solution to it, as it unfolds. This is invaluable, like a scientists notebook in the lab.