Text Classification using Neural Networks

Understanding how chatbots work is important. A fundamental piece of machinery inside a chat-bot is the text classifier. Let’s look at the inner workings of an artificial neural network (ANN) for text classification.

We’ll use 2 layers of neurons (1 hidden layer) and a “bag of words” approach to organizing our training data.

• the algorithm produces a score rather than a probability. We want a probability to ignore predictions below some threshold. This is akin to a ‘squelch’ dial on a VHF radio.
• the algorithm ‘learns’ from examples of what is in a class, but not what isn’t. This learning of patterns of what does not belong to a class is often very important.
• classes with disproportionately large training sets can create distorted classification scores, forcing the algorithm to adjust scores relative to class size
  . This is not ideal.

Join 30,000+ people who read the weekly