Question Answering with BERT using content

Overview

Teaching: 15 min min
Exercises: 0 min

Questions

How to create an AI chatbot with content using BERT

Objectives

8. Question Answering with BERT using content

In this problem we use the given text content and construct Q&A based on that.

8.1 Q&A with Adam and Eve content.

Assuming we have the Adam and Eve’s story as following, saved as Stody1_Adam_Eve.txt

The story of Adam and Eve is a well-known biblical narrative that appears in the Book of Genesis. According to the story, God created Adam, the first man, and placed him in the Garden of Eden, a paradise where all of his needs were provided for. However, God saw that Adam was alone and decided to create a partner for him, so he created Eve, the first woman, from one of Adam's ribs.
Adam and Eve lived in the Garden of Eden and enjoyed a close relationship with God, but they were given one commandment: they were not allowed to eat from the tree of the knowledge of good and evil. However, one day, a serpent came to Eve and convinced her to eat from the forbidden tree, telling her that it would make her wise. Eve ate the fruit and gave some to Adam, who also ate it.
After they ate from the tree, Adam and Eve became aware of their nakedness and were ashamed. They tried to hide from God, but God knew what they had done and cursed them, expelling them from the Garden of Eden and condemning them to a life of toil and hardship. The story of Adam and Eve is often interpreted as an allegory for the fall of humanity and the origin of sin and suffering.

We have list of questions for the model to answer, saved as Questions-Adam.txt

"Who is Adam",
"Where is Garden of Eden?",
"Who invented Apple?",
"Do you believe in God?"

We build the model saves as qabert-Adam-Eve.py

from transformers import AutoTokenizer, TFAutoModelForQuestionAnswering
import tensorflow as tf

tokenizer = AutoTokenizer.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")
qa_model = TFAutoModelForQuestionAnswering.from_pretrained("bert-large-uncased-whole-word-masking-finetuned-squad")

def answer_question(question, text):
  # Function that simplifies answering a question
  for question in questions:
    # Concatenate the question and the textx
    inputs = tokenizer(question, text, add_special_tokens = True, return_tensors = 'tf')
    # Get the input ids (numbers) and convert to tokens (words)
    input_ids = inputs["input_ids"].numpy()[0]
    text_tokens = tokenizer.convert_ids_to_tokens(input_ids)
    # Run the pretrained model to get the logits (raw scores) for the scores
    output = qa_model(inputs)

    # Get the most likely beginning and end
    answer_start = tf.argmax(output.start_logits, axis = 1).numpy()[0]
    answer_end = (tf.argmax(output.end_logits, axis = 1)+1).numpy()[0]
    # Turn the tokens from the ids of the input string, indexed by the start and end tokens back into a string
    answer = tokenizer.convert_tokens_to_string(tokenizer.convert_ids_to_tokens(input_ids[answer_start:answer_end]))

    print("Question {} \nAnswer: {}".format(question, answer))

with open("Story1_Adam_Eve.txt") as f:
    bow1 = f.read()
    
with open('Questions-Adam.txt') as f1:
    q1 = f1.read()
questions = q1.split("\n")    

answer_question(questions, bow1)

The response would be:

We can see that the response is very much following the given content, how about we have another story of Apple Inc and asking the same questions?

8.2 Q&A with Apple content.

Here is our content copied from Wikipedia for Apple Inc:

Apple Computers, Inc. was founded on April 1, 1976, by college dropouts Steve Jobs and Steve Wozniak, who brought to the new company a vision of changing the way people viewed computers.
Jobs and Wozniak wanted to make computers small enough for people to have them in their homes or offices. Simply put, they wanted a computer that was user-friendly.
Jobs and Wozniak started out building the Apple I in Jobs' garage and sold them without a monitor, keyboard, or casing (which they decided to add on in 1977). 
The Apple II revolutionized the computer industry with the introduction of the first-ever color graphics. Sales jumped from $7.8 million in 1978 to $117 million in 1980, the year Apple went public.
Wozniak left Apple in 1983 due to a diminishing interest in the day-to-day running of Apple Computers. Jobs then hired PepsiCo's John Sculley to be president. However, this move backfired and after much controversy with Sculley, Jobs left in 1985 and went on to new and bigger things.
He founded his own company NeXT Software and he also bought Pixar from George Lucas, which would later become a huge success in computer animation of such movies as Toy Story, A Bug's Life, Monsters, Inc., and Finding Nemo, but not the bible Adam and Eve

Running similar python file with similar question from Questions-Adam.txt we have following answer:

Comparing the same question, “Who invented Apple”, under different context, we have different response from the chatbot model

Key Points

Question Answering, chatbot, BERT

previous episode

SMU Research Computing

next episode