I’m currently on a multi-week mission to explore the Hugging Face (HF) code library for Transformer Architecture (TA) systems for natural language processing (NLP) and today I did a question-answer (QA) example. Whew! That’s a lot of acronyms in an introductory sentence (IS)!

TA systems are extraordinarily complex, so implementing a TA system from scratch or using a low-level library like PyTorch or or Keras is only barely feasible. The HF library makes writing TA systems much simpler — with the downside that customizing a TA system built on HF can be very difficult.

My approach to learning a new technology is to 1.) get a documentation example working, 2.) refactor the example, 3.) repeat until the overall picture gels in my head.

My recent example is extractive question-answer. I set up a raw text source corpus of a few sentences from the Wikipedia article on Transformers. Then I created a BERT-based model using HF and used the model to answer the question, “How do transformers work?” The computed answer was “deep learning model that adopts the mechanism of attention, differentially weighing the significance of each part of the input data.”

The moral of the story is that there are no shortcuts when it comes to learning a complex new software library or framework. You have to take one step at a time. Today was one of those steps for me.

Here are three beach photos where there is no answer to the question, “Why?”

Demo code:

# qa_test.py

# Python 3.7.6 (Anaconda3-2020.02)
# PyTorch 1.9.0 CPU, HugFace 4.2.2, Windows 10

# extractive question-answering using Hugging Face

from transformers import AutoTokenizer, \
  AutoModelForQuestionAnswering
import torch as T

def main():
  print("\nBegin extractive question-answer using Hugging Face ")

  corpus = r"""
    A transformer is a deep learning model that adopts the
    mechanism of attention, differentially weighing the
    significance of each part of the input data. It is used
    primarily in the field of natural language processing
    (NLP) and in computer vision (CV).

    Like recurrent neural networks (RNNs), transformers are 
    designed to handle sequential input data, such as natural 
    language, for tasks such as translation and text 
    summarization. However, unlike RNNs, transformers do not
    necessarily process the data in order. Rather, the 
    attention mechanism provides context for any position in 
    the input sequence. 
  """

  toker = \
    AutoTokenizer.from_pretrained \
    ("bert-large-uncased-whole-word-masking-finetuned-squad")
  model = \
    AutoModelForQuestionAnswering.from_pretrained \
    ("bert-large-uncased-whole-word-masking-finetuned-squad")

  quest = "How do transformers work?"
  print("\nThe question: ")
  print(quest)

  inpts = toker(quest, corpus, add_special_tokens=True,
    return_tensors="pt")
  inpt_ids = inpts["input_ids"].tolist()[0]
  oupts = model(**inpts)

  ans_start_scores = oupts.start_logits
  ans_end_scores = oupts.end_logits

  ans_start = T.argmax(ans_start_scores)
  ans_end = T.argmax(ans_end_scores) + 1

  ans = \
    toker.convert_tokens_to_string \
    (toker.convert_ids_to_tokens(inpt_ids[ans_start:ans_end]))
  print("\nThe answer: ")
  print(ans)

  print("\nEnd demo ")

if __name__ == "__main__":
  main()