CS224N lecture20 笔记

发布于 2020-08-04  62 次阅读

The Future of Deep Learning + NLP


  • Using unlabeled data

    • Back-translation
    • Cross lingual embeddings and shared encoder gives the model a stare point
    • Objectives encourage language-agnostic representation
  • Scaling up pre-training

  • Diagnostic/Probing Classifiers

    • Diagnostic classifier takes representation produced by a model(e.g. Bert) as input and do some task
    • Only the diagnostic classifier is trained
    • Diagnositic classifier are usually very simple(e.g. simple softmax)