CS224N lecture20 笔记

发布于 2020-08-04  175 次阅读


The Future of Deep Learning + NLP

Slides

  • Using unlabeled data

    • Back-translation
    • Cross lingual embeddings and shared encoder gives the model a stare point
    • Objectives encourage language-agnostic representation
  • Scaling up pre-training

  • Diagnostic/Probing Classifiers

    • Diagnostic classifier takes representation produced by a model(e.g. Bert) as input and do some task
    • Only the diagnostic classifier is trained
    • Diagnositic classifier are usually very simple(e.g. simple softmax)

阿克西斯上没有什么重要的东西