CS224N lecture09 笔记

发布于 28 天前  49 次阅读

Practical Tips for Final Projects


  • Default Final Project
    build a textual questional answering sysytem for SQuAD
  • some useful sites and data for NLP researchers
  • simple review of RNN; OOV; NMT on evaluation


  • datasets site

  • processing data

  • data hygiene

    • search hyperparameters
    • decide when to stop training
  • paper structure

Suggested Readings

  1. Practical Methodology(Deep Learning book chapter)
    • PR curve
      precision on the y-axis; recall on the x-axis
    • Coverage
      coverage is the fraction of examples for which the machine learning system is able to produce a response
    • Batch normalization
      can have a dramatic effect on optimization performance, especially for convolutional networks and networks with sigmoidal nonlinearities;
      it is reasonable to omit batch normalization from the very first baseline
    • Select hyperparameters
      • manualy hyperparameter tuning
      • automatic hyperparameter optimization algorithms
      • grid search
      • random search