Practical Tips for Final Projects

Slides

• Default Final Project
  build a textual questional answering sysytem for SQuAD
• some useful sites and data for NLP researchers
• simple review of RNN; OOV; NMT on evaluation

Notes

• datasets site

• processing data

• data hygiene

  • search hyperparameters
  • decide when to stop training

• paper structure

Suggested Readings

1. Practical Methodology(Deep Learning book chapter)
   • PR curve
     precision on the y-axis; recall on the x-axis
   • Coverage
     coverage is the fraction of examples for which the machine learning system is able to produce a response
   • Batch normalization
     can have a dramatic effect on optimization performance, especially for convolutional networks and networks with sigmoidal nonlinearities;
     it is reasonable to omit batch normalization from the very first baseline
   • Select hyperparameters
     • manualy hyperparameter tuning
     • automatic hyperparameter optimization algorithms
     • grid search
     • random search