Black box analysis of deep neural language models

Neural networks have achieved spectacular results in many language applications (speech recognition, translation, etc), but the way in which these systems work is still poorly understood, and the systems sometimes show spectacular failure modes. This internship proposes to deploy experimental tasks used in cognitive psychology to (1) better understand the performance of these systems in terms of theories of the human linguistic competence (2) identify specific situations where artificial systems fails compared to humains. The outcome of this internship may be implemented in AI benchmarks or challenges. This internship requires both a familiarity with human language (linguistics or psycholinguistics major) and competence in data analysis and scripting.