Topics in Neural Information Retrieval

Advanced lecture, 6 ECTS credits, summer semester 2019

Basic Information

  • Type: Advanced lecture
  • Lecturer: Dr. Andrew Yates
  • Credits: 6 ECTS credits
  • Time: Tuesdays (and a few Thursdays), 14-16:00 in Room 029 E1.5
  • Mailing list for discussion and announcements
  • Please register for the mailing list if you'd like to take the class!

Overview

Summary

In this course we will be investigating advanced topics in Information Retrieval, with a focus on neural network methods and how they contrast with prior work. In lecture we'll work to understand important problems in IR and how they're solved by both neural and traditional methods, and how these methods relate to IR theory. In contrast with the common claim that neural models are difficult to make sense of, we'll explore how the design of neural IR architectures often follows directly from theory. Assignments will require gaining an in-depth understanding of several related methods by reading scientific reports and demonstrating this understanding by writing an essay analyzing the methods. While beneficial, no background in Deep Learning will be assumed; we'll start with overviews of both IR and DL before moving on to more advanced topics like state-of-the-art retrieval models for assessing the relevance of a document to a given query, diversifying search results based on their novelty (with respect to each other), training neural models with weak supervision, and applications of neural IR to other tasks.

Objectives

By the end of the course, students will be able to describe and contrast state-of-the-art traditional and neural IR approaches, to examine and critique the assumptions made by these approaches, and to critically read and analyze relevant scientific literature.

Prerequisites

Students should have a basic knowledge of Machine Learning. Prior knowledge of Information Retrieval and Neural Networks will be helpful but is not required.

Schedule (tentative)

 Date Topic AssignmentReading
 9 AprilIntroduction  
 16 Aprilno class  
 23 AprilAxiomatic Thinking: how should a model behave?#1 assigned

Background: [1] Sec 8.1-8.6 + 11.4.3

Required: [2]

 30 AprilKeywords: of words and bags 

Background: [3] Ch 4 + Sec 6.1

Required: [4]

Optional: [5]

 7 Mayno class  
 14 Mayno class  
 21 MayPhrases: to be or not to be? 

Required: [6]

Optional: [12], [13], [7] Ch 3 (expands on [6])

 23 May (Thurs)Phrases: (to be) continued#1 due, #2 assigned

Background: [3] Sec 6.2

Required: [8], [9]

Optional: [10], [11]

 28 Mayno class  
 30 May (Thurs)Passages: it's turtles all the way down 

Background: [14]

Required: [15]

Optional: [16]

 4 JuneSemantic scoring TBA
 6 June (Thurs)Semantic matching#2 due, #3 assignedTBA
 11 Juneno class  
 18 JuneDiversification TBA
 25 JuneQuery expansion#3 due, #4 assignedTBA
 2 JulyWeak supervision TBA
 9 JulyApplications TBA
 16 JulyWrap-Up#4 due 
 30 Julyoral exams  
 24 Septre-exams  

Each class has several related readings. It is recommended that you read the required readings before lecture, supplementing them with the background readings as needed. It is not necessary to read the optional readings before class, though students will read many of them as part of the class assigments. 

References

[1] Christopher D. Manning, Manning Raghavan, & Manning Schütze. 2008. Introduction to Information Retrieval. [chapter pdfs]

[2] Hui Fan, Tao Tao, & ChengZiang Zhai. 2004. A formal study of information retrieval heuristics. Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '04). [pdf]

[3] Andriy Burkov. The Hundred-Page Machine Learning Book. 2019. [chapter pdfs]

[4] Jiafeng Guo, Yixing Fan, Qingyao Ai, and W. Bruce Croft. 2016. A Deep Relevance Matching Model for Ad-hoc Retrieval. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management (CIKM '16). [pdf]

[5] Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, and Russell Power. 2017. End-to-End Neural Ad-hoc Ranking with Kernel Pooling. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '17). [pdf]

[6] Donald Metzler and W. Bruce Croft. 2005. A Markov random field model for term dependencies. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '05). [pdf]

[7] Donald Metzler. 2011. A Feature-Centric View of Information Retrieval. [chapter pdfs] (link only works from uni network)

[8] Zhuyun Dai, Chenyan Xiong, Jamie Callan, and Zhiyuan Liu. 2018. Convolutional Neural Networks for Soft-Matching N-Grams in Ad-hoc Search. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM '18). [pdf]

[9] Kai Hui, Andrew Yates, Klaus Berberich, Gerard de Melo. 2017. PACRR: A Position-Aware Neural IR Model for Relevance Matching. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP '17). [pdf]

[10] Andrew Yates, Kai Hui. 2017. DE-PACRR: Exploring Layers Inside the PACRR Model. In the SIGIR 2017 Workshop on Neural Information Retrieval (NeuIR '17). [pdf]

[11] Kai Hui, Andrew Yates, Klaus Berberich, Gerard de Melo. 2018. Co-PACRR: A Context-Aware Neural IR Model for Ad-hoc Retrieval. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM '18). [pdf]

[12] Samuel Huston and W. Bruce Croft. 2014. A Comparison of Retrieval Models using Term Dependencies. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM '14). [pdf]

[13] Tao Tao and ChengXiang Zhai. 2007. An exploration of proximity measures in information retrieval. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '07). [pdf]

[14] Chris Olah. 2015. Understanding LSTM Networks. Blog post. [link]

[15] Yixing Fan, Jiafeng Guo, Yanyan Lan, Jun Xu, Chengxiang Zhai, and Xueqi Cheng. 2018. Modeling Diverse Relevance Patterns in Ad-hoc Retrieval. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR '18). [pdf]

[16] Zhiwen Tang and Grace Hui Yang. DeepTileBars: Visualizing Term Distribution for Neural Information Retrieval. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19). [pdf]

Course Structure

This advanced course consists of 2 hours of lecture per week, four writing assignments that require students to critically read and contrast several scientific articles, and an oral exam. Given the assignments' focus on reading and analyzing scientific literature to achieve a deep understanding of a topic, there will be no weekly tutorials.

Students' final grades will be determined based on the final exam, which will be an oral exam covering material from lectures and their associated readings. In order to be eligible to take the final exam, students must pass all four assignments. Final exam time slots for each student will be announced via email. Students should inform the lecturer of any potential conflicts with the (re-)exam dates as soon as possible.

Assignments will involve reading several scientific papers in order to answer an essay prompt by critically discussing the papers. For each assignment, students will individually read one or more research papers and submit a report discussing the reading and answering the assignment questions. Reports must critically discuss the assigned papers and demonstrate understanding of the topic; simply summarizing them will not be sufficient to receive a passing grade. Reports must cite all sources used. The recommended report length is three pages. Reports are due at noon on the deadline.

Assignments will be given one of four grades: Fail, Pass, Good, or Excellent. Students are allowed to re-submit one failed assignment within two weeks of the assignment deadline. Any assignment that is not submitted by the deadline will be considered failed. Receiving a grade of Excellent gives you one bonus point. Two Good grades count as one Excellent. Each bonus point will improve your final grade by 1/3rd of a point, up to 1 point maximum, on the condition that you pass the final exam. For example, if your final exam grade is 1.6 and you have one bonus point, your final grade will be 1.3. Students who fail the final exam also fail the course regardless of their assignment grades.