Incident Streams is a TREC track designed to bring together academia and industry to research technologies to automaticaly process social media streams during emergency situations with the aim of categorizing information and aid requests made on social media for emergency service operators.

The mass adoption of mobile internet-enabled devices paired with wide-spread use of social media platforms for communication and coordination has created new ways for the public on-the-ground to contact response services. With the rise of social media, emergency service operators are now expected to monitor those channels and answer questions from the public However, they do not have adequate tools or manpower to effectively monitor social media, due to the large volume of information posted on these platforms and the need to categorise, cross-reference and verify that information. TREC-IS aims to provide a base for research to tackle this technology gap.

TREC-IS provides multiple Twitter datasets collected from a range of past wildfire, earthquake, flood, typhoon/hurricane, bombing and shooting events (labeled pandemic data will be added later this year). We have had human annotators manually label this data into 25 information types based on the information each tweet contains, such as 'contains location' or is a 'search and rescue request'. Each tweet is also assigned a 'priority' label, that indicates how critical the information within that tweet is for a response officer to see. To date, TREC-IS has manually annotated tweet streams for 48 emergency events, comprising 50,000 tweets and producing over 125,000 labels.

TREC-IS is split into editions. During each edition, researchers and practitioners can submit the output of systems aiming to perform tweet labelling automatically for the 25 information types for a set of unseen events. The organisers then produce a leaderboard of systems for each edition, allowing for a fair comparison of systems, as well as demonstrating the current state-of-the-art approaches. TREC-IS has run five editions so far (2018-B, 2019-A, 2019-B, 2020-A and 2020-B). The next upcomming edition is 2021-A, with submission due by the 3rd of March 2020. 2021-A will cover general natural and man-made disasters, as well as COVID-19.