Incident Streams is a TREC track designed to bring together academia and industry to research technologies to automaticaly process social media streams during emergency situations with the aim of categorizing information and aid requests made on social media for emergency service operators.

The mass adoption of mobile internet-enabled devices paired with wide-spread use of social media platforms for communication and coordination has created new ways for the public on-the-ground to contact response services. With the rise of social media, emergency service operators are now expected to monitor those channels and answer questions from the public However, they do not have adequate tools or manpower to effectively monitor social media, due to the large volume of information posted on these platforms and the need to categorise, cross-reference and verify that information. TREC-IS aims to provide a base for research to tackle this technology gap.

TREC-IS provides multiple Twitter datasets collected from a range of past wildfire, earthquake, flood, typhoon/hurricane, bombing and shooting events (labeled pandemic data will be added later this year). We have had human annotators manually label this data into 25 information types based on the information each tweet contains, such as 'contains location' or is a 'search and rescue request'. Each tweet is also assigned a 'priority' label, that indicates how critical the information within that tweet is for a response officer to see. To date, TREC-IS has manually annotated tweet streams for 33 emergency events, comprising 35,000 tweets and producing over 125,000 labels.

TREC-IS is split into editions. During each edition, researchers and practitioners can submit the output of systems aiming to perform tweet labelling automatically for the 25 information types for a set of unseen events. The organisers then produce a leaderboard of systems for each edition, allowing for a fair comparison of systems, as well as demonstrating the current state-of-the-art approaches. TREC-IS has run three editions so far (2018-B, 2019-A and 2019-B). The next upcomming edition is 2020-A, with submission due in June 2020. 2020-A will cover both general natural and man-made disasters, as well as COVID-19.

Submission instructions for 2020-A can be found here, the submission deadlines are June 1st for Tasks 1/2 and June 8th for Task 3: