Incident Streams is a TREC track designed to bring together academia and industry to research technologies to automaticaly process social media streams during emergency situations with the aim of categorizing information and aid requests made on social media for emergency service operators.

The mass adoption of mobile internet-enabled devices paired with wide-spread use of social media platforms for communication and coordination has created new ways for the public on-the-ground to contact response services. With the rise of social media, emergency service operators are now expected to monitor those channels and answer questions from the public However, they do not have adequate tools or manpower to effectively monitor social media, due to the large volume of information posted on these platforms and the need to categorise, cross-reference and verify that information. TREC-IS aims to provide a base for research to tackle this technology gap.

TREC-IS provides multiple Twitter datasets collected from a range of past wildfire, earthquake, flood, typhoon/hurricane, bombing and shooting events, as well as Covid-19. We have had human annotators manually label this data into 25 information types based on the information each tweet contains, such as 'contains location' or is a 'search and rescue request'. Each tweet is also assigned a 'priority' label, that indicates how critical the information within that tweet is for a response officer to see. To date, TREC-IS has manually annotated tweet streams for 122 emergency events, comprising over 130,000 labelled tweets.

TREC-IS is split into editions. During each edition, researchers and practitioners can submit the output of systems aiming to perform tweet labelling automatically for the 25 information types for a set of unseen events. The organisers then produce a leaderboard of systems for each edition, allowing for a fair comparison of systems, as well as demonstrating the current state-of-the-art approaches. TREC-IS ran for seven editions (2018-B, 2019-A, 2019-B, 2020-A, 2020-B, 2021-A and 2021-B). TREC-IS is no longer running further editions, however, you can still submit runs to the 2021-B edition via an online leaderboard or get involved in our new track CrisisFACTs, links below