IEEE ICME 2024 Grand Challenge

2024-FEB-05 The challenge has started and links to the development dataset and registration are available.

2024-MAR-10 Update on the baseline codes to correct issue with the evaluation metric.

2024-MAR-15 The link to the evaluation dataset is available.

2024-MAR-19 The link to the final result submission is available. See the updates below.

2024-MAR-22 The challenge has ended. We received submissions from 13 teams. The final results of the challenge will be announced soon.

2024-MAR-29 The challenge results have been announced. Congratulations to the winners!

The top 5 teams are invited to submit the paper to the ICME 2024 Workshop. Papers should be submitted via the ICME 2024 CMT. Please select the GC-ASC track. The workshop papers have the same format as regular papers required by the ICME 2024 guidelines.

Acoustic scene classification (ASC) is a crucial research problem in computational auditory scene analysis, and it aims to recognize the unique acoustic characteristics of an environment. One of the challenges of the ASC task is domain shift caused by a distribution gap between training and testing data. Since 2018, ASC challenges have focused on the generalization of ASC models across different recording devices. Although this task in recent years has achieved substantial progress in device generalization, the challenge of domain shift between different regions, involving characteristics such as time, space, culture, and language, remains insufficiently explored at present. In addition, considering the abundance of unlabeled acoustic scene data in the real world, it is important to study the possible ways to utilize these unlabelled data. Therefore, we introduce the "Semi-supervised Acoustic Scene Classification under Domain Shift" in the ICME 2024 grand challenge dealing with the problems. We encourage participants to innovate with semi-supervised learning techniques, aiming to develop more robust ASC models under domain shift.

The CAS 2023 dataset is a large-scale dataset that serves as a foundation for research related to environmental acoustic scenes. The dataset includes 10 common acoustic scenes, with a total duration of over 130 hours. Each audio clip is 10 seconds long with metadata about the recording location and timestamp. The dataset was collected by members of the Joint Laboratory of Environmental Sound Sensing at the School of Marine Science and Technology, Northwestern Polytechnical University. The data collection spanned from April 2023 to September 2023, covering 22 different cities across China.

The ASC challenge dataset consists of development and evaluation datasets, all derived from the CAS 2023 dataset. The development dataset is about 24 hours including the recordings from 8 cities. We provided scene labels for 20% of the data in the development dataset to allow participants to develop effective semi-supervised methods. In the evaluation dataset, data are selected from 12 cities, with 5 unseen cities specifically chosen to provide a more comprehensive evaluation of submissions under domain shift.

Acoustic scenes (10): Bus, Airport, Metro, Restaurant, Shopping mall, Public square, Urban park, Traffic street, Construction site, Bar

Cities (22): Xi'an, Xianyang, Changchun, Jinan, Hefei, Sanya, Nanning, Haikou, Guilin, Guangzhou, Chongqing, Shenyang, Beijing, Baishan, Taiyuan, Tianjin, Nanchang, Shanghai, Luoyang, Liupanshui, Shangrao, Dandong.

More details can be found at https://arxiv.org/abs/2402.02694.

The development dataset is released at https://zenodo.org/records/10616533.

The evaluation dataset is released at https://zenodo.org/records/10820626.

The participants can register by filling out the Google form.

If you have any questions, please join Google Groups for discussion.

The baseline system for the ICME 2024 "Semi-supervised Acoustic Scene Classification under Domain Shift" challenge is based on a semi-supervised framework with a Squeeze-and-Excitation and Transformer (SE-Trans) model pre-trained on the TAU Urban Acoustic Scenes (UAS) 2020 Mobile development dataset. Baseline codes are released in github. Systems will be ranked by macro-average accuracy (average of the class-wise accuracies). If two teams got the same score on the evaluation dataset, the team with the smaller model size will be ranked higher.

• Only the TAU UAS 2020 Mobile development dataset and CochlScene dataset are allowed for model pre-training and to facilitate the training process in this challenge.
• Model ensembles are NOT allowed in this challenge.

Authors of the top 3 ranked solutions will be awarded prizes funded by Xi'an Lianfeng Acoustic Technologies Co., Ltd.

Top 1: 600 USD

Top 2: 500 USD

Top 3: 400 USD

Note: All winning teams will receive corresponding challenge bonuses. Personal income tax or other forms of taxes on bonuses will be borne by the winners and paid by the challenge organizion. The prize money of the winning team will be distributed to the team captain. The team captain shall be responsible for allocating and distributing the prize money and prizes among its members.

Participants must adhere to the following terms and conditions. Failure to comply will result in disqualification from the Challenge.

The model should not be trained on any data from the evaluation dataset.

Participants are required to submit the results by the challenge submission deadline.
To ensure the results can be reproduced by the organizers, the submission should include:
1) A .csv file including the classification results for each audio in the evaluation set with the following template:

2) Inference codes and trained models for evaluation by the organizers.

3) Comprehensive documentation, instructions, or other relevant information to facilitate the execution of the codes by the organizers.

4) A technical report (*.pdf) explaining the method in sufficient detail (2-6 pages including references). This report will be publicly available on the challenge website. The ICME paper template is recommend.

All files should be packaged into a zip file for submission. You can submit your final results through the Google Form. Each team is limited to submitting only ONE system. If multiple submissions are made, only the last submission before the deadline will be considered. Please carefully provide the correct information: Team Name, Institute, Team Contact Name(Last name, First name) and Institutional E-mail.