CADA - Aneurysm Segmentation Banner

Submission Format

For the segmented output, we expect the following filename pattern "$INPUTNAME_output.nii.gz". E.g., for the input file "A038_R", the output needs to be named "A038_R_output.nii.gz". Furthermore, we suppose the segmentation results as NIFTI file, with 0 for the background and 1, 2, 3... for the segmented aneurysms (exactly like the labeledMasks files).

In addition to the nii.gz labeledMasks images, a JSON file needs to be provided with information on hardware specification, method automization level, and processing time. The JSON file needs to be named "results-task2.json".

For the submission, we expect one compressed gzip (standard GNU) file, containing the JSON file and the segmented NIFTI files. This gzip file should be named "$USERNAME-result-task2.zip", where $USERNAME is your grand-challenge user name.

Here is an example for the gzip file structure:

  • $USERNAME-result-task2.gz
    • results-task2.json
    • A001_output.nii.gz
    • A003_output.nii.gz
    • A005_output.nii.gz


JSON Submission Format Example

The following is just an abbreviated example illustrating the submission format. Always make sure to validate your submission using the JSON schema provided at the page bottom!

{
    "grand_challenge_username": "my_user_name_on_grand_challenge",
    "used_hardware_specification": {
        "CPU": "Intel Core i9 9900K 8x 3.60GHz",
        "GPU": "NVIDIA RTX 2080 Ti",
        "#GPUs": 1,
        "RAM_in_GB": 4,
        "additional_remarks": "special hardware requirements, other comments"
    },
    "level_of_automation": "WithUserInteraction", // Options: "FullyAutomated" (without seed points and interaction), "WithUserInteraction", "AutomatedUsingSeedPoints" (using the seed points)
    "task_2_results": [
        {
            "dataset_id": "A018",
            "candidates" : [
                {
                    "aneurysm_label": 1,
                    "automatic_processing_time_in_seconds": 22.7,
                    "manual_interaction_time_in_seconds": 10.9 // set to 0 for "FullyAutomated" algorithms
                }
            ]
        },
        {
            "dataset_id": "A038_R",
            "candidates" : [
                {
                    "aneurysm_label": 1,
                    "automatic_processing_time_in_seconds": 22.7,
                    "manual_interaction_time_in_seconds": 6.2
                },
                {
                    "aneurysm_label": 2,
                    "automatic_processing_time_in_seconds": 14.1,
                    "manual_interaction_time_in_seconds": 13.1
                }
            ]
        }
    ]
}

JSON Schema for Submission Validation 

In order for you to check if your JSON file is syntactically correct and complete, please check your file against this JSON schema: 

Submissions that cannot be validated with the above schema cannot be evaluated and thus not be considered for the challenge!

To check your submission against the schema, you can use online validators like https://www.jsonschemavalidator.net, or your favourite programming language (e.g. python has a jsonschema package).

Paper submissions

We expect the participants to write either a full paper or a short paper (4 pages), describing their approach in detail. The paper needs to be submitted with the solution to grand-challenge. For docker submissions, the paper should be included in the output folder, named paper.pdf. A template for the paper format can be found here: LNCS

Docker submissions

We will provide you with an account to the Fraunhofer GitLab, where you will get a docker registry accessible only to you and the challenge organizers for the duration of the challenge. For access to the docker registry, please send an email to cada.challenge(at)gmail.com. When submitting a new image matching the criteria, it will be run on the test data and submitted to grand-challenge (this might take some time to appear in the leaderboard).

Just like for pre-processed results submissions, we will evaluate one per day. For the docker images, we will pull any new images matching the naming criteria (see 'Docker build' below) at CEST 00:00 for processing.

Docker build

We ask you to build your image accordingly to the exemplary build command:

docker build -t task2:$USERNAME-YYYYMMDD

where $USERNAME is your grand-challenge username and YYYYMMDD the date of your push to the docker registry (e.g. 20200528)

Docker run

Your containers will be run on our processing server with CUDA-based GPU support, but without internet connection. Here is an exemplary docker run command:

docker run -d --rm --network none --mount type=bind,source=data/input,target=/input,readonly --mount type=bind,source=results/$DOCKER-IMAGE-HASH,target=/output task2:testuser-20200528

where the /input folder contains the test images in compressed NIFTI format, as well as the SeedPoints.json file, which has the same format as the one you downloaded for the training data.

  • A001.nii.gz
  • A003.nii.gz
  • ...
  • SeedPoints.json

The /output folder is expected to contain the results-task2.json file (as specified above) and the produced masks in compressed NIFTI format. Additionally, the paper.pdf needs to be included.

  • A001_labeledMasks.nii.gz
  • A003_labeledMasks.nii.gz
  • ...
  • results-task2.json
  • paper.pdf