iaa_od.visualisation.show_fatigue_evaluation

Functions

show_fatigue_data_with_agreement(sessions)

Visualise the fatigue evaluation of annotators by plotting Alpha values against normalised timestamps for each GT.

show_fatigue_data_with_granularity(sessions, ...)

Visualise annotator fatigue through per-image granularity instead of Krippendorff's alpha.
iaa_od.visualisation.show_fatigue_evaluation.show_fatigue_data_with_agreement(sessions)[source]

Visualise the fatigue evaluation of annotators by plotting Alpha values against normalised timestamps for each GT. The graph also shows a mean Alpha value across all GTs to provide an overall trend of annotator fatigue over time. :type sessions: dict[str, dict[int, list[AnnotationTimestamp]]] :param sessions: A dictionary where keys are GT names and values are dictionaries mapping session IDs to lists of AnnotationTimestamp objects. :type sessions: dict[str, dict[int, list[AnnotationTimestamp]]]

Return type:

None

iaa_od.visualisation.show_fatigue_evaluation.show_fatigue_data_with_granularity(sessions, granularities)[source]

Visualise annotator fatigue through per-image granularity instead of Krippendorff’s alpha.

For each GT, this plots the per-image count granularity (top) and size granularity (bottom) against the normalised timestamps, one line per annotation session, together with a mean trend line across all sessions. Under the dataset-wide benchmark normalisation used by per_image_fatigue_granularities, a value of 1.0 is the benchmark (median) annotator; values drifting above or below 1.0 over the course of a session indicate the annotator shifting their granularity strategy (e.g. toward larger, group-level boxes as fatigue sets in: lower count granularity, higher size granularity).

Parameters:

  • sessions (dict[str, dict[int, list[AnnotationTimestamp]]]) – A dictionary where keys are GT names and values are dictionaries mapping session IDs to lists of AnnotationTimestamp objects, as returned by evaluate_annotator_fatigue.

  • granularities (dict[str, Granularities]) – A mapping from image filename to that image’s Granularities, as returned by per_image_fatigue_granularities.

Return type:

None