Configurations

Simplified workflow

Direct Submit

Using the classical annotation workflow, when an annotator works on a task, a series of actions are necessary for creating a new annotation and submitting it as ground truth:

1. Create the completion
2. Save the completion
3. Submit the completion
4. Confirm submission
5. Load next task

This process is adapted for more complex workflows and large tasks. For simple projects with smaller tasks, Annotation Lab now offers a simplified workflow. Annotators can submit a completion with just one click.

The Project Owner/Manager can activate this option from the Settings dialog (Customize Labels) in the Configuration step of the Setup page. Once enabled, annotators can see the submit button on the labeling page. A second option is available on the same dialog for Project Owner/Manager: Serve next task after completion submission. Once enabled, annotators can see the next task on the labeling page after submitting the completion for the current task.

Note:

1. Annotator can Save/Update completion using CTRL+Enter
2. Annotator can Submit completion using ALT+Enter

Accept Prediction

When predictions are available for a task, Annotator can accept the predictions with just one click and navigate automatically to the next task. When users click on Accept Prediction, a new completion is created based on the prediction, then submitted as ground truth, and the next task in line (assigned to the current annotator/reviewer and with Incomplete or In Progress status) is automatically served.

NOTE: Press backspace key (on windows) or delete key (on mac) to delete the selected relation from the labeling editor or use the delete action icon on the Relations widget.

Labeling editor Settings

The labeling editor offers some configurable features. For example, you can modify the editor’s layout, show or hide predictions, annotations, or the confidence panel, show or hide various controls and information. It is also possible to keep a label selected after creating a region, display labels on bounding boxes, polygons and other regions while labeling, and show line numbers for text labeling.

Enable labeling hotkeys

This option enables/disable the hotkeys assigned to taxonomy labels to use the hotkeys during the annotation process.

Show hotkey tooltips

This option shows/hides the hotkey and tooltip on the taxonomy label and the control buttons. Enable labeling hotkeys must be enabled for this option to work.

Show labels inside the regions

When you enable this option, the labels assigned to each annotated region are displayed on the respective region.

Keep label selected after creating a region

This option helps users quickly annotate sequences of the same label by keeping the label selected after the annotation of a region.

With the option unchecked:

With the option checked:

Select regions after creating

This option keeps the annotated region selected after annotation. In this way, it will be easier for users to quickly change the assigned label for the last selected region if necessary.

Show line numbers for Text

This option adds line numbers to the text content to annotate in the labeling editor.

Label all occurrences of selected text

When checked, this option allow users to annotate all occurences of a text in the current task in one step.

Labeling editor Customizations

The Labeling editor is highly customizable. Project Owners and Managers can change the layout of their projects based on their needs.

Search filter for a large number of labels

When a project has a large number of NER/Assertion labels in the taxonomy, the display of the taxonomy takes a lot of screen space, and it is difficult for annotators to navigate through all labels. To tackle this challenge, Annotation Lab supports search for labels in NER projects (an autocomplete search option).

To add the search bar for NER Labels or Choices, use the Filter tag as shown in the following XML configuration.

<Filter />

<View>
*** enclose labels tags here ***
</View>

<View>
*** enclose text tags here ***
</View>

Parameters:

The following parameters/attributes can be used within the Filter tag.

Usage Example:

<Filter placeholder="Quick Filter"/>

For obtaining the above display on a NER project, the config should look as follows:

<View>
  <Filter name="fl" toName="label" hotkey="shift+f" minlength="1" />

  <Labels name="label" toName="text">
    <Label value="CARDINAL" model="ner_onto_100" background="#af906b"/>
    <Label value="EVENT" model="ner_onto_100" background="#f384e1"/>
    ...
    <Label value="LANGUAGE" model="ner_onto_100" background="#c0dad2"/>
  </Labels>

  <Text name="text" value="$text"/>
</View>

Notice how users can search for the desired label using the filter bar:

Resizable label and text container

While annotating longer text documents annotators may need to scroll to the top of the document for selecting the label to use, and then scroll down to create a label. Also, if the text is large, annotators have to scroll to a certain section because the textbox size is fixed. In those cases, the annotation experience can be improved by creating a scrollable labeling area and textbox area.

To add the scroll bar, the View tag with a fixed height and overflow-y:scroll style property can be used as shown in the following XML config structure:

<View style="background:white; height: 100px; overflow-y:scroll; resize:vertical; position:sticky; top:0;">
*** enclose labels tags here ***
</View>

<View style="resize:vertical; margin-top:10px; max-height:400px; overflow-y:scroll;">
**** enclose text tags here**
</View>

Once it has been added and saved to the Project Configuration, the scroll bar should be visible.

Example

Using the following Project Configuration

<View>
  <Filter name="fl" toName="label" hotkey="shift+f" minlength="1" />
  <View style="background:white; height: 100px; overflow-y:scroll; resize:vertical; position:sticky; top:0;">
    <Labels name="label" toName="text">
      <Label value="CARDINAL" model="ner_onto_100" background="#af906b"/>
      <Label value="EVENT" model="ner_onto_100" background="#f384e1"/>
      <Label value="WORK_OF_ART" model="ner_onto_100" background="#0fbca4"/>
      ...
      <Label value="LANGUAGE" model="ner_onto_100" background="#c0dad2"/>
    </Labels>
  </View>
  <View style="resize:vertical; margin-top:10px; max-height:400px; overflow-y:scroll;">
    <Text name="text" value="$text"></Text>
  </View>
</View>

we’ll obtain the output illustrated below:

Comments on the Labeling Page

Since version 4.10.0, NLP Lab offers enhanced comment feature for labeling pages, enabling users to easily add, update, and delete comments within labeling pages. This feature enhances the communication between the annotators, improves work efficiency and enhances productivity. In order to use this feature, there is a New Burger menu at the top right corner of the labeling page. The dropdown through this menu allows users to add, update, and delete comments.

Tags from the Labeling Screen

From version 4.10 onwards, NLP Lab introduces an enhanced tags feature for labeling pages. This addition offers users a convenient method to create, attach, and delete tags directly on the labeling page. It greatly enhances organization and boosts productivity by streamlining task management, granting users greater flexibility in classifying and monitoring their labeled data. Similar to the aforementioned comment feature, this can also be accessed from the burger menu at the top right corner of the labeling page. From the dropdown, select “Assign Tags”.

Once the users select the tag/tags, they will be displayed against the tasks on the tasks page.

Toggle Preview Window

Label configuration editor and Preview Window covers 50/50 part of the screen. It can make editing larger XML configurations difficult. For a better editing experience, we can use the Toggle Preview Window button to have the editor use full screen width.

Switch Role

For users having multiple roles (Annotator/Reviewer/Manager) the labeling page can get confusing. Switch Role filter present on the top-right corner can help address this problem. This filter was introduced in Annotation Lab from version 2.6.0, previously refered to as View As filter. When selecting Annotator option, the view changes to facilitate annotating the task. Similar changes to the view applies when switching to Reviewer or Manager option. The selection persists even when the tab is closed or refreshed.