Skip to main content
In order to run the example code in this guide, please sign up for your free test api key here.
In addition to de-identification and redaction, Limina also supports entity detection and validation. The analyze/text route described below is an essential tool for exploring and structuring your data as well as creating statistics around your data. In this guide, we demonstrate how to use the analyze/text endpoint introduced in 4.1 to return the analysis results of the detected entities, with examples of how these results can be used to meet your own use cases.

​
Analyze entities in text (new in 4.1)

The analyze/text route returns a list of detected entities along with the formatted text for each entity and a description of its subtypes. In this guide, we provide payloads to the Limina’s analyze/text REST API route and document the associated responses. To better illustrate how this information can be used, we proceed by giving a series of common use cases.

​
Validation and custom redaction of credit card numbers

Some numerical entities integrate a checksum in their values. This checksum is used to confirm the entity’s validity and to minimize the chance of error during transcription. This is the case for credit card numbers, which must satisfy the Luhn algorithm. The analyze/text route implements this algorithm on top of the NER model detection. This provides an additional safeguard by ensuring that the detected number is indeed a valid credit card number. Let’s look at three specific examples including credit card numbers. 
{
  "text": [
    "Okay, hang on just a second because I got to get it. Okay, it is 6578-7790-4346-2237. Expiration. 1224.",
    "All right, I'm ready. 800 678-457-7896. Expiration is one. 224.",
    "CC_type: Diners Club International RuPay Visa JCB Amex CCN: 30569309025904 4242424242424242 4222222222222 6172873484776530 378282246310005 CC_CVC: 480 902 182 765 143 CC_Expiredate: 5/28 6/67 12/67 11/29 9/70"
  ],
  "locale": "en-US",
  "entity_detection": {
    "accuracy": "high",
    "entity_types": [
      {
        "type": "ENABLE",
        "value": ["CREDIT_CARD"]
      }
    ]
  }
}
The above request contains two fields, text and entity_detection, that are shared by the analyze/text, the ner/text and the process/text routes. The text field contains the text to analyze and the entity_detection field contains the NER configurations (e.g., the list of entities to detect). One last field in the request, locale, is unique to the analyze/text request. The locale field is used as a hint to the analyzer to help parse dates and other locale-dependent entities. For example, setting locale to en-US will force the analyzer to interpret the date 12-10-2020 as December 10, 2020 instead of October 12, 2020. Several example of values that can take these fields are provided below. The full response above is a mouthful, so let’s look at the first example’s response in more detail.
JSON Response with CREDIT_CARD entity
{
  "entities": [
    {
      "text": "6578-7790-4346-2237",
      "location": {
        "stt_idx": 65,
        "end_idx": 84
      },
      "best_label": "CREDIT_CARD",
      "labels": {
        "CREDIT_CARD": 0.9022786834023215
      },
      "analysis_result": {
        "formatted": "6578 7790 4346 2237",
        "subtypes": [],
        "validation_assertions": [
          {
            "provider": "luhn",
            "status": "invalid"
          }
        ]
      }
    }
  ],
  "entities_present": true,
  "characters_processed": 103,
  "languages_detected": {
    "en": 0.9202778935432434
  }
}
The response contains three main parts:
  • the entity information including its text and its location. Those fields are shared across other routes including the ner/text and process/text routes and have the same use.
  • the formatted text of the entity. This field is unique to the analyze/text route and provides a “standard” format for the entity. This can facilitate the introduction of post-processing logic on detected entities. The formats are described in the following table.
Entity TypeFormatExample
CREDIT_CARDspace-separated groups of 3 to 5 digits6578 7790 4346 2237
DATEISO-86012025-03-20T18:00:00+00:00
DOBISO-86012025-03-20
AGEdecimal numeral12
All other entity typesno formatting-
  • a list of validation assertions on the entity, which is also unique to the analyze/text route. It contains a list of objects that are specific to the entity being detected. In this example, the provider is the Luhn algorithm that was run on the credit card number and the result of the algorithm is provided as part of the status field. Currently, only credit card numbers contain validation assertions but more assertion providers will be added in the future.
The analysis result of this first example can be summed up in the following way. The credit card was successfully parsed and the parsed result is placed in the formatted field. However, although the number matches the credit card number format, the Luhn check failed on the number, so it is not a valid credit card number. This could be the result of a transcription error, for example. The information included in the analysis result allows the creation of custom redaction of entities, using the post-processing framework, as shown in this section.

​
Date shifting and custom redaction of dates

Dates are one type of PII that is encountered in almost every dataset. Redaction is one way to ensure that sensitive dates do not create privacy issues. However, fully redacting dates often reduces the utility of the redacted data. For dates, it is often preferable to use other obfuscation methods that preserve their utility. Two well-known techniques are date shifting and date bucketing. Let’s consider three examples containing dates. 
{
  "text": [
    "$MDT $MRK $QRVO $TSS & 5 more stock picks for LONG swings:  https://t.co/CbkieXxqoR (July 10 2018) https://t.co/eit53RUY4g",
    "Short sale volume (not short interest) for $KBE on 2018-07-09 is 42%. https://t.co/7pWbgjJ8Ag $FOXA 38% $TVIX 34% $LITE 54% $HIG 60%",
    "$WLTW high OI range is 160 to 155 for option expiration 07/20/2018 #options https://t.co/BnVElKBKkJ"
  ],
  "locale": "en-US",
  "entity_detection": {
    "entity_types": [
      {
        "type": "ENABLE",
        "value": ["DATE", "DOB", "DAY", "MONTH", "YEAR"]
      }
    ]
  }
}
Let’s look at one specific date entity in the above response.
Response body with formatted date entities
{
  "text": "July 10 2018",
  "location": {
    "stt_idx": 89,
    "end_idx": 101
  },
  "best_label": "DATE",
  "labels": {
    "DATE": 0.9400081038475037,
    "MONTH": 0.3111259341239929,
    "DAY": 0.31207050879796344,
    "YEAR": 0.29245950778325397
  },
  "analysis_result": {
    "formatted": "2018-07-10T00:00:00",
    "subtypes": [
      {
        "text": "10",
        "formatted": "10",
        "label": "DAY",
        "location": {
          "stt_idx": 94,
          "end_idx": 96
        }
      },
      {
        "text": "July",
        "formatted": "7",
        "label": "MONTH",
        "location": {
          "stt_idx": 89,
          "end_idx": 93
        }
      },
      {
        "text": "2018",
        "formatted": "2018",
        "label": "YEAR",
        "location": {
          "stt_idx": 97,
          "end_idx": 101
        }
      }
    ],
    "validation_assertions": []
  }
}
Many pieces of information are accessible from the analysis_result object. First, it is possible to access the formatted date “2018-07-10T00:00:00” from the field analysis_result.formatted. If you plan to implement logic on the dates found in the text, it might be easier to access the formatted dates rather than the original, non-standard date formats (e.g., “July 10 2018”). Also, it is possible to directly access the day, month, and year of the date entity via the response fields in analysis_result.subtypes. This information can be used to partially redact or to bucketize dates.
An example of redacting the day and month but keeping the year is provided in the custom redaction of dates guide.

​
Age bucketing and custom redaction of numbers

Similar to dates, it is possible to analyze ages and other numerical entities to create custom redaction. Consider these two examples. 
{
  "text": [
    "A 32-year old Black female German citizen living in Germany wants to travel to the United States for leisure.",
    "West Point Public School division provides school-based preschool services for children from two through nine years of age who are children at risk and children with identified disabilities or delays."
  ],
  "link_batch": false,
  "locale": "en-US",
  "entity_detection": {
    "entity_types": [
      {
        "type": "ENABLE",
        "value": ["AGE"]
      }
    ]
  }
}
Using the Limina python client, one can use the above analyze/text response to bucketize ages, as shown here.

​
Custom redaction of addresses

The GDPR and other privacy legislations impose strict requirements regarding the redaction of addresses. In the following scenario, we demonstrate how to partially redact an address by leaving only the less sensitive characters of a zip/postal code and removing all other address information (e.g., civic number, street name, and so on). 
{
  "text": [
    "Please deliver this to 45, Clybaun Heights, Galway City, Ireland H91 AKK3",
    "3255 M-A-D-D-A-M-S street, huntington, west virginia is his birthplace",
    "My favorite city is San Francisco, California 94110, United States, 37.7749° N, 122.4194° W"
  ],
  "locale": "en-US"
}
The above request contains three examples containing addresses. The corresponding analyze/text response contains the result of the analysis. This response, along with the corresponding Limina client post-processing code, can be used to mask street addresses, in order to hide the most sensitive information.

​
Relation detection

Relation detection refers to the broader natural language processing (NLP) capability of understanding how entities in a text are connected. While entity recognition tells us what the entities are (e.g., a person’s name, a company, a location), relation detection tells us how those entities are related. Relation detection covers tasks like coreference resolution and relation extraction, both of which are supported, and together provide a deeper understanding of unstructured text. The analyze/text route can be used to configure relation detection by using the optional relation_detection field in the request.

​
Coreference Resolution

Coreference resolution is the task of identifying different entity mentions in a given text that refer to the same real-world entity. The relation_detection field offers a configurable option for coreference resolution:
  • coreference_resolution: Specifies the method for identifying coreferential entities:
    • heuristics: Uses rule-based methods
    • model_prediction: Uses machine learning models
    • combined: Uses both approaches
{
  "text": [
    "Nikola Jokić (Serbian Cyrillic: Никола Јокић, pronounced [nǐkola jôkitɕ] ⓘ; born February 19, 1995) is a Serbian professional basketball player who is a center for the Denver Nuggets of the National Basketball Association (NBA). Jokić was born in the city of Sombor in the northern part of Serbia. He grew up in a cramped two-bedroom apartment that housed him and his two brothers."
  ],
  "entity_detection": {
    "accuracy": "high"
  },
  "locale": "en-US",
  "relation_detection": {
    "coreference_resolution": "model_prediction"
  }
}
The response includes a key element for each entity:
  • coreference_id: A unique identifier added to each entity that groups coreferential entities under a common label. This behavior matches the /process/text endpoint when processed_text is set to MARKER and coreference resolution is applied. For example, “Nikola Jokić”, “Никола Јокић”, “nǐkola jôkitɕ”, and “Jokić” all share the same coreference_id (56c15276-33da-4726-bc81-369074049222), indicating that they refer to the same person.
For an example of how to use the coreference information from the API to replace all mentions of a person with their initials, see the custom redaction of coreferenced names example.

​
Relation Extraction

Relation extraction is the task of identifying meaningful relations between entities in text, such as person-to-person or person-to-location links. It helps unlock document-level understanding by connecting pieces of information and making it easier to de-identify related data. Let’s look at an example:
Relation Extraction Sample Text
Nessa Jonsson was born and raised in Sweden. Her father, Erik, emigrated to the United States when she was a baby. He died in 1980.
In the text above, there are four entities:
  • Nessa Jonsson (NAME)
  • Sweden (LOCATION_COUNTRY)
  • Erik (NAME_GIVEN)
  • the United States (LOCATION_COUNTRY)
  • 1980 (DATE_INTERVAL)
Relation extraction can be used to identify the semantic connections between those entities, such as:
  • Nessa Jonsson is born in Sweden
  • Nessa Jonsson is the daughter of Erik
  • Erik is the father of Nessa Jonsson
  • Erik lived in the United States
  • Erik died in 1980
Relation extraction plays a key role in document understanding by uncovering how entities are connected, which enables systems to move toward structured, contextualized information. In domains like healthcare and finance, this unlocks the potential of unstructured text by identifying relationships like family connections, places of origin, or dates of birth.

​
Limina and Relation Extraction (Beta)

Limina’s de-identification service offers the ability to use relation extraction on its analyze/text endpoint. Relation extraction is currently implemented on top of both the named entity recognition (NER) and the coreference resolution models. It is, therefore, limited to predicting relations between clusters of coreferenced entities. Currently, the system supports a single generalized relation type: RELATED_TO, which is used to capture all of the supported semantic relations between a person and another entity:
  • Kinship - a relation between two NAMEs (or other variants, e.g. NAME_GIVEN) indicating family or close personal relationships between individuals. These may include parent-child, siblings, spouses, etc. A kinship relation is always bi-directional.
  • Place of birth - a relation between NAME and LOCATION entities, indicating the location where the person was born. This can refer to a city, state, country, or region.
  • Citizenship - a relation between NAME and LOCATION or ORIGIN entities, indicating nationality or legal citizenship of the person.
  • Origin - a relation between NAME and ORIGIN entities, indicating the country a person originally comes from, reflecting ancestry or cultural background rather than legal status or birthplace.
  • Date of birth - a relation between NAME and DOB entities, indicating birthdate.
  • Date of death - a relation between NAME and DATE or DATE_INTERVAL entities, indicating the date of death of a person.
For the example above, the system will extract the following relations:
  • Nessa Jonsson → RELATED_TO → Sweden
  • Nessa Jonsson → RELATED_TO → Erik
  • Erik → RELATED_TO → Nessa Jonsson
The relation extraction feature can be enabled as part of the analyze/text endpoint by setting the field enable_relation_extraction to true.
  • enable_relation_extraction: Controls whether relation extraction is performed during analysis.
    • true: Enables relation extraction
    • false (default): Disables relation extraction
Relation Extraction and Coreference ResolutionRelation extraction relies on coreference resolution to group people mentions in text. Make sure a non-null value is set for coreference_resolution before setting enable_relation_extraction to true.
Here is an example of how to enable relation extraction in your request using the analyze/text endpoint. Notice the enable_relation_extraction field within the relation_detection object. 
{
  "text": [
    "Nessa Jonsson was born on March 17, 1995 in Sweden and currently resides there. Her sister, Erika, has a history of hypertension."
  ],
  "entity_detection": {
    "accuracy": "high"
  },
  "locale": "en-US",
  "relation_detection": {
    "coreference_resolution": "model_prediction",
    "enable_relation_extraction": true
  }
}
With relation extraction enabled, each entity in the response now contains an additional field capturing its relations:
  • relations: A list of extracted relations involving the entity. Each relation object includes:
    • coreference_id: The ID of the related entity from the coreference_id field of another entity in the response.
    • label: The type of relation detected. Currently, only one relation is supported, the generic RELATED_TO relation.
Limitations The relation extraction model is provided as an experimental feature and is not intended for production use. It currently supports English text and is constrained to inputs of up to 1024 tokens. Any text beyond this limit will be ignored during processing. Relation predictions may be inaccurate or missed, particularly in complex contexts where related entities occur far apart within the text.