> ## Documentation Index > Fetch the complete documentation index at: https://docs.getlimina.ai/llms.txt > Use this file to discover all available pages before exploring further. # Process Text > Detect entities such as PII, PHI or PCI in the provided text strings using Private AI's entity detection engine. After entity detection, any entities found can be redacted, masked or replaced with AI-generated synthetic entities. ## OpenAPI ````yaml /openapi/privateai_4.3.0.json post /process/text openapi: 3.1.0 info: title: API Reference description: Private AI API Reference termsOfService: https://www.getlimina.ai/en/terms-of-use contact: url: https://www.getlimina.ai/en/contact-us email: info@getlimina.ai version: 4.3.0 servers: - url: https://api.private-ai.com/community description: Private AI Community API - url: https://api.private-ai.com/cloud description: Private AI Cloud API - url: http://localhost:8080 description: Local Server security: [] paths: /process/text: post: summary: Process Text description: >- Detect entities such as PII, PHI or PCI in the provided text strings using Private AI's entity detection engine. After entity detection, any entities found can be redacted, masked or replaced with AI-generated synthetic entities. operationId: process_text_process_text_post requestBody: content: application/json: schema: $ref: '#/components/schemas/ProcessTextRequest' examples: simple_deidentification: summary: Simple De-Identification value: text: - Hello John and Jane link_batch: false entity_detection: return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' synthetic_deidentification: summary: Synthetic PII value: text: - >- Hello, my name is May. I am the aunt of Pieter Parker. We live in Toronto, Canada. link_batch: false entity_detection: return_entity: true processed_text: type: SYNTHETIC synthetic_entity_accuracy: standard enabled_entity_types: summary: Enabled Entity Types value: text: - Hello, My name is Mike and I am 24 years old link_batch: false entity_detection: entity_types: - type: ENABLE value: - NAME return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' allow_list: summary: Allow List value: text: - >- Hello Xavier, I broke my right leg on the 31st. Id of PAI is PAI_ID-44444 yeah, my birthday, My id is 78549 and I'm waiting for my x-ray results. dr. zhang, mercer health centre. link_batch: false entity_detection: filter: - type: ALLOW pattern: (?i)PAI_ID-\d{5} return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' block_list: summary: Block List value: text: - >- Hello Xavier, I broke my right leg on the 31st. Id of PAI is PAI_ID-44444 yeah, my birthday, My id is 78549 and I'm waiting for my x-ray results. dr. zhang, mercer health centre. link_batch: false entity_detection: filter: - type: BLOCK entity_type: MY_5_DIGIT_PAI_ID pattern: (?i)PAI_ID-\d{5} return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' link_batch: summary: Link Batch value: text: - >- Hi, my name is Penelope, could you tell me your phone number please? - Sure, x234 - and your DOB please? - fourth of Feb nineteen 86 link_batch: true entity_detection: return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' masked_pii: summary: Masked PII value: text: - Hello John and Jane link_batch: false entity_detection: return_entity: true processed_text: type: MASK mask_character: '#' multiple_input: summary: Multiple Input value: text: - Hello John and Jane - Mark is 42 years old link_batch: false entity_detection: return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' non_unique_pii_markers: summary: Non Unique PII Markers value: text: - Hello John and Jane link_batch: false entity_detection: return_entity: true processed_text: type: MARKER pattern: '[BEST_ENTITY_TYPE]' required: true responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/ProcessTextResponse' '400': description: Bad Request content: application/json: schema: $ref: '#/components/schemas/UserErrorResponseModel' '500': description: Internal Server Error content: application/json: schema: $ref: '#/components/schemas/InternalErrorResponseModel' 4XX: description: Client Error content: application/json: schema: $ref: '#/components/schemas/UserErrorResponseModel' components: schemas: ProcessTextRequest: properties: text: items: type: string type: array title: Text description: >- UTF-8 encoded message(s) to process. E.g. `["My name is Adam"]` or `["I live at", "263 Spadina Av"]`. Request processing time increases linearly with input text length, therefore maximum length is dependent on provisioned hardware and any timeouts set by the user. Private AI has tested up to 500K characters on the CPU and GPU containers. link_batch: type: boolean title: Link Batch description: >- When set to `True`, the list of inputs provided in `text` will be processed together as a single input by the Private AI PII detection model. This shares context between the different inputs and is useful when processing a sequence of short inputs, such as an SMS chat log. This option should only be enabled when the inputs are related, otherwise PII detection performance could be degraded. default: false entity_detection: $ref: '#/components/schemas/PIIDetectionParams' description: >- This section contains a set of parameters to control the PII detection process. All fields have sensible default that can be changed for specific needs. project_id: type: string maxLength: 60 pattern: ^[a-zA-Z0-9\-_\:]*$ title: Project Id description: >- Used to categorize requests for reporting purposes. Limited to alphanumeric characters or the following special characters :_- default: main processed_text: oneOf: - $ref: '#/components/schemas/MarkerRedactedText' - $ref: '#/components/schemas/SyntheticRedactedText' - $ref: '#/components/schemas/MaskRedactedText' title: Processed Text description: >- This section allows the user to generate redacted (default), synthetic or masked text. default: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' marker_language: en coreference_resolution: heuristics discriminator: propertyName: type mapping: MARKER: $ref: '#/components/schemas/MarkerRedactedText' MASK: $ref: '#/components/schemas/MaskRedactedText' SYNTHETIC: $ref: '#/components/schemas/SyntheticRedactedText' additionalProperties: false type: object required: - text title: ProcessTextRequest description: API 3.0 Spec Definition ProcessTextResponse: items: $ref: '#/components/schemas/ProcessTextResponseItem' type: array title: ProcessTextResponse UserErrorResponseModel: properties: detail: anyOf: - $ref: '#/components/schemas/ErrorMessage' - $ref: '#/components/schemas/ValidationErrorModel' title: Detail description: >- The details of the error, usually relating to an input validation error type: object required: - detail title: UserErrorResponseModel InternalErrorResponseModel: properties: detail: type: string title: Detail description: >- The details of the error, usually relating to an unhandled processing error type: object required: - detail title: InternalErrorResponseModel PIIDetectionParams: properties: accuracy: $ref: '#/components/schemas/AccuracyMode' description: >- Selects the model used to identify PII in the input text. By default, the `high_automatic` accuracy model is used. This default automatically chooses either the high or high_multilingual model. Whilst the models used by the Private AI solution are highly optimized (~25X faster than a reference transformer implementation), in high-throughput cases it is possible to trade accuracy for speed by selecting either the `standard` or `standard_high` accuracy modes. Multilingual support can be enabled by using one of the multilingual models, namely `standard_high_multilingual` (GPU container only) and `high_multilingual`. The multilingual models process all supported languages including English, without the need to specify language. It is advisable to use the English-only models where possible, as they perform slightly better on English. Automatic Models can determine which model to use (English or Multilingual) depending on the languages detected, provided Multilingual models are available. More information on different accuracy modes can be found here: https://docs.getlimina.ai/configuration-and-operations/entity-detection-and-redaction/accuracy-modes entity_types: items: oneOf: - $ref: '#/components/schemas/EnableEntityTypeSelector' - $ref: '#/components/schemas/DisableEntityTypeSelector' discriminator: propertyName: type mapping: DISABLE: $ref: '#/components/schemas/DisableEntityTypeSelector' ENABLE: $ref: '#/components/schemas/EnableEntityTypeSelector' type: array title: Entity Types description: >- Controls which entity types and legislation sets are detected. See [Supported Entity Types](https://docs.getlimina.ai/entities/supported-entity-types) for the list of possible entities and legislation sets. By default, all entities are detected and removed. You can specify one of many selectors, which can be either an individual entity type such as `LOCATION_CITY` or a legislation like `GDPR`. `EnableEntityTypeSelector` selectors will add entity types to detect. On the contrary, `DisableEntityTypeSelector` selectors will ignore entities of the specified types. If only `DisableEntityTypeSelector` selectors are specified, they are assumed to be ignoring entity types from the entire supported list of entity types. filter: items: oneOf: - $ref: '#/components/schemas/AllowFilter' - $ref: '#/components/schemas/BlockFilter' - $ref: '#/components/schemas/AllowTextFilter' discriminator: propertyName: type mapping: ALLOW: $ref: '#/components/schemas/AllowFilter' ALLOW_TEXT: $ref: '#/components/schemas/AllowTextFilter' BLOCK: $ref: '#/components/schemas/BlockFilter' type: array title: Filter description: >- This field contains a list of filters expressed as regular expressions. These regular expressions can help users customize PII detection in a few ways. `ALLOW` filters can disabled redaction of entities containing specific text (e.g. a document id with format ID-1212 that is not sensitive). On the other hand, the `BLOCK` filters can augment the existing set of entities with custom entities (e.g. detecting sensitive medical code with format A18.32 as IDC_NUMBER). Finally, the `ALLOW_TEXT` filters can select a section of text in a document that should not be redacted (e.g. author names in scientific references or dates in an audit trail log). return_entity: type: boolean title: Return Entity description: >- Controls whether the PII list in the response contains the `text` field. Turning this off means that no sensitive PII is returned in the response. default: true enable_non_max_suppression: type: boolean title: Enable Non Max Suppression description: >- When set, if the best label of an entity is disabled then the entity will not be redacted. This could be useful to minimize false-positives when disabling an entity label like `ACCOUNT_NUMBER` that is closely related to another entity like `BANK_ACCOUNT`. Note that, the effect of that flag is difficult to predict when disabling labels like `LOCATION_COUNTRY` or `NAME_GIVEN` that are hierarchical by nature (i.e. a `NAME_GIVEN` entity is also a `NAME` entity). We don't recommend setting that flag to true when some of these hierarchical entities are disabled. default: false additionalProperties: false type: object title: PIIDetectionParams MarkerRedactedText: properties: type: type: string enum: - MARKER const: MARKER title: Type default: MARKER pattern: type: string title: Pattern description: >- Specifies a custom redaction marker format. The format must contain one and only one of these predefined keywords: `BEST_ENTITY_TYPE`, `ALL_ENTITY_TYPES`, `UNIQUE_NUMBERED_ENTITY_TYPE`, `UNIQUE_HASHED_ENTITY_TYPE`, which will be replaced as follows: The keyword `BEST_ENTITY_TYPE` will be replaced by the entity type. For example, `--[BEST_ENTITY_TYPE]--` will transform `My name is John` into `My name is --[NAME_GIVEN]--`. The keyword `UNIQUE_NUMBERED_ENTITY_TYPE` is similar, but it will also make the marker unique by appending an index at the end. For example `--[UNIQUE_NUMBERED_ENTITY_TYPE]--` yields `My name is --[NAME_GIVEN_1]--`. The keyword `ALL_ENTITY_TYPES` will be replaced with all the entity types applicable to the entity. For example `--[ALL_ENTITY_TYPES]--` yields `My name is --[NAME,NAME_GIVEN]--`. It is also possible to set this option via environment variable. See [Environment Variables](https://docs.getlimina.ai/configuration-and-operations/container-management/environment-variables). default: '[UNIQUE_NUMBERED_ENTITY_TYPE]' marker_language: $ref: '#/components/schemas/MarkerLanguage' description: >- The ISO 639 code of the language that the markers will be in. If set to `auto`, the detected language of the text is used. This feature is only available for `nl`, `en`, `fr`, `de`, `hi`, `it`, `ja`, `ko`, `zh`, `pt`, `ru`, `es`, `tl`, `uk`, if this feature is not available for the detected language, `en` will be used. More information on the translated labels used by the marker_language is found here: https://docs.getlimina.ai/entities/translated-entity-labels default: en coreference_resolution: $ref: '#/components/schemas/CoreferenceResolutionMode' description: >- (Experimental) Turns on the experimental coreference resolution. Specifies whether multiple instances of the same entity should be replaced with the same marker or not. For example, with `coreference_resolution` set: "Hi John and Rosha, John nice to meet you" -> "Hi [NAME_1] and [NAME_2], [NAME_1] nice to meet you". Without `coreference_resolution` set: "Hi [NAME_1] and [NAME_2], [NAME_3] nice to meet you". Note that this option is only available if the pattern field contains either the `UNIQUE_NUMBERED_ENTITY_TYPE` keyword or the `UNIQUE_HASHED_ENTITY_TYPE` keyword. default: heuristics additionalProperties: false type: object title: MarkerRedactedText SyntheticRedactedText: properties: type: type: string enum: - SYNTHETIC const: SYNTHETIC title: Type default: SYNTHETIC synthetic_entity_accuracy: $ref: '#/components/schemas/SyntheticPIIAccuracyMode' description: >- (Beta) Enable synthetic entity generation using the specified model. Currently this feature is in beta. Supported values are ['standard', 'standard_multilingual', 'standard_automatic']. Use the value `standard_automatic` to let the service automatically choose the best of `standard` or `standard_multilingual` options. The default value is `standard_automatic` if the container is equipped with the synthetic multilingual model, otherwise, the default value is `standard` coreference_resolution: anyOf: - $ref: '#/components/schemas/CoreferenceResolutionMode' - type: 'null' description: >- (Experimental) Turns on the experimental coreference resolution. Specifies whether multiple instances of the same entity should have the same generated synthetic entity or not. For example, with `coreference_resolution` set: "Hi John and Rosha, John nice to meet you" -> "Hi Harry and Alev, Harry nice to meet you". Without `coreference_resolution` set: "Hi John and Rosha, John nice to meet you" -> "Hi Harry and Alev, Sulav nice to meet you". If this option is not set, no attempt will be made to replace multiple mentions of the same entity with a unique value. Set this field to null if you want to disable coreference resolution for synthetic entities. default: heuristics additionalProperties: false type: object title: SyntheticRedactedText MaskRedactedText: properties: type: type: string enum: - MASK const: MASK title: Type default: MASK mask_character: type: string title: Mask Character description: >- Replaces redacted PII with the specified mask character. Note that this input may only be a single character default: '#' additionalProperties: false type: object title: MaskRedactedText ProcessTextResponseItem: properties: entities: items: $ref: '#/components/schemas/EntityItem' type: array title: Entities description: A list of all entities found in the text. entities_present: type: boolean title: Entities Present description: Returns `True` if the list of detected entities is not empty. characters_processed: type: integer title: Characters Processed description: The number of characters in all the text inputs. languages_detected: additionalProperties: type: number type: object title: Languages Detected description: >- A dictionary containing ISO 639-1 language labels and the likelihood of their detection in the request payload. processed_text: type: string title: Processed Text description: The redacted, synthetic or masked text. type: object required: - entities - entities_present - characters_processed - languages_detected - processed_text title: ProcessTextResponseItem ErrorMessage: type: string title: ErrorMessage description: An error message ValidationErrorModel: properties: description: type: string title: Description description: User-friendly error message description. loc: items: anyOf: - type: string - type: integer type: array title: Loc description: >- The location where the error has occurred. This could be location of a character if the error is a parsing error, or it could be a value in the request if there is a validation error. msg: type: string title: Msg description: The error message. type: type: string title: Type description: The type of error that has been encountered. type: object required: - loc - msg - type title: ValidationErrorModel AccuracyMode: type: string enum: - standard - standard_high - standard_high_multilingual - standard_high_automatic - high - high_multilingual - high_automatic title: AccuracyMode description: >- Selects the model used to identify PII in the input text. By default, the "high_automatic" accuracy model is used. This default automatically chooses either the high or high_multilingual model. Whilst the models used by the Private AI solution are highly optimized (~25X faster than a reference transformer implementation), in high-throughput cases it is possible to trade accuracy for speed by selecting either the "standard" or "standard_high" accuracy modes. Multilingual support can be enabled by using one of the multilingual models, namely "standard_high_multilingual" (GPU container only) and "high_multilingual". The multilingual models process all supported languages including English, without the need to specify language. It is advisable to use the English-only models where possible, as they perform slightly better on English. Automatic Models can determine which model to use (English or Multilingual) depending on the languages detected, provided Multilingual models are available. More information on different accuracy modes can be found here: https://docs.getlimina.ai/configuration-and-operations/entity-detection-and-redaction/accuracy-modes EnableEntityTypeSelector: properties: type: type: string enum: - ENABLE const: ENABLE title: Type default: ENABLE value: items: type: string type: array title: Value description: >- A list of entity types to detect and remove. See [Supported Entity Types](https://docs.getlimina.ai/entities/supported-entity-types) for a complete list of entity types. This can also be one or many legislations. We currently support these legislations ['APPI', 'APPI_SENSITIVE', 'CORE_ENTITIES', 'CPRA', 'GDPR', 'GDPR_SENSITIVE', 'HEALTH_INFORMATION', 'HIPAA_SAFE_HARBOR', 'LIDI', 'PCI', 'QUEBEC_PRIVACY_ACT', 'CCI', 'NUMERICAL_EXCL_PCI']. additionalProperties: false type: object title: EnableEntityTypeSelector DisableEntityTypeSelector: properties: type: type: string enum: - DISABLE const: DISABLE title: Type default: DISABLE value: items: type: string type: array title: Value description: >- A list of entity types to ignore. See [Supported Entity Types](https://docs.getlimina.ai/entities/supported-entity-types) for a complete list of entity types. additionalProperties: false type: object title: DisableEntityTypeSelector AllowFilter: properties: type: type: string enum: - ALLOW const: ALLOW title: Type description: >- Entities with text matching the provided regex pattern will be discarded. It is also possible to set this option via environment variable. See [Environment Variables](https://docs.getlimina.ai/configuration-and-operations/container-management/environment-variables). default: ALLOW pattern: type: string title: Pattern description: >- A python regex pattern (e.g. `r"^ID-[\d]{4}$"`). This pattern will be used to match the entity text. If it matches zero or more characters at the **beginning of the entity text**, the entity will be ignored. Be sure to use the end of string character `$` if you want to only allow entities when the entirety of the text matches. It is also important to note that regex patterns may require escaping when used in JSON objects. additionalProperties: false type: object required: - pattern title: AllowFilter BlockFilter: properties: type: type: string enum: - BLOCK const: BLOCK title: Type description: >- The block feature allows you to extend the functionality of the Private AI models by using regular expressions. This way, you can define a Python regex pattern that will be used to identify additional tokens with the given PII label. Several block list filters can be specified with their own regex pattern. Lastly,for supported labels, if you would like the model to pick up only the tokens from the block list, you can use the enabled entity type feature together with the block list feature. This can be done by defining a list of enabled entity types and not including the supported label you are adding to the block list. For example, if you would like the label `ORGANIZATION` to only pick up Microsoft, you can define the enabled entity types as `[{"type":"ENABLE", "value": "NAME"}, {"type": "ENABLE", "value": "LOCATION"}, {"type": "ENABLE", "value": "AGE"}, ...]` (and omitting `ORGANIZATION`) and the block list as `[{"type": "BLOCK", "entitiy_type": "ORGANIZATION", "pattern": "Microsoft"}]`. default: BLOCK entity_type: type: string title: Entity Type description: >- Name of the custom entity type. It can either be a completely new entity type such as `CUSTOM_ID` or an existing entity, such as `NAME`. pattern: type: string title: Pattern description: >- This is a pattern to match in the text. This feature uses regex patterns, you can either pass a word (e.g. the, word, custom, etc.) or you can pass a valid Python regex pattern. It is important to note that regex patterns may require escaping when used in JSON objects. To give an example, if you would like to send the regex pattern `r"\b\w{4}\b"` which will catch every 4-character word, you need to send it as `"\\b\\w{4}\\b"`. A complete JSON grammar is found here: https://www.json.org/json-en.html. More information on how to write a python regex is found here: https://docs.python.org/3/library/re.html It is important to note also that only non-overlapping matches are returned. threshold: type: number title: Threshold description: >- This is defining a likelihood threshold for custom entity. This likelihood is compared against the predicted model likelihood and if it is greater then the custom entity is outputted instead of the model predicted entity. By default this threshold is set to 1.0 which will ensure that the blocked entities will always be preferred over a matching model predicted entity. This can be any value between 0 and 1. default: 1 additionalProperties: false type: object required: - entity_type - pattern title: BlockFilter AllowTextFilter: properties: type: type: string enum: - ALLOW_TEXT const: ALLOW_TEXT title: Type description: >- Input text matching the provided regex pattern will not be redacted. It is currently not possible to set this option via environment variable. default: ALLOW_TEXT pattern: type: string title: Pattern description: >- A python regex pattern (e.g. `r"^ID-[\d]{4}$"`). This pattern will be used to match the input text. Entities detected **inside the matched text** will be ignored. Note that capturing groups can be used in the regex pattern. If present, only the text matching a capturing group will be left unredacted. It is also important to note that regex patterns may require escaping when used in JSON objects. additionalProperties: false type: object required: - pattern title: AllowTextFilter MarkerLanguage: type: string enum: - auto - nl - en - fr - de - hi - it - ja - ko - zh - pt - ru - es - tl - uk title: MarkerLanguage CoreferenceResolutionMode: type: string enum: - heuristics - combined - model_prediction title: CoreferenceResolutionMode SyntheticPIIAccuracyMode: type: string enum: - standard - standard_multilingual - standard_automatic title: SyntheticPIIAccuracyMode EntityItem: properties: processed_text: type: string title: Processed Text description: >- Either the redaction marker, hash marker or synthetic entity text in `processed_text` corresponding to the entity. text: type: string title: Text description: >- The entity text. When the `return_entity` option is set to `False`, this returns a null string location: $ref: '#/components/schemas/TextLocation' default: >- Location object containing the start-end character indexes of a given entity in the input text and in the processed text. best_label: type: string title: Best Label description: The entity label with the highest likelihood. labels: additionalProperties: type: number type: object title: Labels description: >- A dictionary containing any other labels found, together with associated likelihoods. Note that these are not strictly probabilities and do not sum to 1, as an entity can have multiple types. Note that the likelihoods have also been thresholded, so no additional thresholding is necessary. additionalProperties: false type: object required: - processed_text - text - best_label - labels title: EntityItem TextLocation: properties: stt_idx: type: integer title: Stt Idx description: Start character index of the entity in the original text. end_idx: type: integer title: End Idx description: >- Index of the character immediately following the entity, such that end_idx - stt_idx = number of characters in the entity. stt_idx_processed: type: integer title: Stt Idx Processed description: Start character index of the entity in the processed text. end_idx_processed: type: integer title: End Idx Processed description: >- Index of the character immediately following the entity in the processed text. type: object required: - stt_idx - end_idx - stt_idx_processed - end_idx_processed title: TextLocation description: Start and end indices of the entity in a text. ````