> ## Documentation Index > Fetch the complete documentation index at: https://docs.getlimina.ai/llms.txt > Use this file to discover all available pages before exploring further. # Process Text > Detect entities such as PII, PHI or PCI in the provided text strings using Private AI's entity detection engine. After entity detection, any entities found can be redacted, masked or replaced with AI-generated synthetic entities. ## OpenAPI ````yaml /openapi/privateai_3.5.0.json post /v3/process/text openapi: 3.1.0 info: title: API Reference description: Private AI API Reference termsOfService: https://www.private-ai.com/terms-of-use/ contact: url: https://www.private-ai.com/contact/ email: info@private-ai.com version: 3.5.0 license: name: Private AI url: https://www.private-ai.com servers: - url: https://api.private-ai.com/deid description: Private AI Demo Server - url: http://localhost:8080 description: Local Server security: [] paths: /v3/process/text: post: summary: Process Text description: >- Detect entities such as PII, PHI or PCI in the provided text strings using Private AI's entity detection engine. After entity detection, any entities found can be redacted, masked or replaced with AI-generated synthetic entities. operationId: process_text_v3_process_text_post parameters: - name: x-api-key in: header required: false schema: type: string default: '' title: X-Api-Key requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/ProcessTextRequest' examples: simple_deidentification: summary: Simple De-Identification value: text: - Hello John and Jane link_batch: false entity_detection: accuracy: high return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' synthetic_deidentification: summary: Synthetic PII value: text: - >- Hello, my name is May. I am the aunt of Pieter Parker. We live in Toronto, Canada. link_batch: false entity_detection: accuracy: high return_entity: true processed_text: type: SYNTHETIC synthetic_entity_accuracy: standard preserve_relationships: true enabled_entity_types: summary: Enabled Entity Types value: text: - Hello, My name is Mike and I am 24 years old link_batch: false entity_detection: accuracy: high entity_types: - type: ENABLE value: - NAME return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' allow_list: summary: Allow List value: text: - >- Hello Xavier, I broke my right leg on the 31st. Id of PAI is PAI_ID-44444 yeah, my birthday, My id is 78549 and I'm waiting for my x-ray results. dr. zhang, mercer health centre. link_batch: false entity_detection: accuracy: high filter: - type: ALLOW pattern: PAI_ID-\d{5} return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' block_list: summary: Block List value: text: - >- Hello Xavier, I broke my right leg on the 31st. Id of PAI is PAI_ID-44444 yeah, my birthday, My id is 78549 and I'm waiting for my x-ray results. dr. zhang, mercer health centre. link_batch: false entity_detection: accuracy: high filter: - type: BLOCK entity_type: MY_5_DIGIT_PAI_ID pattern: PAI_ID-\d{5} return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' link_batch: summary: Link Batch value: text: - >- Hi, my name is Penelope, could you tell me your phone number please? - Sure, x234 - and your DOB please? - fourth of Feb nineteen 86 link_batch: true entity_detection: accuracy: high return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' masked_pii: summary: Masked PII value: text: - Hello John and Jane link_batch: false entity_detection: accuracy: high return_entity: true processed_text: type: MASK mask_character: '#' multiple_input: summary: Multiple Input value: text: - Hello John and Jane - Mark is 42 years old link_batch: false entity_detection: accuracy: high return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' non_unique_pii_markers: summary: Non Unique PII Markers value: text: - Hello John and Jane link_batch: false entity_detection: accuracy: high return_entity: true processed_text: type: MARKER pattern: '[BEST_ENTITY_TYPE]' responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/ProcessTextResponse-Input' '400': content: application/json: schema: $ref: '#/components/schemas/UserErrorResponseModel' description: Bad Request '500': content: application/json: schema: $ref: '#/components/schemas/InternalErrorResponseModel' description: Internal Server Error 4XX: content: application/json: schema: $ref: '#/components/schemas/UserErrorResponseModel' description: Client Error components: schemas: ProcessTextRequest: properties: text: items: type: string type: array title: Text description: >- UTF-8 encoded message(s) to process. E.g. `["My name is Adam"]` or `["I live at", "263 Spadina Av"]`. Request processing time increases linearly with input text length, therefore maximum length is dependent on provisioned hardware and any timeouts set by the user. Private AI has tested up to 500K characters on the CPU and GPU containers. link_batch: type: boolean title: Link Batch description: >- When set to `True`, the list of inputs provided in `text` will be processed together as a single input by the Private AI PII detection model. This shares context between the different inputs and is useful when processing a sequence of short inputs, such as an SMS chat log. This option should only be enabled when the inputs are related, otherwise PII detection performance could be degraded. default: false entity_detection: allOf: - $ref: '#/components/schemas/PIIDetectionParams' description: >- This section contains a set of parameters to control the PII detection process. All fields have sensible default that can be changed for specific needs. processed_text: oneOf: - $ref: '#/components/schemas/MarkerRedactedText' - $ref: '#/components/schemas/SyntheticRedactedText' - $ref: '#/components/schemas/MaskRedactedText' title: Processed Text description: >- This section allows the user to generate redacted (default), synthetic or masked text. default: pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' type: MARKER discriminator: propertyName: type mapping: MARKER: $ref: '#/components/schemas/MarkerRedactedText' MASK: $ref: '#/components/schemas/MaskRedactedText' SYNTHETIC: $ref: '#/components/schemas/SyntheticRedactedText' project_id: anyOf: - type: string maxLength: 32 pattern: ^[a-zA-Z0-9\-_\:]*$ - type: 'null' title: Project Id description: >- Used to categorize requests for reporting purposes. Limited to alphanumeric characters or the following special characters :_- additionalProperties: false type: object required: - text title: ProcessTextRequest description: API 3.0 Spec Definition ProcessTextResponse-Input: items: $ref: '#/components/schemas/ProcessTextResponseItem-Input' type: array title: ProcessTextResponse UserErrorResponseModel: properties: detail: anyOf: - $ref: '#/components/schemas/ErrorMessage' - $ref: '#/components/schemas/ValidationErrorModel' title: Detail description: >- The details of the error, usually relating to an input validation error type: object required: - detail title: UserErrorResponseModel InternalErrorResponseModel: properties: detail: type: string title: Detail description: >- The details of the error, usually relating to an unhandled processing error type: object required: - detail title: InternalErrorResponseModel PIIDetectionParams: properties: accuracy: allOf: - $ref: '#/components/schemas/AccuracyMode' description: >- Selects the model used to identify PII in the input text. By default, the `high` accuracy model is used. While the models used by the Private AI solution are highly optimized (~25X faster than a reference transformer implementation), in high-throughput cases it is possible to trade accuracy for speed by selecting either the `standard` or `standard_high` accuracy modes. Multilingual support can be enabled by using one of the multilingual models, namely `standard_high_multilingual` (GPU container only) and `high_multilingual`. The multilingual models process all supported languages including English, without the need to specify language. It is advisable to use the English-only models where possible, as they perform slightly better on English. Automatic Models can determine which model to use (English or Multilingual) depending on the languages detected, provided Multilingual models are available. default: high entity_types: items: anyOf: - $ref: '#/components/schemas/EnableEntityTypeSelector' - $ref: '#/components/schemas/DisableEntityTypeSelector' type: array title: Entity Types description: >- Controls which entity types and legislation sets are detected. See [Supported Entity Types](https://docs.private-ai.com/entities/#supported-entities) for the list of possible entities and legislation sets. By default, all entities are detected and removed. You can specify one of many selectors, which can be either an individual entity type such as `LOCATION_CITY` or a legislation like `GDPR`. `EnableEntityTypeSelector` selectors will add entity types to detect. On the contrary, `DisableEntityTypeSelector` selectors will ignore entities of the specified types. If only `DisableEntityTypeSelector` selectors are specified, they are assumed to be ignoring entity types from the entire supported list of entity types. filter: items: anyOf: - $ref: '#/components/schemas/AllowFilter' - $ref: '#/components/schemas/BlockFilter' type: array title: Filter description: >- Allows the user to customize PII detection via allow and block lists. See [Supported Entity Types](https://docs.private-ai.com/entities/#supported-entities) for the list of possible entities. return_entity: type: boolean title: Return Entity description: >- Controls whether the PII list in the response contains the `text` field. Turning this off means that no sensitive PII is returned in the response. default: true enable_non_max_suppression: type: boolean title: Enable Non Max Suppression description: >- When set, if the best label of an entity is disabled then the entity will not be redacted. default: false additionalProperties: false type: object title: PIIDetectionParams MarkerRedactedText: properties: type: const: MARKER title: Type default: MARKER pattern: type: string title: Pattern description: >- Specifies a custom redaction marker format. The format must contain one and only one of these predefined keywords: `BEST_ENTITY_TYPE`, `ALL_ENTITY_TYPES`, `UNIQUE_NUMBERED_ENTITY_TYPE`, `UNIQUE_HASHED_ENTITY_TYPE`, which will be replaced as follows: The keyword `BEST_ENTITY_TYPE` will be replaced by the entity type. For example, `--[BEST_ENTITY_TYPE]--` will transform `My name is John` into `My name is --[NAME_GIVEN]--`. The keyword `UNIQUE_NUMBERED_ENTITY_TYPE` is similar, but it will also make the marker unique by appending an index at the end. For example `--[UNIQUE_NUMBERED_ENTITY_TYPE]--` yields `My name is --[NAME_GIVEN_1]--`. The keyword `ALL_ENTITY_TYPES` will be replaced with all the entity types applicable to the entity. For example `--[ALL_ENTITY_TYPES]--` yields `My name is --[NAME,NAME_GIVEN]--`. It is also possible to set this option via environment variable. See [Environment Variables](https://docs.private-ai.com/environment-variables). default: '[UNIQUE_NUMBERED_ENTITY_TYPE]' additionalProperties: false type: object title: MarkerRedactedText SyntheticRedactedText: properties: type: const: SYNTHETIC title: Type default: SYNTHETIC synthetic_entity_accuracy: allOf: - $ref: '#/components/schemas/SyntheticPIIAccuracyMode' description: >- (Beta) Enable synthetic entity generation using the specified model. Currently this feature is in beta and supports modes `standard` and `standard_multilingual` default: standard preserve_relationships: type: boolean title: Preserve Relationships description: >- (Beta) Specifies whether multiple instances of the same entity should have the same generated synthetic entity or not. For example, preserve relationships: "Hi John and Rosha, John nice to meet you" -> "Hi Harry and Alev, Harry nice to meet you". No preserve relationships: "Hi John and Rosha, John nice to meet you" -> "Hi Harry and Alev, Sulav nice to meet you". This field has no effects when **synthetic_entity_accuracy** is not set. default: true additionalProperties: false type: object title: SyntheticRedactedText MaskRedactedText: properties: type: const: MASK title: Type default: MASK mask_character: type: string title: Mask Character description: >- Replaces redacted PII with the specified mask character. Note that this input may only be a single character default: '#' additionalProperties: false type: object title: MaskRedactedText ProcessTextResponseItem-Input: properties: processed_text: type: string title: Processed Text description: The redacted, synthetic or masked text. entities: items: $ref: '#/components/schemas/EntityItem' type: array title: Entities description: A list of all entities found in the text. entities_present: type: boolean title: Entities Present description: Returns `True` if the list of detected entities is not empty. characters_processed: type: integer title: Characters Processed description: The number of characters in all the text inputs. languages_detected: additionalProperties: type: number type: object title: Languages Detected description: >- A dictionary containing ISO 639-1 language labels and the likelihood of their detection in the request payload. type: object required: - processed_text - entities - entities_present - characters_processed - languages_detected title: ProcessTextResponseItem ErrorMessage: type: string title: ErrorMessage ValidationErrorModel: properties: description: type: string title: Description description: User-friendly error message description. loc: items: anyOf: - type: string - type: integer type: array title: Loc description: >- The location where the error has occurred. This could be location of a character if the error is a parsing error, or it could be a value in the request if there is a validation error. msg: type: string title: Msg description: The error message. type: type: string title: Type description: The type of error that has been encountered. type: object required: - loc - msg - type title: ValidationErrorModel AccuracyMode: type: string enum: - standard - standard_high - standard_high_multilingual - standard_high_automatic - high - high_multilingual - high_automatic title: AccuracyMode description: >- Selects the model used to identify PII in the input text. By default, the "high" accuracy model is used. While the models used by the Private AI solution are highly optimized (~25X faster than a reference transformer implementation), in high-throughput cases it is possible to trade accuracy for speed by selecting either the "standard" or "standard_high" accuracy modes. Multilingual support can be enabled by using one of the multilingual models, namely "standard_high_multilingual" (GPU container only) and "high_multilingual". The multilingual models process all supported languages including English, without the need to specify language. It is advisable to use the English-only models where possible, as they perform slightly better on English. Automatic Models can determine which model to use (English or Multilingual) depending on the languages detected, provided Multilingual models are available. EnableEntityTypeSelector: properties: type: const: ENABLE title: Type default: ENABLE value: items: type: string type: array title: Value description: >- A list of entity types to detect and remove. See [Supported Entity Types](https://docs.private-ai.com/entities/#supported-entities) for a complete list of entity types. This can also be one or many legislations. We currently support these legislations ['APPI', 'APPI_SENSITIVE', 'CPRA', 'GDPR', 'GDPR_SENSITIVE', 'HIPAA', 'LIDI', 'PCI', 'PHI', 'QUEBEC_PRIVACY_ACT', 'CORE_ENTITIES', 'NUMERICAL_EXCL_PCI']. additionalProperties: false type: object title: EnableEntityTypeSelector DisableEntityTypeSelector: properties: type: const: DISABLE title: Type default: DISABLE value: items: type: string type: array title: Value description: >- A list of entity types to ignore. See [Supported Entity Types](https://docs.private-ai.com/entities/#supported-entities) for a complete list of entity types. additionalProperties: false type: object title: DisableEntityTypeSelector AllowFilter: properties: type: const: ALLOW title: Type description: >- Any entities in this list that match the regex pattern provided will be discarded. It is also possible to set this option via environment variable. See [Environment Variables](https://docs.private-ai.com/environment-variables) default: ALLOW pattern: type: string title: Pattern description: >- Any string matching this regex pattern will not be caught by the PII detection algorithm. additionalProperties: false type: object required: - pattern title: AllowFilter BlockFilter: properties: type: const: BLOCK title: Type description: >- The block feature allows you to extend the functionality of the Private AI models by using regular expressions. This way, you can define a Python regex pattern that will be used to identify additional tokens with the given PII label. Several block list filters can be specified with their own regex pattern. Lastly,for supported labels, if you would like the model to pick up only the tokens from the block list, you can use the enabled entity type feature together with the block list feature. This can be done by defining a list of enabled entity types and not including the supported label you are adding to the block list. For example, if you would like the label `ORGANIZATION` to only pick up Microsoft, you can define the enabled entity types as `[{"type":"ENABLE", "value": "NAME"}, {"type": "ENABLE", "value": "LOCATION"}, {"type": "ENABLE", "value": "AGE"}, ...]` (and omitting `ORGANIZATION`) and the block list as `[{"type": "BLOCK", "entitiy_type": "ORGANIZATION", "pattern": "Microsoft"}]`. default: BLOCK entity_type: type: string title: Entity Type description: >- Name of the custom entity type. It can either be a completely new entity type such as `CUSTOM_ID` or an existing entity, such as `NAME`. pattern: type: string title: Pattern description: >- This is a pattern to match in the text. This feature uses regex patterns, you can either pass a word (e.g. the, word, custom, etc.) or you can pass a valid Python regex pattern. It is important to note that regex patterns may require escaping when used in JSON objects. To give an example, if you would like to send the regex pattern `r"\b\w{4}\b"` which will catch every 4-character word, you need to send it as `"\\b\\w{4}\\b"`. A complete JSON grammar is found here: https://www.json.org/json-en.html. More information on how to write a python regex is found here: https://docs.python.org/3/library/re.html It is important to note also that only non-overlapping matches are returned. threshold: type: number title: Threshold description: >- This is defining a likelihood threshold for custom entity. This likelihood is compared against the predicted model likelihood and if it is greater then the custom entity is outputted instead of the model predicted entity. By default this threshold is set to 1.0 which will ensure that the blocked entities will always be preferred over a matching model predicted entity. This can be any value between 0 and 1. default: 1 additionalProperties: false type: object required: - entity_type - pattern title: BlockFilter SyntheticPIIAccuracyMode: type: string enum: - standard - standard_multilingual title: SyntheticPIIAccuracyMode EntityItem: properties: processed_text: type: string title: Processed Text description: >- Either the redaction marker, hash marker or synthetic entity text in `processed_text` corresponding to the entity. text: type: string title: Text description: >- The entity text. When the `return_entity` option is set to `False`, this returns a null string location: allOf: - $ref: '#/components/schemas/TextLocation' default: >- Location object containing the start-end character indexes of given and processed text. best_label: type: string title: Best Label description: The entity label with the highest likelihood. labels: additionalProperties: type: number type: object title: Labels description: >- A dictionary containing any other labels found, together with associated likelihoods. Note that these are not strictly probabilities and do not sum to 1, as an entity can have multiple types. Note that the likelihoods have also been thresholded, so no additional thresholding is necessary. type: object required: - processed_text - text - best_label - labels title: EntityItem TextLocation: properties: stt_idx: type: integer title: Stt Idx description: Start character index of the entity in the original text. end_idx: type: integer title: End Idx description: >- Index of the character immediately following the entity, such that end_idx - stt_idx = number of characters in the entity. stt_idx_processed: type: integer title: Stt Idx Processed description: Start character index of the entity in the processed text. end_idx_processed: type: integer title: End Idx Processed description: >- Index of the character immediately following the entity in the processed text. type: object required: - stt_idx - end_idx - stt_idx_processed - end_idx_processed title: TextLocation description: >- Start and end indices of the entity in a text-based file format such as `.txt`. ````