> ## Documentation Index > Fetch the complete documentation index at: https://docs.getlimina.ai/llms.txt > Use this file to discover all available pages before exploring further. # Process Text > Detect entities such as PII, PHI or PCI in the provided text strings using Private AI's entity detection engine. After entity detection, any entities found can be redacted, masked or replaced with AI-generated synthetic entities. ## OpenAPI ````yaml /openapi/privateai_3.3.3.json post /v3/process/text openapi: 3.0.2 info: title: API Reference description: Private AI API Reference termsOfService: https://www.private-ai.com/terms-of-use/ contact: url: https://www.private-ai.com/contact/ email: info@private-ai.com version: 3.3.3 servers: - url: https://api.private-ai.com/deid description: Private AI Demo Server - url: http://localhost:8080 description: Local Server security: [] paths: /v3/process/text: post: summary: Process Text description: >- Detect entities such as PII, PHI or PCI in the provided text strings using Private AI's entity detection engine. After entity detection, any entities found can be redacted, masked or replaced with AI-generated synthetic entities. operationId: process_text_v3_process_text_post parameters: - required: false schema: title: X-Api-Key type: string default: '' name: x-api-key in: header requestBody: content: application/json: schema: $ref: '#/components/schemas/ProcessTextRequest' examples: simple_deidentification: summary: Simple De-Identification value: text: - Hello John and Jane link_batch: false entity_detection: accuracy: high return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' synthetic_deidentification: summary: Synthetic PII value: text: - >- Hello, my name is May. I am the aunt of Pieter Parker. We live in Toronto, Canada. link_batch: false entity_detection: accuracy: high return_entity: true processed_text: type: SYNTHETIC synthetic_entity_accuracy: standard preserve_relationships: true enabled_entity_types: summary: Enabled Entity Types value: text: - Hello, My name is Mike and I am 24 years old link_batch: false entity_detection: accuracy: high entity_types: - type: ENABLE value: - NAME return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' allow_list: summary: Allow List value: text: - >- Hello Xavier, I broke my right leg on the 31st. Id of PAI is PAI_ID-44444 yeah, my birthday, My id is 78549 and I'm waiting for my x-ray results. dr. zhang, mercer health centre. link_batch: false entity_detection: accuracy: high filter: - type: ALLOW pattern: PAI_ID-\d{5} return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' block_list: summary: Block List value: text: - >- Hello Xavier, I broke my right leg on the 31st. Id of PAI is PAI_ID-44444 yeah, my birthday, My id is 78549 and I'm waiting for my x-ray results. dr. zhang, mercer health centre. link_batch: false entity_detection: accuracy: high filter: - type: BLOCK entity_type: MY_5_DIGIT_PAI_ID pattern: PAI_ID-\d{5} return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' link_batch: summary: Link Batch value: text: - >- Hi, my name is Penelope, could you tell me your phone number please? - Sure, x234 - and your DOB please? - fourth of Feb nineteen 86 link_batch: true entity_detection: accuracy: high return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' masked_pii: summary: Masked PII value: text: - Hello John and Jane link_batch: false entity_detection: accuracy: high return_entity: true processed_text: type: MASK mask_character: '#' multiple_input: summary: Multiple Input value: text: - Hello John and Jane - Mark is 42 years old link_batch: false entity_detection: accuracy: high return_entity: true processed_text: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' non_unique_pii_markers: summary: Non Unique PII Markers value: text: - Hello John and Jane link_batch: false entity_detection: accuracy: high return_entity: true processed_text: type: MARKER pattern: '[BEST_ENTITY_TYPE]' required: true responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/ProcessTextResponse' '400': description: Bad Request content: application/json: schema: $ref: '#/components/schemas/UserErrorResponseModel' '500': description: Internal Server Error content: application/json: schema: $ref: '#/components/schemas/InternalErrorResponseModel' 4XX: description: Client Error content: application/json: schema: $ref: '#/components/schemas/UserErrorResponseModel' components: schemas: ProcessTextRequest: title: ProcessTextRequest required: - text type: object properties: text: title: Text type: array items: type: string description: >- UTF-8 encoded message(s) to process. E.g. `["My name is Adam"]` or `["I live at", "263 Spadina Av"]`. Request processing time increases linearly with input text length, therefore maximum length is dependent on provisioned hardware and any timeouts set by the user. Private AI has tested up to 500K characters on the CPU and GPU containers. link_batch: title: Link Batch type: boolean description: >- When set to `True`, the list of inputs provided in `text` will be processed together as a single input by the Private AI PII detection model. This shares context between the different inputs and is useful when processing a sequence of short inputs, such as an SMS chat log. This option should only be enabled when the inputs are related, otherwise PII detection performance could be degraded. default: false entity_detection: title: Entity Detection allOf: - $ref: '#/components/schemas/PIIDetectionParams' description: >- This section contains a set of parameters to control the PII detection process. All fields have sensible default that can be changed for specific needs. processed_text: title: Processed Text oneOf: - $ref: '#/components/schemas/MarkerRedactedText' - $ref: '#/components/schemas/SyntheticRedactedText' - $ref: '#/components/schemas/MaskRedactedText' description: >- This section allows the user to generate redacted (default), synthetic or masked text. default: type: MARKER pattern: '[UNIQUE_NUMBERED_ENTITY_TYPE]' discriminator: propertyName: type mapping: MARKER: $ref: '#/components/schemas/MarkerRedactedText' SYNTHETIC: $ref: '#/components/schemas/SyntheticRedactedText' MASK: $ref: '#/components/schemas/MaskRedactedText' project_id: title: Project Id maxLength: 32 pattern: ^[a-zA-Z0-9\-_\:]*$ type: string description: >- Used to categorize requests for reporting purposes. Limited to alphanumeric characters or the following special characters :_- additionalProperties: false description: API 3.0 Spec Definition ProcessTextResponse: title: ProcessTextResponse type: array items: $ref: '#/components/schemas/ProcessTextResponseItem' UserErrorResponseModel: title: UserErrorResponseModel required: - detail type: object properties: detail: title: Detail anyOf: - $ref: '#/components/schemas/ErrorMessage' - $ref: '#/components/schemas/ValidationErrorModel' description: >- The details of the error, usually relating to an input validation error InternalErrorResponseModel: title: InternalErrorResponseModel required: - detail type: object properties: detail: title: Detail type: string description: >- The details of the error, usually relating to an unhandled processing error PIIDetectionParams: title: PIIDetectionParams type: object properties: accuracy: allOf: - $ref: '#/components/schemas/AccuracyMode' description: >- Selects the model used to identify PII in the input text. By default, the `high` accuracy model is used. While the models used by the Private AI solution are highly optimized (~25X faster than a reference transformer implementation), in high-throughput cases it is possible to trade accuracy for speed by selecting either the `standard` or `standard_high` accuracy modes. Multilingual support can be enabled by using one of the multilingual models, namely `standard_high_multilingual` (GPU container only) and `high_multilingual`. The multilingual models process all supported languages including English, without the need to specify language. It is advisable to use the English-only models where possible, as they perform slightly better on English. default: high entity_types: title: Entity Types type: array items: anyOf: - $ref: '#/components/schemas/EnableEntityTypeSelector' - $ref: '#/components/schemas/DisableEntityTypeSelector' description: >- Controls which entity types and legislation sets are detected. See [Supported Entity Types](https://docs.private-ai.com/entities/#supported-entities) for the list of possible entities and legislation sets. By default, all entities are detected and removed. You can specify one of many selectors, which can be either an individual entity type such as `LOCATION_CITY` or a legislation like `GDPR`. `EnableEntityTypeSelector` selectors will add entity types to detect. On the contrary, `DisableEntityTypeSelector` selectors will ignore entities of the specified types. If only `DisableEntityTypeSelector` selectors are specified, they are assumed to be ignoring entity types from the entire supported list of entity types. filter: title: Filter type: array items: anyOf: - $ref: '#/components/schemas/AllowFilter' - $ref: '#/components/schemas/BlockFilter' description: >- Allows the user to customize PII detection via allow and block lists. See [Supported Entity Types](https://docs.private-ai.com/entities/#supported-entities) for the list of possible entities. return_entity: title: Return Entity type: boolean description: >- Controls whether the PII list in the response contains the `text` field. Turning this off means that no sensitive PII is returned in the response. default: true override: title: Override type: boolean description: >- Determines whether the disabled entity is used to override any other entities that the PII might belong to. Using this will force the system to keep a detected PII in the text. default: false additionalProperties: false MarkerRedactedText: title: MarkerRedactedText type: object properties: type: title: Type enum: - MARKER type: string default: MARKER pattern: title: Pattern type: string description: >- Specifies a custom redaction marker format. The format must contain one and only one of these predefined keywords: `BEST_ENTITY_TYPE`, `ALL_ENTITY_TYPES`, `UNIQUE_NUMBERED_ENTITY_TYPE`, which will be replaced as follows: The keyword `BEST_ENTITY_TYPE` will be replaced by the entity type. For example, `--[BEST_ENTITY_TYPE]--` will transform `My name is John` into `My name is --[NAME_GIVEN]--`. The keyword `UNIQUE_NUMBERED_ENTITY_TYPE` is similar, but it will also make the marker unique by appending an index at the end. For example `--[UNIQUE_NUMBERED_ENTITY_TYPE]--` yields `My name is --[NAME_GIVEN_1]--`. The keyword `ALL_ENTITY_TYPES` will be replaced with all the entity types applicable to the entity. For example `--[ALL_ENTITY_TYPES]--` yields `My name is --[NAME,NAME_GIVEN]--`. It is also possible to set this option via environment variable. See [Environment Variables](https://docs.private-ai.com/environment-variables). default: '[UNIQUE_NUMBERED_ENTITY_TYPE]' additionalProperties: false SyntheticRedactedText: title: SyntheticRedactedText type: object properties: type: title: Type enum: - SYNTHETIC type: string default: SYNTHETIC synthetic_entity_accuracy: allOf: - $ref: '#/components/schemas/SyntheticPIIAccuracyMode' description: >- (Beta) Enable synthetic entity generation using the specified model. Currently this feature is in beta and supports modes `standard` and `standard_multilingual` default: standard preserve_relationships: title: Preserve Relationships type: boolean description: >- (Beta) Specifies whether multiple instances of the same entity should have the same generated synthetic entity or not. For example, preserve relationships: "Hi John and Rosha, John nice to meet you" -> "Hi Harry and Alev, Harry nice to meet you". No preserve relationships: "Hi John and Rosha, John nice to meet you" -> "Hi Harry and Alev, Sulav nice to meet you". This field has no effects when **synthetic_entity_accuracy** is not set. default: true additionalProperties: false MaskRedactedText: title: MaskRedactedText type: object properties: type: title: Type enum: - MASK type: string default: MASK mask_character: title: Mask Character type: string description: >- Replaces redacted PII with the specified mask character. Note that this input may only be a single character default: '#' additionalProperties: false ProcessTextResponseItem: title: ProcessTextResponseItem required: - processed_text - entities - entities_present - characters_processed - languages_detected type: object properties: processed_text: title: Processed Text type: string description: The redacted, synthetic or masked text. entities: title: Entities type: array items: $ref: '#/components/schemas/EntityItem' description: A list of all entities found in the text. entities_present: title: Entities Present type: boolean description: Returns `True` if the list of detected entities is not empty. characters_processed: title: Characters Processed type: integer description: The number of characters in all the text inputs. languages_detected: title: Languages Detected type: object additionalProperties: type: number description: >- A dictionary containing ISO 639-1 language labels and the likelihood of their detection in the request payload. ErrorMessage: title: ErrorMessage type: string description: An error message ValidationErrorModel: title: ValidationErrorModel required: - loc - msg - type type: object properties: description: title: Description type: string description: User-friendly error message description. loc: title: Loc type: array items: anyOf: - type: string - type: integer description: >- The location where the error has occurred. This could be location of a character if the error is a parsing error, or it could be a value in the request if there is a validation error. msg: title: Msg type: string description: The error message. type: title: Type type: string description: The type of error that has been encountered. AccuracyMode: title: AccuracyMode enum: - standard - standard_high - standard_high_multilingual - high - high_multilingual type: string description: >- Selects the model used to identify PII in the input text. By default, the "high" accuracy model is used. While the models used by the Private AI solution are highly optimized (~25X faster than a reference transformer implementation), in high-throughput cases it is possible to trade accuracy for speed by selecting either the "standard" or "standard_high" accuracy modes. Multilingual support can be enabled by using one of the multilingual models, namely "standard_high_multilingual" (GPU container only) and "high_multilingual". The multilingual models process all supported languages including English, without the need to specify language. It is advisable to use the English-only models where possible, as they perform slightly better on English. EnableEntityTypeSelector: title: EnableEntityTypeSelector type: object properties: type: title: Type enum: - ENABLE type: string default: ENABLE value: title: Value type: array items: type: string description: >- A list of entity types to detect and remove. See [Supported Entity Types](https://docs.private-ai.com/entities/#supported-entities) for a complete list of entity types. This can also be one or many legislations. We currently support these legislations ['GDPR', 'GDPR_SENSITIVE', 'HIPAA', 'CPRA', 'QUEBEC_PRIVACY_ACT', 'PCI', 'APPI', 'APPI_SENSITIVE', 'LIDI']. additionalProperties: false DisableEntityTypeSelector: title: DisableEntityTypeSelector type: object properties: type: title: Type enum: - DISABLE type: string default: DISABLE value: title: Value type: array items: type: string description: >- A list of entity types to ignore. See [Supported Entity Types](https://docs.private-ai.com/entities/#supported-entities) for a complete list of entity types. additionalProperties: false AllowFilter: title: AllowFilter required: - pattern type: object properties: type: title: Type enum: - ALLOW type: string description: >- Any entities in this list that match the regex pattern provided will be discarded. It is also possible to set this option via environment variable. See [Environment Variables](https://docs.private-ai.com/environment-variables) default: ALLOW pattern: title: Pattern type: string description: >- Any string matching this regex pattern will not be caught by the PII detection algorithm. additionalProperties: false BlockFilter: title: BlockFilter required: - entity_type - pattern type: object properties: type: title: Type enum: - BLOCK type: string description: >- The block feature allows you to extend the functionality of the Private AI models by using regular expressions. This way, you can define a Python regex pattern that will be used to identify additional tokens with the given PII label. Several block list filters can be specified with their own regex pattern. Lastly,for supported labels, if you would like the model to pick up only the tokens from the block list, you can use the enabled entity type feature together with the block list feature. This can be done by defining a list of enabled entity types and not including the supported label you are adding to the block list. For example, if you would like the label `ORGANIZATION` to only pick up Microsoft, you can define the enabled entity types as `[{"type":"ENABLE", "value": "NAME"}, {"type": "ENABLE", "value": "LOCATION"}, {"type": "ENABLE", "value": "AGE"}, ...]` (and omitting `ORGANIZATION`) and the block list as `[{"type": "BLOCK", "entitiy_type": "ORGANIZATION", "pattern": "Microsoft"}]`. default: BLOCK entity_type: title: Entity Type type: string description: >- Name of the custom entity type. It can either be a completely new entity type such as `CUSTOM_ID` or an existing entity, such as `NAME`. pattern: title: Pattern type: string description: >- This is a pattern to match in the text. This feature uses regex patterns, you can either pass a word (e.g. the, word, custom, etc.) or you can pass a valid Python regex pattern. It is important to note that regex patterns may require escaping when used in JSON objects. To give an example, if you would like to send the regex pattern `r"\b\w{4}\b"` which will catch every 4-character word, you need to send it as `"\\b\\w{4}\\b"`. A complete JSON grammar is found here: https://www.json.org/json-en.html. More information on how to write a python regex is found here: https://docs.python.org/3/library/re.html It is important to note also that only non-overlapping matches are returned. threshold: title: Threshold type: number description: >- This is defining a likelihood threshold for custom entity. This likelihood is compared against the predicted model likelihood and if it is greater then the custom entity is outputted instead of the model predicted entity. By default this threshold is set to 1.0 which will ensure that the blocked entities will always be preferred over a matching model predicted entity. This can be any value between 0 and 1. default: 1 additionalProperties: false SyntheticPIIAccuracyMode: title: SyntheticPIIAccuracyMode enum: - standard - standard_multilingual type: string description: An enumeration. EntityItem: title: EntityItem required: - processed_text - text - best_label - labels type: object properties: processed_text: title: Processed Text type: string description: >- Either the redaction marker, hash marker or synthetic entity text in `processed_text` corresponding to the entity. text: title: Text type: string description: >- The entity text. When the `return_entity` option is set to `False`, this returns a null string location: title: Location allOf: - $ref: '#/components/schemas/TextLocation' default: >- Location object containing the start-end character indexes of given and processed text. best_label: title: Best Label type: string description: The entity label with the highest likelihood. labels: title: Labels type: object additionalProperties: type: number description: >- A dictionary containing any other labels found, together with associated likelihoods. Note that these are not strictly probabilities and do not sum to 1, as an entity can have multiple types. Note that the likelihoods have also been thresholded, so no additional thresholding is necessary. TextLocation: title: TextLocation required: - stt_idx - end_idx - stt_idx_processed - end_idx_processed type: object properties: stt_idx: title: Stt Idx type: integer description: Start character index of the entity in the original text. end_idx: title: End Idx type: integer description: >- Index of the character immediately following the entity, such that end_idx - stt_idx = number of characters in the entity. stt_idx_processed: title: Stt Idx Processed type: integer description: Start character index of the entity in the processed text. end_idx_processed: title: End Idx Processed type: integer description: >- Index of the character immediately following the entity in the processed text. description: >- Start and end indices of the entity in a text-based file format such as `.txt`. ````