New access control and permissions - Textual now uses permissions and permission sets to manage user access. Global permissions control access to features outside of specific datasets and pipelines. Dataset and pipeline permissions manage access within specific datasets and pipelines. Textual comes with built-in permission sets, and you can configure custom permission sets. Administrators can assign global permission sets to users. Users can share their datasets and pipelines with other users in the organization.
Generator selection for custom entity types - For custom entity types, when you select Synthesis as the handling option, you can now select and configure the generator to use to create the replacement value.
To fix an H11 Python vulnerability, upgraded to 0.16.0.
Enabled support for the LOCATION_COMPLETE_ADDRESS in PDFs and images.
For connections to Amazon S3, you can now choose whether to use Amazon S3 encryption (the default) or use KMS encryption.
You can now assign tags to datasets and pipelines.
Self-hosted instances can now configure custom endpoints for connections to AWS.
Added the Password entity type to the named-entity recognition (NER) model.
When you select an AWS Region, the dropdown list now includes the ISO Regions.
Additional consistency between Textual and Structural - A new environment variable allows you to configure a statistics seed to enable you to ensure that synthesized values in Textual match corresponding generated values in Structural. The statistics seed is also displayed in the Textual application. To ensure consistency between the application, provide the same statistics seed value in both.
The consistency applies to non-HIPAA addresses, to match the Address generator in Structural, and US telephone numbers, to match the US Phone Number generator. For these types of values, when the statistics seeds match, the same input value produces the same output value in both Textual and Structural.
Custom entity types - You can now define custom entity types, to identify sets of values that are unique to your industry or organization. For each custom entity type, you provide one or more regular expressions to define the matching values. You can then enable or disable the entity type for each dataset and pipeline.
Fixed an issue that caused slow load times for customers with large datasets. Calls to GET /api/dataset and GET /api/dataset/{datasetid} no longer return entity information for the dataset files. Instead, the new GET /api/dataset/{datasetid}/pii_info endpoint returns the entity information for a dataset’s files.
A new dataset settings option controls the output in .docx tables. By default, table content goes through the regular scan and redaction process, and detected entity values are handled based on the dataset’s entity type handling configuration. You can also choose to completely block out all table cells, in which case each table cell is covered by a black box.
File statistics for pipelines - The pipeline details page now displays a summary of information about the pipelines files, including the number of files, the number of words in the files, the number of detected entity types, and the number of detected topics. For entity types, the display includes the number of detected values for each type. For topics, the display includes the number of files that involve each topic.
On the dataset details page, the preview count for each entity type now reflects the count of values that are assigned that type in the output files. Previously, values that matched multiple entity types were included in the preview count for all of the matching types.
New Textual Home page - The Textual Home page now contains an updated version of the Playground, where you can see how Textual detects and replaces entity values in text. There is no longer a separate Playground page, and there is no LLM Synthesis option. For each entity type, you can configure handling options and added or excluded values. Textual generates Python and cURL versions of the request that you can copy.
From the Request Explorer, in addition to testing added and excluded values, you can now also select the handling type for each entity type. The Unified toggle is replaced with options to view either the original values with their corresponding types (Identification) or the actual output with the replacement values (Replacement).
Edit and replay recorded requests - When you use the Request Explorer to preview a recorded redaction request from the SDK, you can now edit the request to add and exclude entity values. You can then re-run the redaction and view the differences between the original request and the edited request.
When you configure excluded values for an entity type, you can block detection of a specific type within a matching phrase. For example, if you add the phrase "one moment, please" to an excluded value for numeric values, the word “one” is not detected as a numeric value in that specific context.
Fixed an issue that prevented users from saving the dataset configuration for .docx comments.
Record and view redaction requests - When you make a redact call to redact a plain text string, you can now record the request. You specify the amount of time to keep the recording, and any tags to assign to the request. From the new API Explorer page, you can then view and analyze the recorded redaction requests, to assess the quality of the redaction.
New redaction options for datasets - The settings panel for datasets now includes additional configuration options:
- You can configure whether to redact or remove images in .docx files.
- You can choose to remove comments from .docx files.
- For PDF files, you can choose to detect and redact scanned signatures.
Added the Tonic NER model version to the model information. The API endpoint /api/environment/models reports version strings for NER models.
Entity manager for entity types - The new entity manager allows you to view all of the occurrences of each entity type in a dataset. it displays the original value, the context in the original file, and the context in the transformed file. To view the entities manager, from the entity value preview list, click Open Entities Manager. Note that by default, for the NUMERIC_VALUE entity type, Textual only provides context information for the first 20 occurrences. To change this, set the SOLAR_NER_OCCURRENCE_IGNORE_NUMERIC_VALUE environment variable to false.
Fixed a rare issue where Azure OCR returned a400 response when the file upload stream contained corrupted data.
Improved synthesis on days of the week and ordinal numbers that are flagged as DATE_TIME.
Textual now only disables a numeric span when it overlaps one of the following disabled types: DATE_TIME, DOB, LOCATION, LOCATION_ADDRESS, LOCATION_ZIP, MONEY, CREDIT_CARD, PHONE_NUMBER.