New redaction options for datasets - The settings panel for datasets now includes additional configuration options:
- You can configure whether to redact or remove images in .docx files.
- You can choose to remove comments from .docx files.
- For PDF files, you can choose to detect and redact scanned signatures.
Added the Tonic NER model version to the model information. The API endpoint /api/environment/models reports version strings for NER models.
Entity manager for entity types - The new entity manager allows you to view all of the occurrences of each entity type in a dataset. it displays the original value, the context in the original file, and the context in the transformed file. To view the entities manager, from the entity value preview list, click Open Entities Manager. Note that by default, for the NUMERIC_VALUE entity type, Textual only provides context information for the first 20 occurrences. To change this, set the SOLAR_NER_OCCURRENCE_IGNORE_NUMERIC_VALUE environment variable to false.
Fixed a rare issue where Azure OCR returned a400 response when the file upload stream contained corrupted data.
Improved synthesis on days of the week and ordinal numbers that are flagged as DATE_TIME.
Textual now only disables a numeric span when it overlaps one of the following disabled types: DATE_TIME, DOB, LOCATION, LOCATION_ADDRESS, LOCATION_ZIP, MONEY, CREDIT_CARD, PHONE_NUMBER.
Improved the Textual NER model throughput on long strings that contain a large number of detected entities.
Added support to store dataset files in a specified S3 bucket, instead of in the Textual application database.
When Textual replaces first name values, it now attempts to use a name with the same gender.
For the DOB (date of birth) entity type, you can now configure synthesis options. You can set how to shift the date.
You can now configure the entity type handling for a dataset before you upload the dataset files.
You can now provide added and excluded entity values when you use the SDK to redact individual strings and files.
Added a new method to the SDK. redact_xml works similarly to redact_json. To produce a redacted output, you pass in a redact_xml string.
Removed support for en_core_web_trf and en_core_web_lg auxiliary models. Disabled model inference for ORGANIZATION, PERSON, LOCATION, and MONEY entity types. Updated the auxiliary model configuration environmental variables to have new default values:TEXTUAL_AUX_MODEL_GPU: falseTEXTUAL_AUX_MODEL: en_core_web_sm
Fixed a redaction issue that was caused by a regression from v140.
Improved the Textual NER model, specifically for datetime values and and electronic health records.
Fix for correctly re-synthesizing files as part of pipelines
When you call the dataset.add_file method in the Textual SDK, you can now pass in IO bytes.
You can now specify a list of additional values to include for each entity in a datasets. This allows Textual to identify values that it might not identify because they are specific to your organization or industry. The list can contain both specific values and regular expressions.
Improved the file list display for datasets to better accommodate longer file names.
For an uploaded file pipeline, added a Back to Files breadcrumb to return the user to the main file list.
On the dataset details page, the bulk edit function for entity type handling is now a dropdown instead of separate buttons.
You can now use the Python SDK to delete files from a dataset.
To improve performance, enabled date synthesis inference on GPU. Added the environment variable TEXTUAL_DATE_ SYNTH_ GPU to manage whether to use it.
Renamed the following environment variables:
SOLAR_PREFER_GPUtoTEXTUAL_AUX_MODEL_GPUSOLAR_AUX_MODELtoTEXTUAL_AUX_MODEL
The responses for textual.redact and textual.llm_synthesis now include:
br<
- The
languagefor each entity value. - The start location (
new_start) of each entity value in the redacted content. - The end location (
new_end) of each entity value in the redacted string.
For uploaded file pipelines that also generate redacted files, you can now configure the handling option for each entity type.
You can now set up a pipeline to connect to a Databricks workspace. You can also configure a Databricks pipeline to generate redacted files in addition to the JSON output.
Textual can now detect values in multiple languages. Textual Cloud supports a set of non-English languages. For a self-hosted instance, you must enable multi-language support and provide the language models for Textual to use.
NOTE: This version was removed because of a regression.
Fixed an issue where pipelines failed to run.
Updated the Python SDK to add options to create pipelines, delete pipelines, upload pipeline files, and download pipeline results.
Disabled the Run Pipeline option when a pipeline does not have a configured output path.
You can now delete a newly uploaded file without having to refresh the page.
Fixed an issue with the status information for pipeline files.
For plain text pipeline files, removed the option to toggle between Markdown and the original content.
Fixed an issue with previewing PDF files.
The file list for a pipeline run no longer displays while the run is queued.
