Path expressions for document-based data - On Connection View, and for JSON columns that use Document View, you can now configure generators to apply to fields that match a JSONPath expression and that contain specific types of data.
The Jobs page for a workspace now displays a paginated table of jobs, filterable by status, type, or ID. You also can now search and view the full job history. Previously, only the 100 most recent jobs were displayed, and you could not view any older jobs.
Workspace schema caching - To improve the loading performance of the Structural application, you can now configure a workspace to cache the source database schema.
Fixed an issue where when a workspace was transferred to a new owner, and the original creator was deleted, the connection data became inaccessible.
During upsert, to accommodate schema changes in the destination database, we can now automatically add columns, whether those columns are nullable or not nullable, and whether or not the columns have default values.
Fixed an issue that blocked logins using Duo SSO.
Fixed an issue that caused sensitivity scans to fail on workspaces that contained document columns and where Tonic Table Parallelism was set to 2 or above.
On the workspace settings page, replaced the Block Data Generation on Schema Changes toggle with a set of options. The new options are Do Not Block, Block On Sensitive Changes, and Block On All Changes.
Structural now correctly preserves the original value for columns that contain constant values and that are assigned the Continuous generator. A previous fix addressed this partially, but some edge cases still caused the original value to be lost or triggered an error during generation. This is now fully resolved.
SQL Server - Fixed a performance regression when masking tables that have large row volumes.
Fixed an issue where when the Character Substitution generator was applied to a primary key column in a subsetting target table, data generation failed.
Fixed a regression introduced in v1459 where workspaces that used a secrets manager for password retrieval did not load if an error occurred during the secret retrieval.
Upsert - Changed the upsert pre-job check to no longer fail when the destination database is missing tables that exist in the source.
SQL Server - Fixed a performance regression when masking tables that have large row volumes.
Oracle: Tonic Structural now bases the list of required privileges and catalog packages on the features in use in the source database. The privilege check performed at the start of a generation job now gives an error instead of a warning if privileges are missing.
PostgreSQL - Fixed an issue issue where connection pool limits were not always being respected when creating PostgreSQL connection strings.
Fixed an issue where setting query filter on a non-target subsetting table caused errors.
The Structural API now provides an api/Privacy/history/search endpoint to allow you to issue paginated queries of privacy events. These are the same events that display on the Protection Audit Trail on Privacy Hub.
Revised schema change type definition and handling - We have changed how we categorize and respond to detected schema changes.
Sensitive schema changes include schema changes that could result in data leakage. These include new tables and columns, new collections and path types, changes to column data types, and changes to nullability. These types of schema changes always block data generation. Schema Changes view includes options to resolve these changes.
Notifications are schema changes that Structural resolves automatically for each data generation, unless the workspace is configured to block data generation for all schema changes. Notifications include removed tables and columns, removed collections and paths, and changing a column to a foreign key. To permanently resolve these changes, you address them on Schema Changes view.
Fixed an issue where in specific cases when tables had 4 or fewer rows, columns that contained credit card numbers were sometimes not identified as containing sensitive data.
Fixed an issue where Structural could not import a JSON file of virtual foreign keys that were previously exported. As of this update, for data connectors that do not support schemas, you cannot provide a schema value.
File connector - For file mounts, removed the environment setting to configure a single file mount location. The file mount location is always configured in the workspace.
You can now override the following environment settings in individual workspaces:
TONIC_READ_RANGES_PARALLELISMTONIC_PARTITION_PARALLELISMTONIC_FILE_GROUP_PARALLELISMTONIC_BIGQUERY_READ_PARALLELISMTONIC_ORACLE_DATA_PUMP_PARALLELISMTONIC_DATABRICKS_SKIP_CREATE_DB
Removed the environment setting TONIC_DATABRICKS_ENABLE_WORKSPACE_SKIP_CREATE_DB, and also deprecated the skipCreateDB workspace property in the Structural API.
Now, to control whether to use your own schema for Databricks, use the setting TONIC_DATABRICKS_SKIP_CREATE_DB, which you can also configure in individual workspaces.
The encryption and decryption keys for Structural data encryption can now be configured as workspace overrides to the environment settings TONIC_DATA_DECRYPTION_KEY and TONIC_DATA_ENCRYPTION_KEY.
TONIC_DATA_DECRYPTION_KEY and TONIC_DATA_ENCRYPTION_KEY can now be configured from the Environment Settings tab on Structural Settings .
MySQL - Fixed an issue with output to a container repository caused by a broken dependency update chain (kubernetes-client → YamlDotNet).
On the job details page, the Reports and Logs dropdown list now includes a View Gantt option, which allows you to view where time was spent during job execution. The option is only available for data generation jobs that used the Data Pipeline V2 process.
For the Conditional generator, fixed an issue where when a condition was based on other columns, the generator preview was not correct.
MySQL: Structural now restores preserved tables and triggers in the event of a failure.
SQL Server - Fixed a bug where history table settings and indexes were not appropriately replicated in the destination database.
Webhooks - new GitHub Actions webhook type - You can now configure a webhook that runs a GitHub action when the webhook is triggered. You specify the GitHub repository and credentials, and select the workflow. Structural then prompts you to provide the input values. The original type of webhook is now referred to as a custom webhook.
Webhooks - new schema change trigger type - You can now trigger a webhook when Structural detects a schema change. Previously, you could only trigger webhooks based on a job status.
Fixed an issue where nullable boolean environment settings were saved as null but could not be edited in the application.
Fixed an issue where nullable integer settings with default values could not be set to null when adding or editing.
MySQL - When a workspace writes output to a container repository, if the sql_mode cannot be determined from the source database - either from the SESSION or GLOBAL config - instead of leaving sql_mode unset, data packer now falls back to a sensible default value.
MySQL, Oracle: When calculating the percentage for subsetting target tables, Structural now uses the correct row counts.
Fixed an issue with unexpected behavior when the environment setting TONIC_STATISTICS_SEED is not set.
Updated the styling for Schema Changes view.
On Schema Changes view, added a button to display filtered tDatabase Viewo display all of the schema change columns.
Created a new generator for the Finnish Personal Identity Code.
You must specify a date range, and the generator will validate whether the input data falls within that period and if they are valid identity codes.
After that, a new Finnish Personal Identity Code will be generated so that the new birth date is within the selected range. A random valid century indicator will also be chosen, as well as a sequential number between 002 and 899.
Finally, the check digit will be calculated.
The new generator supports data consistency.
When using Google SSO, if TONIC_SSO_DOMAIN is configured, only users with a matching domain are permitted to log in.
For an individual column on Database View or Table View, the option to apply a recommended generator is now included in the column configuration panel, under the generator selection dropdown. Previously it was a separate panel.
Fixed an issue with the Constant generator where the time could not be set for timestamps that included a time zone.
On Structural Cloud, the login screen now includes a Remember Me checkbox. When checked, you are not prompted to re-enter your email address the next time you return to the login screen. To switch accounts, click Back to email login. This returns you to the email entry page. You can then enter a different account.
New Document View option to configure file connector JSON columns - For JSON file connector columns, instead of the JSON Mask generator, you can use the new Document View option to view the JSON structure and to assign generators to each combination of path and data type within the JSON. On the column configuration panel for a file connector JSON column, toggle Use Document View to the on position. Document View displays as a new tab on the workspace management view.
On the recommended generator panel for columns, fixed an issue where the preview data was not generated.
SQL Server - The TONIC_SQL_SERVER_SCRIPT_NO_FILE_GROUPS configuration setting is now respected when creating constraints in the destination database.
SQL Server - Improved column length mismatch handling for SQL Server bulk copy.
File connector - On self-hosted instances, you can now configure a file mount as the source for file connector workspace files. After you set up the file mount, to enable the file mount option, you configure the file mount path as the value of the environment setting TONIC_FILE_MOUNT_POINT. For a file mount workspace, you can write the output to Amazon S3 or back to the file mount. Similar to cloud storage workspaces, you can select files or folders, and filter the file types to include.
