New features and enhancements
Redesigned the user experience for the subsetting feature. The new subsetting view displays a list of the source database tables and indicates whether each table is in the subset. When you click a table, the table details display in a new right-hand panel. From the details panel, you can configure target tables. You can also identify lookup tables (previously referred to as reference tables), and indicate how to handle tables that are not in the subset.
If subsetting is configured, when you run a data generation job, you can enable or disable subsetting.
Added latitude and longitude processing to the HIPAA Address generator.
You can now filter Database View based on the applied generator. On the filter panel, in the Applied Generator field, you can provide the list of generators to include.
Redesigned the Schema Changes page. The Actions list is now called Conflicting Schema Issues. Tonic provides clearer warnings when a schema change resolution will result in a change to the workspace configuration. Resolving a conflicting issue or bulk dismissing non-conflicting issues now includes a confirmation step.
Other updates
Renamed the Events generator to Event Timestamps.
Fixed some small display issues in the new Privacy Hub and Subsetting displays.
Improved event generation for the AI Synthesizer.
Improved how we retrieve CloudWatch logs to include the job ID and to use the correct Tonic version.
Corrected the processing of downstream, multi-index tables during subsetting. Corrected an issue in the initial fix.
Improved memory usage during data generation.
Amazon Redshift
- Corrected the handling of time zones in timestamps.
- Fixed test connection and generation in response to an AWS API change.
MongoDB
- UUIDs larger than 16 bytes are now truncated.
- Improved the use of MongoDB resources.
- Improved the display format of LUUIDs.
PostGreSQL
- Corrected how permissions are validated for Heroku.
Oracle
- Tonic now prevents data generation jobs from running when the Oracle versions for the source and destination database do not match.
- Improved the performance of writes to the destination database.
- After deleting a table in the destination database, Tonic now also clears out the recycle bin.
- Fixed generation for LONG and LONG RAW fields.
Snowflake
- Fixed test connection and generation in response to an AWS API change.
New features and enhancements
We enhanced Privacy Hub to add expanded top-level metric panels. These panels show the number of unprotected sensitive columns, protected columns, and unprotected non-sensitive columns. From these panels, you can display column details, select and configure the column generator, view sample data, and add column comments.
You can now add or update a Tonic license from the Tonic application. For a new instance, you are prompted to provide the license key before Tonic displays the login screen. Tonic displays a message when the current license is expired. The Admin Panel includes an option to update the license key.
For the Address generator, added City with State and City with State Abbr to the available options for the column format. You use these options for column values such as San Francisco, California or Boston, MA.
Tonic now supports subsetting for MongoDB databases.
Workspaces can now have a description (up to 200 characters) as well as a name. Use the description field to provide additional context for the workspace and how it is used.
In the Tonic API, you can now sort workspaces based on the last generation date.
Other updates
Tonic now prevents a job from running when the worker determines that the server is running a different version of Tonic.
Refactored the underlying implementation of the subsetting feature.
Made some small memory improvements for data generation.
Implemented performance improvements when applying parallel constraints.
Corrected errors for edge cases related to the Audit Trail.
When the selected workspace changes, the identifier in the URL is now updated correctly.
The workspace configuration is migrated to a data type that enables more efficient querying.
Tonic now validates uploaded foreign keys against the table definitions.
MongoDB
- MongoDB aggregations can now use temporary files on disk to store data that exceeds the MongoDB size limit. This expands the possible generations for a MongoDB database.
- Corrected the generation and display of UUIDs.
- ObjectIds can now be used as primary keys for subsetting in MongoDB.
- Improved the usability of the Add Foreign Key Relationships tab on the Foreign Key Relationships view.
Oracle
- A destination database can now have more than 1000 tables that have the table mode set to Preserve Destination.
- Improved how Tonic handles maximum lengths when it generates the following data types:NCHAR, NVARCHAR2, CHAR, VARCHAR2
PostgreSQL
- Batch sizes are now set dynamically based on the average row size.
- Memory improvements for PostgreSQL data generation that involve large rows.
Spark
- Improved support for foreign keys.
- Added support for Apache Livy and HDFS.
- Improved performance and added SDK support for the Integer Key generator.
Refreshed the Audit Trail user interface on Privacy Hub. The new Protection Audit Trail provides a paginated list of the updates to the sensitivity designation and generator assignments.
Deep links now work correctly when you use Google SSO to authenticate.
Error messages from Oracle are now displayed in response to invalid where clauses in subset configuration.
Made minor memory improvements to the Address generator.
Snowflake
- Reduced the frequency of schema change detection on Snowflake databases. This can result in cost savings on Snowflake clusters, because the clusters can sleep more often.
- Snowflake generation now works correctly when there are foreign key constraints.
SQL Server
- Added support for security policies, sequences, check constraints, and system versioned temporal tables.
Improved cross-tab support for automatic logouts when you configure an inactivity period.
The Update option in the actions menu now takes you directly to the System tab on the Admin Panel instead of the Users tab.
Corrected the password length requirement to be 12 characters or greater instead of greater than 12 characters.
Improved the estimated row progress for scaled tables.
Eliminated a race condition that occurred when applying constraints.
Databricks:
- Can now run concurrent jobs that use different versions of Databricks.
- Tonic now supports ORC and Hive tables in Databricks.
SQL Server:
- The Categorical generator can now support more than 2 billion rows in a category.
When users create a new password, Tonic now displays a panel with the password requirements, and indicates whether the password meets those requirements.
Improved the parallelization and concurrency for processing foreign key constraints.
Databricks and Spark EMR
- Improved the performance of the Noise generator.
MongoDB
- Improved display of longer key values in the Key column. Widened the column and added truncation.
- Optimized queries against the Tonic database.
Features
For PostgreSQL databases, Tonic now supports name and char data types.
For Tonic single sign-on, Tonic now supports Azure Active Directory.
From the administration screen, administrators for customers that run Tonic on Kubernetes and Docker can now download logs from all containers that run Tonic.
For Spark-powered integrations, Tonic now supports the Address generator as a sub-generator.
Bugs
Improved performance for:
- Random Hash generator on Spark and Databricks
- UUID Generator on Spark
- Subsetting
Improved the browsing experience on low resolution displays.
Features
Skip batch instead of failing generations on Postgres in some cases when values fail to be inserted
Improvement to Synthesis Reports for AI Synthesizer generated data
Changing Lambda deployment from ECR to ZIP for integrations using Lambda functions. Images no longer need to be manually deployed to ECR.
Bugs
Fix for One-Click Tonic update when the PostgreSQL application database is deployed in a Docker container
Features
Support for arrays with Struct Mask generator on Amazon EMR/Spark and Databricks
Bugs
Fix issue causing privacy scans on Dremio to fail
Minor UI fixes to workspace page
Fix for data generation issue on Db2 iSeries when destination is empty
More robust handing of Databricks host URLs
Features
Add data type advanced search for Oracle, Amazon Redshift, and BigQuery
Performance improvements for constraint restoration
Improved performance of Random Integer generator on Spark
Bugs
Fix issue on SQL Server preserve destination mode when table names contain a ".".
PostgreSQL: Fix issue copying arrays with trailing slashes
Features
For Amazon S3/EMR data source, support separately testing each component of the connection (Glue catalog, EMR cluster, and S3 bucket)
The Additive Noise Generator is renamed to Noise Generator with two noise options - Additive (existing) and Multiplicative (new)
Improve Javadocs for Spark SDK and other enhancements
Bugs
Fixes for the SQL beautifier on post-job scripts, including scenarios that may have crashed the app on Safari
Skip processing Temporal Types when not supported (SQL Server 2012) to prevent job failures
Features
Support graceful cancellation of privacy metric computation for AI Synthesizer
Java SDK documentation available from the SDK Setup dialog
Bugs
Only show SSO and Enterprise licensing information to users when appropriate
Minor UI fixes for scrolling on the workspace view table
Minor UI fixes for the Edit Workspace page
Features
Add support for Azure Datalake Storage Gen 2 as an output destination for Databricks
Don't include statistics for Oracle schema copy
MongoDB: Support uploading foreign key file, support deleting configured foreign keys
Bugs
Do not stop generation in SQL Server if schema pre-fetch fails
Improved deletion of old logs to prevent timeouts
Features
Update subsetting tooltip help text
Don't check destination recovery mode on Azure SQL
Improve performance of table constraint application when subsetting by applying single and cross table constraints in parallel
Bugs
Fix logo navigation redirect when signed in as a user with no workspaces
Fix issue with setting a partition filter on Spark
Fix issue with collection picker on MongoDB showing removed collections
Fix excessive memory usage of Hostname generator
Features
Improves performance of Custom Categorical generator on Spark and Databricks
Add support for Azure Databricks
Display SQL Server schema generation progress as a percentage
Bugs
Do not fetch column data on configuration popover open to reduce queries against the source database
Reduce the number of queries to the source database for table/column details during generation
Features
Spark 2.4 and Databricks Library Improvements
Add ability to remove users in the Admin Panel
Bugs
Fixed issue with the displayed port when connecting to MongoDB via connection string
Fix for data table column resizing not rendering properly
Support random latitude and longitudes with HIPAA Address generator
Features
Add support for Spark SDK on Spark v2.4
Bugs
Fix UI crash when selecting incremental as Table Mode
Fix AI Synthesizer when no categorical columns present
Ensure AI Synthesizer model is not trained if only one column is selected
Render HTML Mask in the generator model panel on Table View
Fix issue with not being able to set incremental table mode column if you refresh after setting it to incremental
