In a CI/CD environment, everything from development to testing to deployment moves quickly. It can be difficult to pass data between environments at all—let alone worry about whether that data is safe. Development and staging databases might have different schemas to account for what’s currently under development, which is, of course, totally different from what’s currently being tested in staging. Maybe you’ve deployed a container or two to test new code without affecting a staging environment… but it hasn’t totally solved your problems.  

But you’re a smart developer. You’ve come up with a solution or two. Creating new PostgreSQL images solved the problem of database changes for testing, but you still need the actual data. In many scenarios, having real-world production data would be the ideal way to test your application instead of using simple dummy data that doesn’t cover all use cases—-but the times, they are a-changing, and you can’t send prod data to your lower environments anymore. You need a way to create production-like data that doesn’t infringe on data privacy or violate compliance regulations. 

That’s where Tonic comes in. Tonic gets you and your lower environments the realistic, safe data you need to fuel your CI/CD workflows. Today, we’ll cover how to integrate Tonic into your CI/CD pipeline with Jenkins, plus why that’s definitely the right move to keep development and testing moving quickly.

Prerequisites for Adding Tonic with Jenkins

You need a few things set up before you start. These prerequisites might seem obvious to some, but for anyone new to Tonic, they might not be so obvious.

The first thing you need is an EC2 instance created and started. On the EC2 instance, you need:

  • Docker compose installed
  • A PostgreSQL docker-compose.yml configuration file created. Here is an example of a docker-compose.yml configuration file:

version: '3'
services:
        testdb_postgres:
                image: postgres:12
                restart: always
                ports:
                        - 5432:5432
                environment:
                        POSTGRES_USER: user
                        POSTGRES_PASSWORD: password
                        POSTGRES_DB: test_data
                container_name: testdb_postgres
                volumes:
                        - /tmp:/tmp

  • Jenkins installed and running on your CI/CD server.
  • A Tonic instance running and configured to point to the EC2 PostgreSQL host.

Four Steps to Create the Data

There are four main steps to integrate Tonic with your CI/CD pipeline. This example uses Jenkins, but you can use other automation tools to integrate Tonic data with your deployments. 

The four main steps are below.

1. Start Postgres

You can start Postgres in your terminal with the following commands:

cd /opt/postgres

/usr/bin/docker-compose up -d

2. Run Tonic data generation

Tonic can be set up to generate data using an API. You need a Tonic API key, host, and workspace name to generate data, which you should have as a customer. You can write scripts that call Tonic endpoints to generate your test data. An example Python script is provided in our documentation. Note that we also provide Bash, Node, and PowerShell examples.

In the Python script, we create a class used to pass credentials to the Tonic API, generate data, and add it to your workspace. In the class methods, we perform a simple call to the API, generate data using the /api/generateData endpoint, and then provide you with JSON results in your workspace. This script can be used to create test data that will be used in the next step to deploy a PostgreSQL test container.

You can create your own script or use our example. The following command will execute the Python script (named tonic_generation.py) to generate data:

python /opt/scripts/tonic_generation.py

3. Create a new PostgreSQL image

The next step is to create the PostgreSQL image with the data and schema that you just generated in step two. The following is an example Bash script to deploy a PostgreSQL image with your new data, which is stored in a .tar file:

#!/bin/bash

SHA=$(docker commit tonic_db)
ID=${SHA:7:12}
echo $ID
TAG_NAME=tonic_db_image-$(date +'%Y%m%d')
echo $TAG_NAME
docker tag $ID $TAG_NAME
docker save $TAG_NAME > /opt/data/$TAG_NAME.tar

Save this file, name it create_image.sh, and use the following command to deploy your image:

sh /opt/scripts/create_image.sh

4. Stop PostgreSQL

At some point, you will want to stop the instance to destroy it for your next test image. The following command will stop PostgreSQL:

cd /opt/postgres

/usr/bin/docker-compose down

FAQ About Tonic + Jenkins

As the above shows, Tonic is designed to fit seamlessly into your CI/CD automation processes. It’s a platform built by developers, for developers, to get you the safe and useful data you need to do your work. With Tonic, you’ll have refreshed data for every run of your Jenkins projects (or other automation tools).

A few considerations and answers to questions that you might have:

  • Tonic automatically detects schema changes and can be configured to block generation until any new columns have been protected to prevent data leakage.
  • Other alerts from Tonic are also available, as you can see in step two’s Python script example. In the example, we have error handling that displays Tonic API alerts. Tools like Jenkins have their own alert system too.
  • You don’t need a standing Tonic instance. Instead, you can import JSON or the metadata from Tonic.

Integrate Tonic Into Your CI/CD Pipeline with Jenkins

CI/CD is integral to a fast-paced development environment, so you need a way for your test data generation to be as quick as your developers. Tonic makes it easy for you to integrate test data, keep your environment compliant, ensure data privacy and security, and empower developers to continue their current CI/CD pace. 

Pro tip: Tonic can actually integrate with any automation framework, not just Jenkins!

Want to learn more about how to integrate Tonic into your CI/CD pipeline? Check out our discord for real live help from real live devs, or shoot us a line at hello@tonic.ai

Thomas Eckerle
Solutions Manager
Head of Solutions Architecture, the people responsible for taking customers from zero to Tonic as quickly as possible.

Fake your world a better place

Enable your developers, unblock your data scientists, and respect data privacy as a human right.