Modern data generation tools allow you to generate not just dummy data, but quite meaningful test data. Data such as personal names, streets names, email addresses, phone numbers, bank codes and much, much more can be easily generated.
The development of new functionality often leads to the creation of new tables or modification of the existing ones. Therefore, in order to check the efficiency of the new functionality, you will need to fill these tables with data. You can do this quickly and easily using generation tools.
Taking data for testing functionality from a real customer has many disadvantages and inconveniences. The main one is that customers are often reluctant to share data since they need to spend time on data masking or other measures to protect their confidential information.
You can also fill the database with data using replications, but this method will most likely require a production server load and a database administrator involvement. And most importantly, during business hours you will receive a refusal from the customer to take pictures since you will simply lock the tables. Yet, the CI process implies that it can be performed up to several times a day.
Generation tools allow you to generate an unlimited amount of realistic data. And if your functionality works properly with this data, then it will work properly on a customer's server as well.
Generation tools allow you to fill the database from scratch, replace the existing data or add new data. Also, it is often the case that the customer is ready to share their data with you, but this data is not enough to fully test the performance of the new functionality. At this point, data generation tools will always come to assistance.
Generation tools allow you to populate the same tables with different data sets. This is particularly beneficial when testing new functionality since it is quite possible that one generated sample can pass a set of tests, while the next - generated from the same process - will lead to their failure.
Generation tools allow you to quickly create one or more data generation projects for your database and include them in the CI process. When a project is created, logically meaningful generators for table columns are automatically assigned. Relations between tables, check constraints, and others are also taken into account.
Testing must not involve sensitive production data, legally protected at multiple levels by a number of privacy laws and regulations:
As mentioned above, a customer can share real data for testing with a provision that some data will be masked. For example, bank codes, phone numbers, email addresses and more. This can make it difficult or completely impossible for you to test your functionality since testing requires a correct or real data set.
It is often necessary to obtain data from a customer to test new functionality or upgrade. But each contact with the customer is another waste of time which you may lack if you need to meet the deadline for the product. In this case, you can save a lot of time using a data generator.
As you know, development of new functionality or testing of the existing one raises the problem of getting loads of test data quickly. It is possible to use import tools, but their usage imposes many restrictions. For instance, it is necessary to consider such factors as relations between tables and check constraints, which may lead to a big waste of time. Hopefully, modern generation tools can factor all this in.
Situations such as the development of new features or the lack of real data from the customer, they are willing to share, can also be solved through data generation. You can always generate an unlimited amount of data.
Now, when the project for populating the database with data is ready, you can add it to the Source Control repository and use it in CI when you call the cmdlet to fill the database with test data. The generator project can be used in PowerShell cmdlets Invoke-DevartDatabaseTests and Invoke-DevartPopulate, as well as during the dbForge DevOps Automation for SQL Server - Populate for Azure DevOps step and the Generate test data section of the dbForge DevOps Automation for SQL Server - Test a database using the tSQLt step for the Jenkins/Bamboo/TeamCity project.
Invoke-DevartDatabaseTests -InputObject 'Server=MSSQLSERVER; Initial Catalog=TestDB; User ID=sa' -DataGeneratorProject 'D:\SourceDG\TestDB(MSSQLLocalDB).dgen' -InstalltSQLtFramework -RewriteReport -UnInstalltSQLtFramework -IncludeTestData
$Connection = New-DevartSqlDatabaseConnection -Database TestDB -Server MSSQLSERVER -WindowsAuthentication true $DataGeneratorProject = "D:\SourceDG\TestDB(MSSQLLocalDB).dgen" Invoke-DevartPopulate -DataGeneratorProject $DataGeneratorProject -Connection $Connection
For more information on how to use cmdlets, see the Devart documentation center.