GPARS Concurrent Collection Processor In Grails Integration Test

Recently I came across one scenario in which we have to test one service in grails application

We want this to be done via multithreading so that multiple thread can execute the service method concurrently using grails integration test

So the very effective solution we found was GPARS

In GPars the GParsPool and GParsExecutorsPool classes give you access to low-level data parallelism techniques

This utility gives you facility of processing collections concurrently. So the size of collection will be the

No. of user for us and the collection data can be use as param pass to the service method

Below is the sample code chunk.

You need to import the below class in you class

import com.mongodb.gridfs.GridFS

also need to configure NO_OF_THREADS require for processing

suppose we have the list of size 1000 containing the user ID as string data

def userList = new ArrayList(1000);

userList.add(“1”)

Now in test Method we write the Closure which will process our collection as shown below.

GParsPool.withPool(NO_OF_THREADS){ final Closure processUser = {userId, index -> def response = service.processUser(userId) assertNotNull response } assertNotNull(userList) userList.eachWithIndex(processUser.async()) }

This will work as concurrent collection processor for our userList with given NO_OF_THREADS . we can pass the index and its value as parameter to the service method.

Install the MYSQL along with the InnoDB plugin

Introduction

This blog represent the procedure to install the MYSQL along with the InnoDB plugin.

InnoDB can be install from SQL INSTALL command or by configuring my.cnf, here we are using my.cnf to load and configure InnoDB plugin.

NOTE: If you already have MYSQL install and just want to install the InnoDB plugin please don't follow these steps.

Download MYSQL

Download the MYSQL packages using below linux command

Note : User must have download rights to download these packages

MySQL-client-community :- The standard MySQL client programs. You probably always want to install this package.

wget http://downloads.skysql.com/archives/mysql-5.1/MySQL-client-community-5.1.54-1.rhel5.x86_64.rpm

MySQL-server-community :- The MySQL server. You need this unless you only want to connect to a MySQL server running on another machine.

wget http://downloads.skysql.com/archives/mysql-5.1/MySQL-server-community-5.1.54-1.rhel5.x86_64.rpm

Create or Modify before mysqlserver is installed /etc/my.cnf

Below is the standard MYSQL my.cnf configuration referred from QA database server

[mysqld] configuration

[mysqld]

datadir=/var/lib/mysql

socket=/var/lib/mysql/mysql.sock

user=mysql

#Default to using old password format for compatibility with mysql 3.x

#clients (those using the mysqlclient10 compatibility package).

#you can/should enable old passwords by setting old_passwords=1 else comment it

old_passwords=1

#Max no of clients to connect

max_connections=750

#unique id for the server

server-id=1

#Innodb configuration

innodb_flush_method=O_DIRECT

#set these value as per memory available

innodb_buffer_pool_size=6G

innodb_flush_log_at_trx_commit=1

#set these value as per memory available

innodb_log_file_size=64M

innodb_log_files_in_group=2

ignore_builtin_innodb

#Be careful while copying.There should be no space

plugin-load=innodb=ha_innodb_plugin.so;innodb_trx=ha_innodb_plugin.so;innodb_locks=ha_innodb_plugin.so;innodb_lock_waits=ha_innodb_plugin.so;innodb_cmp=ha_innodb_plugin.so;innodb_cmp_reset=ha_innodb_plugin.so;innodb_cmpmem=ha_innodb_plugin.so;innodb_cmpmem_reset=ha_innodb_plugin.so

default-storage-engine=InnoDB

[mysql.server] configuration

[mysql.server]

user=mysql

[mysqld_safe] configuration

[mysqld_safe]

log-error=/var/log/mysqld.log

pid-file=/var/run/mysqld/mysqld.pid

Installation

Install it using below command

rpm -ivh MySQL-client-community-5.1.54-1.rhel5.x86_64.rpm

rpm -ivh MySQL-server-community-5.1.54-1.rhel5.x86_64.rpm

[edit]

Set The Password

Once you have install all the package's please remember to set password for the MySQL root USER ! To do so, issue the below linux commands:

/usr/bin/mysqladmin -u root password 'new-password'

Note: above mysqladmin command will execute only when mysql server is up, here installation of MySQL-server-community-5.1.54-1.rhel5.x86_64.rpm package will start the server. if its not running then start the server using command

/etc/init.d/mysql start

Create User

execute below commands on Mysql

Mysql>create user 'userName'@'hostName' identified by 'password';

Mysql>GRANT SELECT, INSERT, UPDATE, DELETE, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, SHOW VIEW ON databaseName.* TO 'userName'@'hostName'

Note: hostName here is the remote(app server) machine's host name

Xtrabackup Installation and Restore For MYSQL + INNODB

Installation

a) Using Percona Software Repositories

Run below command. This will create the Percona YUM repository

sudo rpm -Uhv http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm

Testing the Repository: This will display the repository list

sudo yum list | grep percona

Check the Xtrabackup install or not

rpm -qi xtrabackup

If it is not installed then install using below command and verify again using above command

sudo yum install xtrabackup

b) Using Generic .tar.gz binary packages

If you don’t want to install using Percona YUM repository use the below link to download ready to use Xtrabackup tool

http://www.percona.com/downloads/XtraBackup/XtraBackup-1.5/Linux/binary/

Note: - For Innodb database backup you must have innodb_file_per_table set in my.cnf.

Creating Backup

sudo ./innobackupex-1.5.1 --user=username --password=password /path/to/save/backup --databases=”databaseName”

Note: - the user should have the read/write access to the both data and target folder, also check .frm file copied or not in database backup folder

Backup Completes with following statements:

120209 01:57:22 innobackupex-1.5.1: completed OK!

Restore Backup

Prepare using binary package

Sudo ./innobackupex-1.5.1 --apply-log /path/to/save/backup/timestamp_folder

Copy Data directory

You could simply use cp to copy the files, also make sure the data directory and files are owned by the mysql user.

Stop the server and then run the below copy command

cp -R /path/to/save/backup/timestamp_folder/* /var/lib/mysql/

And run the below command

sudo chown –R mysql:mysql /var/lib/mysql

Start the server and check the database

Reference:

http://www.percona.com/doc/percona-xtrabackup/howtos/recipes_xbk_restore.html

http://agiletesting.blogspot.com/2010/09/mysql-innodb-hot-backups-and-restores.html

http://www.ovaistariq.net/590/on-hot-backups-and-restore-using-xtrabackup/