Loading...

Wednesday, April 23, 2014

Groovy Goodness: Customize ToString Creation

The @ToString AST transformation has several parameters we can define to customize the generated code for the toString method. We have already seen some of the parameters in an earlier blog post, but in new Groovy releases some extra parameters were added.

For example we can leave out the package name of the class with the parameter includePackage. If we set the value to false the package name is not included:

package com.mrhaki.blog.groovy

import groovy.transform.*

@ToString(includePackage=false /* default true */)
class Course {
    String title
    Integer maxAttendees
}

final Course course = new Course(title: 'Groovy 101', maxAttendees: 200)

assert course.toString() == 'Course(Groovy 101, 200)'

We can skip properties with null value from the toString method output with the paramater ignoreNulls:

package com.mrhaki.blog.groovy

import groovy.transform.*

@ToString(ignoreNulls=true /* default false */)
class Course {
    String title
    Integer maxAttendees
}

final Course course = new Course(title: 'Groovy 101')

assert course.toString() == 'com.mrhaki.blog.groovy.Course(Groovy 101)'

And since Groovy 2.1 we can cache the result of toString when we know the class is immutable or at lease we know the values won't change:

package com.mrhaki.blog.groovy

import groovy.transform.*

@ToString(cache=true /* default false */)
class Course {
    String title
    Integer maxAttendees
}

Course course = new Course(title: 'Groovy 101', maxAttendees: 200)

assert course.toString() == 'com.mrhaki.blog.groovy.Course(Groovy 101, 200)'

// Value change will not be reflected in toString().
course.title = 'Grails with REST'

assert course.toString() == 'com.mrhaki.blog.groovy.Course(Groovy 101, 200)'
assert course.title == 'Grails with REST'

Code written with Groovy 2.2.2.

Groovy Goodness: Define Compilation Customizers With Builder Syntax

Since Groovy 2.1 we can use a nice builder syntax to define customizers for a CompileConfiguration instance. We must use the static withConfig method of the class CompilerCustomizationBuilder in the package org.codehaus.groovy.control.customizers.builder. We pass a closure with the code to define and register the customizers. For all the different customizers like ImportCustomizer, SecureASTCustomizers and ASTTransformationCustomizer there is a nice compact syntax.

In the following sample we use this builder syntax to define different customizers for a CompileConfiguration instance:

package com.mrhaki.blog

import org.codehaus.groovy.control.customizers.ASTTransformationCustomizer
import org.codehaus.groovy.control.CompilerConfiguration
import org.codehaus.groovy.control.customizers.builder.CompilerCustomizationBuilder
import groovy.transform.*


def conf = new CompilerConfiguration()

// Define CompilerConfiguration using 
// builder syntax.
CompilerCustomizationBuilder.withConfig(conf) {
    ast(TupleConstructor)
    ast(ToString, includeNames: true, includePackage: false)
    
    imports {
        alias 'Inet', 'java.net.URL'
    }
    
    secureAst {
        methodDefinitionAllowed = false
    }
}


def shell = new GroovyShell(conf)
shell.evaluate '''
package com.mrhaki.blog

class User {
    String username, fullname
}

// TupleConstructor is added.
def user = new User('mrhaki', 'Hubert A. Klein Ikkink')

// toString() added by ToString transformation.
assert user.toString() == 'User(username:mrhaki, fullname:Hubert A. Klein Ikkink)'

// Use alias import.
def site = new Inet('http://www.mrhaki.com/')
assert site.text
'''

Code written with Groovy 2.2.2.

Groovy Goodness: Restricting Script Syntax With SecureASTCustomizer

Running Groovy scripts with GroovyShell is easy. We can for example incorporate a Domain Specific Language (DSL) in our application where the DSL is expressed in Groovy code and executed by GroovyShell. To limit the constructs that can be used in the DSL (which is Groovy code) we can apply a SecureASTCustomizer to the GroovyShell configuration. With the SecureASTCustomizer the Abstract Syntax Tree (AST) is inspected, we cannot define runtime checks here. We can for example disallow the definition of closures and methods in the DSL script. Or we can limit the tokens to be used to just a plus or minus token. To have even more control we can implement the StatementChecker and ExpressionChecker interface to determine if a specific statement or expression is allowed or not.

In the following sample we first use the properties of the SecureASTCustomizer class to define what is possible and not within the script:

package com.mrhaki.blog

import org.codehaus.groovy.control.customizers.SecureASTCustomizer
import org.codehaus.groovy.control.CompilerConfiguration
import org.codehaus.groovy.ast.stmt.*
import org.codehaus.groovy.ast.expr.*
import org.codehaus.groovy.control.MultipleCompilationErrorsException

import static org.codehaus.groovy.syntax.Types.PLUS
import static org.codehaus.groovy.syntax.Types.MINUS
import static org.codehaus.groovy.syntax.Types.EQUAL

// Define SecureASTCustomizer to limit allowed
// language syntax in scripts.
final SecureASTCustomizer astCustomizer = new SecureASTCustomizer(
    // Do not allow method creation.
    methodDefinitionAllowed: false,

    // Do not allow closure creation.
    closuresAllowed: false,

    // No package allowed.
    packageAllowed: false,

    // White or blacklists for imports.
    importsBlacklist: ['java.util.Date'],
    // or importsWhitelist
    staticImportsWhitelist: [],
    // or staticImportBlacklist
    staticStarImportsWhitelist: [],
    // or staticStarImportsBlacklist

    // Make sure indirect imports are restricted.
    indirectImportCheckEnabled: true,

    // Only allow plus and minus tokens.
    tokensWhitelist: [PLUS, MINUS, EQUAL],
    // or tokensBlacklist

    // Disallow constant types.
    constantTypesClassesWhiteList: [Integer, Object, String],
    // or constantTypesWhiteList
    // or constantTypesBlackList
    // or constantTypesClassesBlackList
    
    // Restrict method calls to whitelisted classes.
    // receiversClassesWhiteList: [],
    // or receiversWhiteList
    // or receiversClassesBlackList
    // or receiversBlackList

    // Ignore certain language statement by
    // whitelisting or blacklisting them.
    statementsBlacklist: [IfStatement],
    // or statementsWhitelist

    // Ignore certain language expressions by
    // whitelisting or blacklisting them.
    expressionsBlacklist: [MethodCallExpression]
    // or expresionsWhitelist
)

// Add SecureASTCustomizer to configuration for shell.
final conf = new CompilerConfiguration()
conf.addCompilationCustomizers(astCustomizer)

// Create shell with given configuration.
final shell = new GroovyShell(conf)

// All valid script.
final result = shell.evaluate '''
def s1 = 'Groovy'
def s2 = 'rocks'
"$s1 $s2!"
'''

assert result == 'Groovy rocks!'

// Some invalid scripts.
try {
    // Importing [java.util.Date] is not allowed
    shell.evaluate '''
    new Date() 
    '''
} catch (MultipleCompilationErrorsException e) {
    assert e.message.contains('Indirect import checks prevents usage of expression')
}


try {
    // MethodCallExpression not allowed
    shell.evaluate '''
    println "Groovy rocks!" 
    '''
} catch (MultipleCompilationErrorsException e) {
    assert e.message.contains('MethodCallExpressions are not allowed: this.println(Groovy rocks!)')
}

To have more fine-grained control on which statements and expression are allowed we can implement the StatementChecker and ExpressionChecker interfaces. These interfaces have one method isAuthorized with a boolean return type. We return true if a statement or expression is allowed and false if not.

package com.mrhaki.blog

import org.codehaus.groovy.control.customizers.SecureASTCustomizer
import org.codehaus.groovy.control.CompilerConfiguration
import org.codehaus.groovy.ast.stmt.*
import org.codehaus.groovy.ast.expr.*
import org.codehaus.groovy.control.MultipleCompilationErrorsException

import static org.codehaus.groovy.control.customizers.SecureASTCustomizer.ExpressionChecker
import static org.codehaus.groovy.control.customizers.SecureASTCustomizer.StatementChecker


// Define SecureASTCustomizer.
final SecureASTCustomizer astCustomizer = new SecureASTCustomizer()

// Define expression checker to deny 
// usage of variable names with length of 1.
def smallVariableNames = { expr ->
    if (expr instanceof VariableExpression) {
        expr.variable.size() > 1
    } else {
        true
    }
} as ExpressionChecker

astCustomizer.addExpressionCheckers smallVariableNames


// In for loops the collection name
// can only be 'names'.
def forCollectionNames = { statement ->
    if (statement instanceof ForStatement) {
        statement.collectionExpression.variable == 'names'
    } else {    
        true
    }
} as StatementChecker

astCustomizer.addStatementCheckers forCollectionNames


// Add SecureASTCustomizer to configuration for shell.
final CompilerConfiguration conf = new CompilerConfiguration()
conf.addCompilationCustomizers(astCustomizer)

// Create shell with given configuration.
final GroovyShell shell = new GroovyShell(conf)

// All valid script.
final result = shell.evaluate '''
def names = ['Groovy', 'Grails']
for (name in names) {
    print "$name rocks! " 
}

def s1 = 'Groovy'
def s2 = 'rocks'
"$s1 $s2!"
'''

assert result == 'Groovy rocks!'

// Some invalid scripts.
try {
    // Variable s has length 1, which is not allowed.
    shell.evaluate '''
    def s = 'Groovy rocks'
    s
    '''
} catch (MultipleCompilationErrorsException e) {
    assert e.message.contains('Expression [VariableExpression] is not allowed: s')
}


try {
    // Only names as collection expression is allowed.
    shell.evaluate '''
    def languages = ['Groovy', 'Grails']
    for (name in languages) {
        println "$name rocks!" 
    }
    '''
} catch (MultipleCompilationErrorsException e) {
    assert e.message.contains('Statement [ForStatement] is not allowed')
}

Code written with Groovy 2.2.2.

Thursday, April 17, 2014

Grails Goodness Notebook is Published

Today Grails Goodness Notebook is published. This book is an electronic publication with all blog posts about Grails Goodness bundled. The posts are slightly edited and categorized into sections.

The book is published at Leanpub and is available in three formats: PDF, MOBI (for Kindle) and EPUB (for iPad). Updates for the book are free. So when new Grails Goodness blog posts will be added to the book you will get those updates for free.

It is also possible to buy a Goodness Notebooks bundle. This bundle contains the Groovy and Grails Goodness Notebook books.

I hope you will enjoy the book and I will keep it up-to-date with new content when I publish new Grails Goodness blog posts.

Wednesday, April 16, 2014

Grails Goodness: Extending IntegrateWith Command

We can extend the integrate-with command in Grails to generate files for a custom IDE or build system. We must add a _Events.groovy file to our Grails projects and then write an implementation for the eventIntegrateWithStart event. Inside the event we must define a new closure with our code to generate files. The name of the closure must have the following pattern: binding.integrateCustomIdentifier. The value for CustomIdentifier can be used as an argument for the integrate-with command.

Suppose we want to extend integrate-with to generate a simple Sublime Text project file. First we create a template Sublime Text project file where we define folders for a Grails application. We create the folder src/ide-support/sublimetext and add the file grailsProject.sublimetext-project with the following contents:

{
    "folders": [
        {
            "name": "Domain classes",
            "path": "grails-app/domain"
        },
        {
            "name": "Controllers",
            "path": "grails-app/controllers"
        },
        {
            "name": "Taglibs",
            "path": "grails-app/taglib"
        },
        {
            "name": "Views",
            "path": "grails-app/views"
        },
        {
            "name": "Services",
            "path": "grails-app/services"
        },
        {
            "name": "Configuration",
            "path": "grails-app/conf"
        },
        {
            "name": "grails-app/i18n",
            "path": "grails-app/i18n"
        },
        {
            "name": "grails-app/utils",
            "path": "grails-app/utils"
        },
        {
            "name": "grails-app/migrations",
            "path": "grails-app/migrations"
        },
        {
            "name": "web-app",
            "path": "web-app"
        },
        {
            "name": "Scripts",
            "path": "scripts"
        },
        {
            "name": "Sources:groovy",
            "path": "src/groovy"
        },
        {
            "name": "Sources:java",
            "path": "src/java"
        },
        {
            "name": "Tests:integration",
            "path": "test/integration"
        },
        {
            "name": "Tests:unit",
            "path": "test/unit"
        },
        {
            "name": "All files",
            "follow_symlinks": true,
            "path": "."
        }
    ]
}

Next we create the file scripts/_Events.groovy:

includeTargets << grailsScript("_GrailsInit")

eventIntegrateWithStart = {

    // Usage: integrate-with --sublimeText
    binding.integrateSublimeText = {

        // Copy template file.
        ant.copy(todir: basedir) {
            fileset(dir: "src/ide-support/sublimetext/")
        }

        // Move template file to real project file with name of Grails application.
        ant.move(file: "$basedir/grailsProject.sublime-project", 
                 tofile: "$basedir/${grailsAppName}.sublime-project", 
                 overwrite: true)

        grailsConsole.updateStatus "Created SublimeText project file"
    }
}

We are done and can now run the integrate-with command with the new argument sublimeText:

$ grails integrate-with --sublimeText
| Created SublimeText project file.
$

If we open the project in Sublime Text we see our folder structure for a Grails application:

Code written with Grails 2.3.7.

Grails Goodness: Generate Default .gitignore Or .hgignore File

We can use the integrateWith command with Grails to generate for example IDE project files and build system files. We specify via an extra argument the type of files to be generated. We can use this command also to create a .gitignore file with some default settings for files to be ignored in Grails projects.

$ grails integrate-with --git

In the root of our project we have now have a .gitignore file with the following contents:

*.iws
*Db.properties
*Db.script
.settings
stacktrace.log
/*.zip
/plugin.xml
/*.log
/*DB.*
/cobertura.ser
.DS_Store
/target/
/out/
/web-app/plugins
/web-app/WEB-INF/classes
/.link_to_grails_plugins/
/target-eclipse/

If we would use Mercurial then we can generate a .hgignore file with the argument --hg:

$ grails integrate-with --hg

The .hgignore file has the following contents:

syntax: glob
*.iws
*Db.properties
*Db.script
.settings
stacktrace.log
*.zip
plugin.xml
*.log
*DB.*
cobertura.ser
.DS_Store
target/
out/
web-app/plugins
web-app/WEB-INF/classes

Samples written with Grails 2.3.7.

Friday, April 11, 2014

Coloring Different Data Sources in IntelliJ IDEA

The database plugin in IntelliJ IDEA is a useful tool to work with data in databases. As long as we got a JDBC driver to connect to the database we can configure a data source. And then we can run queries, inspect the contents of tables and change data with the database tool window. It is not uncommon to have multiple data sources, for example development and test environment databases, which will have the same tables. When we open the tables or run queries we don't have a visual feedback to see to which data source such a table belongs. To have a visual feedback we can colorize our data source. This means we assign a color to a data source and when we open a table from that data source the tab color in the editor window will have a different color than other tabs or the background color of the data source objects have a color.

To add a color to a data source we must open the database tool window and right click on a data source. We select the option Color Settings... from the popup window:

Next a new dialog opens where we can select a color:

We can make a selection for one of the predefined colors or create a custom color we want to use. Also we can select in the Appearance Settings where in IntelliJ IDEA the colored data source must appear. We click on the OK button to save our settings. We can repeat these steps for other data sources and given them different colors.

Once we have added color to our data source we can see for example in the tabs of our editor window the different colors:

Or when we open the data sources in the database tool window to get a list of all objects in the data source:

Even we open a dialog to see recently changed files we can see the colorized data source objects:

Sample with IntelliJ IDEA 13.1.1

Friday, April 4, 2014

Groovy Goodness: Closure as Writable

In a previous post we learned about the Writable interface and how the GString implementation implements this interface. In Groovy we can also use a closure as an implementation of the Writable interface. The Closure class has the method asWritable() that will return a version of the closure with an implementation of the writeTo() method. The Writer object that is used as an argument for the writeTo() method will be passed as argument to the closure. The asWritable() method also adds a toString() implementation for the closure to return the result of a closure as a String.

In the following code we write a sample make() method. The make() method return a Writable closure. The closure is only executed when the writeTo() or toString() method is invoked.

Writable make(Map binding = [:], Closure template) {
    // Use asWritable() to make the closure
    // implement the Writable interface.
    def writableTemplate = template.asWritable()
    
    // Assing binding map as delegate so we can access
    // the keys of the maps as properties in the 
    // closure context.
    writableTemplate.delegate = binding
    
    // Return closure as Writable.
    writableTemplate
}

// Use toString() of Writable closure.
assert make { Writer out -> out <<  "Hello world!" }.toString() == 'Hello world!'

// Provide data for the binding.
// The closure is not executed when the 
// make method is finished.
final writable = make(user:'mrhaki', { out ->
    out.println "Welcome ${user},"
    out.print "Today on ${new Date(year: 114, month: 3, date: 4).format('dd-MM-yyyy')}, "
    out.println "we have a Groovy party!"
})

// We invoke toString() and now the closure
// is executed.
final result = writable.toString()

assert result == '''Welcome mrhaki,
Today on 04-04-2014, we have a Groovy party!
'''

// Append contents to a file.
// NOTE: The leftShift (<<) operator on File is implemented
// in Groovy to use the File.append() method.
// The append() method creates a new Writer and
// invokes the write() method which 
// is re-implemented in Groovy if the argument
// is a Writable object. Then the writeTo() method
// is invoked:
// Writer.write(Writable) becomes Writable.writeTo(Writer).
// So a lot of Groovy magic allows us to use the following one-liner
// and still the writeTo() method is used on Writable.
new File('welcome.txt') << writable

assert new File('welcome.txt').text == '''Welcome mrhaki,
Today on 04-04-2014, we have a Groovy party!
'''

Code written with Groovy 2.2.2

Groovy Goodness: GString as Writable

The Groovy API has the interface Writable. Classes that implement this interface are capable of writing their selves to a java.io.Writer object. The interface has one method writeTo() where the code is that writes the contents to a given Writer instance. Most implementations will also use the implementation of the writeTo() method in their toString() implementation.

The GString implementation in Groovy also implements the Writable interface. This means we can redirect the GString contents to some Writer instance if we want to. In the following code we use a FileWriter to write the contents of a GString to a file:

def data = [
    new Expando(id: 1, user: 'mrhaki', country: 'The Netherlands'),
    new Expando(id: 2, user: 'hubert', country: 'The Netherlands'),
]

data.each { userData ->
    new File("${userData.id}.txt").withWriter('UTF-8') { fileWriter ->
        // Use writeTo method on GString to save
        // result in a file.
        "User $userData.user lives in $userData.country".writeTo(fileWriter)
    }
}


assert new File('1.txt').text == 'User mrhaki lives in The Netherlands'
assert new File('2.txt').text == 'User hubert lives in The Netherlands'

Code written with Groovy 2.2.2

Thursday, April 3, 2014

Groovy Goodness: Converting Byte Array to Hex String

To convert a byte[] array to a String we can simply use the new String(byte[]) constructor. But if the array contains non-printable bytes we don't get a good representation. In Groovy we can use the method encodeHex() to transform a byte[] array to a hex String value. The byte elements are converted to their hexadecimal equivalents.

final byte[] printable = [109, 114, 104, 97, 107, 105]

// array with non-printable bytes 6, 27 (ACK, ESC)
final byte[] nonprintable = [109, 114, 6, 27, 104, 97, 107, 105]


assert new String(printable) == 'mrhaki'
assert new String(nonprintable) != 'mr  haki'


// encodeHex() returns a Writable
final Writable printableHex = printable.encodeHex()
assert printableHex.toString() == '6d7268616b69'
final nonprintableHex = nonprintable.encodeHex().toString()
assert nonprintableHex == '6d72061b68616b69'


// Convert back
assert nonprintableHex.decodeHex() == nonprintable

Code written with Groovy 2.2.1