Groovy Goodness: Working with Dates

Thanks to Groovy's extensions to the JDK Date classes we can work with dates more easily. For example we can now add and subtract days with the plus() and minus() methods. And because this methods are mapped to the operators + and - we can write dense code.

Also the next() and previous() methods are implemented so we can use the ++ and -- operators to get to a next or previous day. We can even use the subscript operator ([]) to get date fields, because of the getAt() method Groovy adds to the Date class.

The Date class also contains parse() and format() methods to convert string to a date and to format a date into a string.

// Date.parse() to convert String to Date.
date = new Date().parse('yyyy/MM/dd', '1973/07/09')

// We can use [] or getAt() to get date fields.
assert 1973 == date[Calendar.YEAR]
assert 6 == date[Calendar.MONTH]
assert 9 == date.getAt(Calendar.DATE)

dateNext = date.clone()
datePrevious = date.clone()

// We can use the + and - operators to add or 
// subtract days.
nextDay = date + 1  // Or date.plus(1)
previousDay = date - 1  // Or date.minus(1)

// ++ operator to move one day ahead.
dateNext++  // Or dateNext.next()
assert dateNext == nextDay

// -- operator to move one day back.
datePrevious--  // Or datePrevious.previous()
assert datePrevious == previousDay

otherDate = new Date().parse('yyyy/MM/dd', '1973/07/21')
// Dates can be used in ranges.
assert 12 == (otherDate..<date).size()

// Set Locale to assert date formatting.
Locale.setDefault(Locale.US)

// Date.format() uses java.text.SimpleDateFormat.
assert '9 July, 1973' == date.format("d MMMM, yyyy")
assert '7/9/73' == date.getDateString()

Groovy Goodness: The Spread-Dot Operator

Groovy adds some nice operators to the language to write brief code. We already learned about the Elvis operator and the Spaceship operator. And now we see what the spread-dot operator is about and what it does.

The spread-dot operator (*.) is used to invoke a method on all members of a Collection object. The result of using the spread-dot operator is another Collection object. Here is some sample code:

class Language {
    String lang
    def speak() { "$lang speaks." }
}

// Create a list with 3 objects. Each object has a lang 
// property and a speak() method. 
def list = [     
    new Language(lang: 'Groovy'),     
    new Language(lang: 'Java'),     
    new Language(lang: 'Scala') 
]  

// Use the spread-dot operator to invoke the speak() method. 
assert ['Groovy speaks.', 'Java speaks.', 'Scala speaks.'] == list*.speak() 
assert ['Groovy speaks.', 'Java speaks.', 'Scala speaks.'] == list.collect{ it.speak() }  

// We can also use the spread-dot operator to access 
// properties, but we don't need to, because Groovy allows 
// direct property access on list members. 
assert ['Groovy', 'Java', 'Scala'] == list*.lang 
assert ['Groovy', 'Java', 'Scala'] == list.lang

Groovy Goodness: Delegate to Simplify Code

Groovy supports the @Delegate annotation. With this annotation we can import all the methods of the class the annotation is used for. For example if we use the delegate annotation for the Date class we get all the methods of the Date class in our class. Just like that. This is best explained with a little sample in which we use the @Delegate annotation for properties of type Date and List:

class SimpleEvent {
    @Delegate Date when
    @Delegate List<String> attendees = []
    int maxAttendees = 0
    String description
}

def event = new SimpleEvent(when: new Date() + 7, description: 'Small Groovy seminar', maxAttendees: 2)

assert 0 == event.size()  // Delegate to List.size()
assert event.after(new Date())  // Delegate to Date.after()
assert 'Small Groovy seminar' == event.description
assert 2 == event.maxAttendees

event << 'mrhaki' << 'student1'  // Delegate to List.leftShift()
assert 2 == event.size()
assert 'mrhaki' == event[0]

event -= 'student1'  // Delegate to List.minus()
assert 1 == event.size()

We have used the @Delegate annotations and as by magic the SimpleEvent has all methods of both the Date class and List interface. The code reads naturally and the meaning is obvious. Because the SimpleEvent class has all methods from the List interface we can override the methods as well. In our sample we override the add() so we can check if the number of attendees doesn't exceed the maximum number of attendees allowed:

class SimpleEvent {
    @Delegate Date when
    @Delegate List<String> attendees = []
    int maxAttendees = 0
    String description

    boolean add(Object value) {
        if (attendees.size() < maxAttendees) {
            return attendees.add(value)
        } else {
            throw new IllegalArgumentException("Maximum of ${maxAttendees} attendees exceeded.")
        }
    }
}

def event = new SimpleEvent(when: new Date() + 7, description: 'Small Groovy seminar', maxAttendees: 2)
event << 'mrhaki' << 'student1'

try {
    event << 'three is a crowd.'
    assert false
} catch (IllegalArgumentException e) {
    assert 'Maximum of 2 attendees exceeded.' == e.message
}

Grassroots Groovy: the Spring Route (part 2)

We can introduce Groovy into our projects at grassroots level. In a previous post we learned how to invoke Groovy code from Spring configured applications. In this post we learn how to create the Spring configuration files with Groovy. We are using the BeanBuilder from Grails to achieve this. The builder implements a Spring DSL to configure beans. We can define beans and wire beans together using Groovy code, so this means we can add logic to the configuration. For example we could check an environment variable and depending on the value we define certain beans.

To define the beans we use Groovy, but we can still use Java code to read the configuration and get a valid Spring ApplicationContext. This way the impact on the Java code is minimal, but we can create the Spring configuration more dynamically. First we start with the Spring BeanBuilder file SpringBeansConfig.groovy:

import com.mrhaki.blog.*
import org.apache.commons.dbcp.*

beans {
    // We can use namespaces in the builder.
    xmlns util:"http://www.springframework.org/schema/util"
    
    // We can write code in this configuration file and reference
    // binding variables.
    def bindingEnv = env
    util.list(id: 'runningMode') {
        value new Date().format('yyyy-MM-dd')
        value bindingEnv
    }
    
    // Abstract bean definition with custom destroy-method.
    abstractDataSource(BasicDataSource) { bean ->
        bean.destroyMethod = 'close'
        driverClassName = 'com.mysql.jdbc.Driver'
        username = ''
        password = ''
    }
    
    // Create bean definitions dynamically and use abstract parent.
    def urls = [
        dev:  'jdbc:mysql://localhost/db',
        test: 'jdbc:mysql://localhost/test',
        prod: 'jdbc:mysql://prod'
    ]
    urls.each { key, value ->
        "${key}DataSource"(BasicDataSource) { bean -> 
            bean.parent = abtractDataSource
            url = value
        }
    }
    
    // Spring bean definitions for bean with id personDao
    // and class PersonDaoImpl. Sample switch statement to show
    // we can add Groovy code in this file.
    personDao(PersonDaoImpl) {
        switch (env) {
            case 'dev':  dataSource = devDataSource; break
            case 'test': dataSource = testDataSource; break
            case 'prod': dataSource = ref(prodDataSource); break
        }
    }
    
    // Equivalent of
    // <bean id="personService"
    //    class="com.mrhaki.blog.PersonServiceJavaImpl">
    //    <property name="dao" ref="personDao"/>
    // </bean>
    personService(PersonServiceJavaImpl) {
        dao = ref(personDao)
    }
}

We have now defined the following beans: runningMode, devDataSource, testDataSource, prodDataSource, abstractDataSource, personService, personDao and wired some together. With the following Java application we read this file and use the Spring ApplicationContext:

package com.mrhaki.blog;

import java.io.IOException;
import java.util.List;
import java.util.Date;
import java.text.SimpleDateFormat;
import grails.spring.BeanBuilder;
import groovy.lang.Binding;
import org.springframework.context.ApplicationContext;
import org.apache.common.dbcp.BasicDataSource;

public class SpringBuilderApp {
    public static void main(String[] args) throws IOException {
        // Create a binding to get "env" into the configuration script.
        final Binding binding = new Binding();
        binding.setProperty("env", "dev");
        
        // Create a builder and find all files on the classpath
        // ending with BeansConfig.groovy.
        final BeanBuilder builder = new BeanBuilder();
        builder.setBinding(binding);
        builder.loadBeans("classpath:**/**BeansConfig.groovy");
        
        // Make the Spring ApplicationContext.
        final ApplicationContext context = builder.createApplicationContext();
        
        assert 7 == context.getBeanDefinitionNames().length;
        assert context.containsBean("prodDataSource");
        assert context.containsBean("testDataSource");
        assert context.containsBean("devDataSource");
        assert context.containsBean("abstractDataSource");
        assert context.containsBean("personDao");
        assert context.containsBean("personService");
        assert context.containsBean("runningMode");

        final List mode = (List) context.getBean("runningMode");
        final Date now = new Date();
        assert new SimpleDateFormat("yyyy-MM-dd").format(now).equals(mode.get(0));
        assert "dev".equals(mode.get(1));
        
        final BasicDataSource ds = (BasicDataSource) context.getBean("testDataSource");
        assert "jdbc:mysql://localhost/test".equals(ds.getUrl());
        assert "com.mysql.jdbc.Driver".equals(ds.getDriverClassName());
        assert "".equals(ds.getUsername());
        assert "".equals(ds.getPassword());
    }
}

We can run this application with assertions enabled and we get no errors. We must include the grails-spring-<version>.jar and grails-bootstrap-<version>.jar, the Commons DBCP libraries and the Groovy libraries to the classpath. Now we can define the Spring context with Groovy and we can run Groovy code from Spring. More and more Groovy creepes into our Java projects.

Copy Resources in NetBeans Java Project

When we need to copy resources in our Java project to the build directory, we have to extend the project's build.xml file. Suppose we have a default Java project in NetBeans. We create a new directory resources in the project directory. Here we place properties files, images or anything else we think is a resource and doesn't need to be compiled. We open build.xml and add the following ANT targets:

<target name="-post-compile-single" depends="-post-compile"/>

<target name="-post-compile">
    <copy todir="${build.classes.dir}">
        <fileset dir="resources/">
            <include name="**/**"/>
        </fileset>
    </copy>
</target>

Anytime we compile a single source file, or the complete project, the files from the resources directory are copied to the build/classes directory.

Groovy Goodness: Working with Files

When we write code in Java to work with files we must write a lot of boilerplate code to make sure all streams are opened and closed correctly and provide exception handling. The Commons IO package already helps, but Groovy makes working with files so easy. Groovy adds a lot of useful methods to the java.io.File class. We can use simple properties to write and read text, methods to traverse the file system and methods to filter contents.

Here is a Groovy script with different samples of working with files:

// Normal way of creating file objects.
def file1 = new File('groovy1.txt')
def file2 = new File('groovy2.txt')
def file3 = new File('groovy3.txt')

// Writing to the files with the write method:
file1.write 'Working with files the Groovy way is easy.\n'

// Using the leftShift operator:
file1 << 'See how easy it is to add text to a file.\n'

// Using the text property:
file2.text = '''We can even use the text property of
a file to set a complete block of text at once.'''

// Or a writer object:
file3.withWriter('UTF-8') { writer ->
    writer.write('We can also use writers to add contents.')
}

// Reading contents of files to an array:
def lines = file1.readLines()
assert 2 == lines.size()
assert 'Working with files the Groovy way is easy.' == lines[0]

// Or we read with the text property:
assert 'We can also use writers to add contents.' == file3.text

// Or with a reader:
count = 0
file2.withReader { reader -> 
    while (line = reader.readLine()) {
        switch (count) {
            case 0: 
                assert 'We can even use the text property of' == line
                break
            case 1:
                assert 'a file to set a complete block of text at once.' == line
                break
        }
        count++
    }
}

// We can also read contents with a filter:
sw = new StringWriter()
file1.filterLine(sw) { it =~ /Groovy/ }
assert 'Working with files the Groovy way is easy.\r\n' == sw.toString()

// We can look for files in the directory with different methods.
// See for a complete list the File GDK documentation.
files = []
new File('.').eachFileMatch(~/^groovy.*\.txt$/) { files << it.name }
assert ['groovy1.txt', 'groovy2.txt', groovy3.txt'] == files

// Delete all files:
files.each { new File(it).delete() }

Grassroots Groovy: the Templating Route

We can introduce Groovy into our projects at grassroots level. If we need some templating functionality in our Java application we can use Groovy templates. Groovy templates allow us to write Groovy code, use JSP expressions or use variables in the template. Basically a template contains static text with placeholders and we combine the template with 'live' objects through binding. The result is a the template with all placeholders replaced with the correct values from the binding and expressions.

Let's start with a template named mail.gtpl:

Hello $person.firstname $person.lastname,

Thank you registering with our company at ${ new Date().format('MMM dd, yyyy') }.

Details:
--------
Your username is <%= person.username %>.
You have signed up for the following <% out << (person.topics.size() == 1) ? ' topic' : person.topics.size() + ' topics' %>:
<% person.topics.each { %>
- $it
<% } %>


${greeting}

Because this is an example we use all possible ways to express values and Groovy code. We use the <% %>, <%= %>, $var and ${expression} syntax. At line 7 we use the out variable, which is bound to a Writer object. Now we write the Java code that can use and execute this Groovy template:

package com.mrhaki.blog;

import java.io.*;
import java.util.*;
import groovy.text.*;
import groovy.lang.*;

public class TemplateApp {
    public static void main(String[] args) throws IOException, ClassNotFoundException {
        // Initialize template.
        final TemplateEngine engine = new GStringTemplateEngine();
        final Template template = engine.createTemplate(new File("mail.gtpl"));
        
        // Create Map for binding objects to template.
        final Map<String, Object> binding = new HashMap<String, Object>();
        binding.add("greeting", "Kind regards,\n\nThe Company");
        binding.add("person", new PersonObject());
        
        // Bind Map to template.
        final Writeable out = template.make(binding);
        
        // We use a Writer object, can be any Writer e.g. FileWriter.
        final StringWriter writer = new StringWriter();
        out.writeTo(writer);
        System.out.println(writer.toString());
    }
}

class PersonObject {
    public String getFirstname() { 
        return "Hubert"; 
    }
    
    public String getLastname() { 
        return "Klein Ikkink"; 
    }
    
    public String getUsername() { 
        return "mrhaki"; 
    }
    
    public List<String> getTopics() { 
        return Arrays.asList(new String[] { "Groovy", "Grails", "Java" }); 
    }
}

When we run the Java application (make sure the Groovy libraries are in the Java classpath) we get the following output:

Hello Hubert Klein Ikkink,

Thank you registering with our company at Aug 26, 2009.

Details:
--------
Your username is mrhaki.
You have signed up for the following 3 topics:
- Groovy
- Grails
- Java


Kind regards,

The Company

In this post we learned another way to introduce Groovy in Java projects: templates. Groovy templates are flexible and allow the use of Groovy code in the template. Integration with Java is simple and easy.

Grassroots Groovy: the Spring Route (part 1)

We can introduce Groovy into our projects at grassroots level. If we are already using Spring on our project we can use Groovy to define classes and define them in our Spring configuration files. It is even possible to have Groovy code in the configuration files themselves. The nice thing about the Groovy support in Spring is that we can define a reload interval on Groovy source files. This means new versions of the Groovy source file are automatically picked up and ready to be used.

Let's see how we can leverage the Groovy support with Spring by a little example. We have a simple Data Access Object (DAO) class and interface to get a list of Person objects. The DAO class is used by a service to get any person without an e-mail address. We are going to write the service in Groovy. All these classes are connected by a Spring configuration file. We start with a simple Person class to hold a username and e-mail address:

package com.mrhaki.blog;

public class Person {

    private String username;
    private String email;

    public Person() {
    }

    public Person(String username, String email) {
        this.username = username;
        this.email = email;
    }

    public String getEmail() {
        return email;
    }

    public void setEmail(String email) {
        this.email = email;
    }

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    @Override
    public String toString() {
        return "[Person: username=" + username + ", email=" + email + "]";
    }
}

Next we define a PersonDao interface and a Java implementation class PersonDaoImpl:

package com.mrhaki.blog;

import java.util.List;

public interface PersonDao {

    List<Person> list();
}

package com.mrhaki.blog;

import java.util.ArrayList;
import java.util.List;

public class PersonDaoImpl implements PersonDao {

    public List<Person> list() {
        final List<Person> result = new ArrayList<Person>();
        result.add(new Person("mrhaki", "test@email.com"));
        result.add(new Person("test", null));
        return result;
    }
}

We have our DAO part ready. It is time to create the interface PersonService:

package com.mrhaki.blog;

import java.util.List;

public interface PersonService {
    List<Person> listPersonsNoEmail();
}

Notice we only have created plain old Java classes, no Groovy to be seen. But that changes when we create the Spring configuration file. To use Groovy support we must have the Groovy libraries in our classpath, and we must use the lang namespace in the configuration file. First we write the Groovy code in our Spring configuration file. Later on we move the code from the configuration file to a separate source file. We create the file spring-config.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:lang="http://www.springframework.org/schema/lang"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang-2.5.xsd">

    <bean id="personDao" class="com.mrhaki.blog.PersonDaoImpl"/>

    <lang:groovy id="personService">
        <lang:inline-script>
            <![CDATA[
import com.mrhaki.blog.*

class PersonServiceImpl implements PersonService {

    def personDao

    List<Person> listPersonsNoEmail() {
        personDao.list().grep{ !it.email }
    }
}           
            ]]>
        </lang:inline-script>
       <lang:property name="dao" ref="personDao"/>
    </lang:groovy>

</beans>

At lines 9-25 we define the Groovy script which implements the PersonService interface. At line 9 we assign a value to the attribute id, so we can get the resulting object. The Groovy script has one property personDao and at line 24 we assign our Java DAO object to this property. The implementation method is simple and Groovy.

We are ready to use our Groovy service in a Java application:

package com.mrhaki.blog;

import org.springframework.context.support.ClassPathXmlApplicationContext;
import org.springframework.context.support.ApplicationContext;

public class SpringApp {

    public static void main(String[] args) {
        final ApplicationContext context =
                new ClassPathXmlApplicationContext("classpath:com/mrhaki/blog/spring-config.xml");
        final PersonService service = (PersonService) context.getBean("personService");

        assert 1 == service.listPersonsNoEmail().size();
        assert "test".equals(service.listPersonsNoEmail().get(0).getUsername());
        assert null == service.listPersonsNoEmail().get(0).getEmail();
    }

}

We can run the application SpringApp and if we enable assertions we don't get any errors. Our Groovy code is invoked correctly and in the Java code we cannot see the service is implemented in Groovy. We are going to make an adjustment to the Spring configuration file. We move the script in the file to a separate source file PersonServiceImpl.groovy. And we use the attribute refresh-check-delay to define a reload interval for the source file. Spring will check if the file has changed and if so will load the new script.

package com.mrhaki.blog

class PersonServiceImpl implements PersonService {

    def personDao

    List<Person> listPersonsNoEmail() {
        personDao.list().grep{ !it.email }
    }
}

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:lang="http://www.springframework.org/schema/lang"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang-2.5.xsd">

    <bean id="personDao" class="com.mrhaki.blog.PersonDaoImpl"/>

    <lang:groovy id="personService" refresh-check-delay="3000"
        script-source="classpath:com/mrhaki/blog/PersonServiceImpl.groovy">
       <lang:property name="dao" ref="personDao"/>
    </lang:groovy>

</beans>

In this post we have seen how easy it is to integrate Groovy code in our Java based project using Spring. In a next post we see how we can create the Spring configuration files in a Groovy way.

Groovy Goodness: the Spaceship Operator

Groovy adds some nice operators to the language. One of them is the spaceship operator. It's called the spaceship operator, because we use the following syntax <=> and that looks a bit like a UFO. The operator is another way of referring to the compareTo method of the Comparable interface. This means we can implement the compareTo method in our own classes and this will allow us to use the <=> operator in our code. And of course all classes which already have implemented the compareTo method can be used with the spaceship operator. The operator makes for good readable sort methods.

class Person implements Comparable {
    String username
    String email

    int compareTo(other) {
        this.username <=> other.username
    }
}

assert -1 == ('a' <=> 'b')
assert 0 == (42 <=> 42)
assert -1 == (new Person([username:'mrhaki', email: 'test@email.com']) <=> new Person([username:'zavaria', email:'tester@email.com']))
assert [1, 2, 3, 4] == [4, 2, 1, 3].sort{ a, b -> a <=> b }

Run this script in GroovyConsole.

Grassroots Groovy: Configuration with ConfigSlurper

We can introduce Groovy into our projects at grassroots level. In this post we learn how to use Groovy configuration scripts instead of Java properties files to configure our application. Groovy has the ConfigSlurper class. We use a ConfigSlurper object to read in a Groovy script. This scripts contains a tree of objects with different values. These values can be Java types, which is not possible in a plain properties file. We can even add code to the script file.

The ConfigSlurper is capable of supporting different environments. We pass the environment to the constructor. This way we can set a default value for a property and override it for a specific environment in the same configuration file.

We start with a simple Java application to read in a Groovy configuration script:

package com.mrhaki.sample.config;

import java.net.*;
import java.util.*;
import groovy.util.*;

public class JavaConfig {
    public static void main(String[] args) throws MalformedURLException {
        // Read in 'config.groovy' for the development environment.
        ConfigObject conf = new ConfigSlurper("development").parse(new File("config.groovy").toURI().toURL());
        
        // Read in 'mail.groovy'.
        final ConfigObject mailConf = new ConfigSlurper().parse(new File("mail.groovy").toURI().toURL());
        
        // Merge both configurations into one.
        conf = (ConfigObject) conf.merge(mailConf);
        
        // Flatten configuration for easy access keys with dotted notation.
        final Map config = conf.flatten();
        
        assert "1.0".equals(config.get("app.version"));
        assert new Integer(20).equals(config.get("app.setting2"));
        assert new Date().after((Date) config.get("app.date"));
        assert config.get("app.active");
        assert "http://localhost".equals(config.get("server.URL"));
        assert "smtpservername".equals(config.get("mail.smtp.host"));
    }
}

It is not difficult to invoke the ConfigSlurper from our Java code. We must make sure the Groovy libraries are in the classpath when we run the application ($GROOVY_HOME/embeddable/groovy-all-<version>.jar). Next we make the configuration script config.groovy:

// We can group settings.
// The following settings are created:
// app.version, app.setting1, app.setting2, app.setting3, app.date, app.active
app {
    version = "1.0"
    
    // We can write code to set up settings.
    [1, 2, 3].each {
        this."setting${it}" = it * 10
    }
    
    // We can use Java objects
    date = new Date()
    active = true
}

server.URL = 'http://default'

// Environment specific settings override settings with the
// same name.
environments {
    development {
        server.URL = 'http://localhost'
    }
    test {
        server.URL = 'http://test:9080'
    }
    integrationtest {
        server.URL = 'http://integrationtest/url'
    }
    production {
        server.URL = 'http://prod/url'
    }
}

And we make the configuration script mail.groovy which is merged together with config.groovy:

// Classic way of specifying properties without grouping, 
// but we can use Java objects instead of only strings.
mail.smtp.host = "smtpservername"
mail.smtp.ssl = true

We are ready to run our Java application - $ java -ea com.mrhaki.sample.config.JavaConfig - and all works fine. We shouldn't get any assertion errors. In a real world application we can make a class to read in the Groovy configuration files. Other classes use this class to get the values of the settings. All in plain old Java, but we create the settings and their values in a Groovy way.

Groovy Goodness: the Grep Method

The grep method is used to filter elements in a collection. The argument of the grep method is a filter Object. This is related to Groovy's switch statement, because the same isCase() method is used to evaluate the filter.

assert [true] == ['test', 12, 20, true].grep(Boolean), 'Class isInstance'
assert ['Groovy'] == ['test', 'Groovy', 'Java'].grep(~/^G.*/), 'Pattern match'
assert ['b', 'c'] == ['a', 'b', 'c', 'd'].grep(['b', 'c']), 'List contains'
assert [15, 16, 12] == [1, 15, 16, 30, 12].grep(12..18), 'Range contains'
assert [42.031] == [12.300, 109.20, 42.031, 42.032].grep(42.031), 'Object equals'
assert [100, 200] == [10, 20, 30, 50, 100, 200].grep({ it > 50 }), 'Closure boolean'

Groovy Goodness: the Switch Statement

The Java switch statement looks pale compared to Groovy's switch statement. In Groovy we can use different classifiers for a switch statement instead of only an int or int-derived type. Anything that implements the isCase() method can be used as a classifier. Groovy already added an isCase() method to Class (uses isInstance), Object (uses (equals), collections (uses contains) and regular expressions (uses matches). If we implement the isCase method in our own Groovy classes we can use it as a classifier as well. Finally we can a closure as a classifier. The closure will be evaluated to a boolean value.

def testSwitch(val) {
    def result
    switch (val) {
        case ~/^Switch.*Groovy$/:
            result = 'Pattern match'
            break
        case BigInteger:
            result = 'Class isInstance'
            break
        case 60..90:
            result = 'Range contains'
            break
        case [21, 'test', 9.12]:
            result = 'List contains'
            break
        case 42.056:
            result = 'Object equals'
            break
        case { it instanceof Integer && it < 50 }:
            result = 'Closure boolean'
            break
        default:
            result = 'Default'
            break
    }    
    result
}

assert 'Pattern match' == testSwitch("Switch to Groovy")
assert 'Class isInstance' == testSwitch(42G)
assert 'Range contains' == testSwitch(70)
assert 'List contains' == testSwitch('test')
assert 'Object equals' == testSwitch(42.056)
assert 'Closure boolean' == testSwitch(20)
assert 'Default' == testSwitch('default')

Grassroots Groovy: the Maven Route (part 2)

We can introduce Groovy into our projects at grassroots level. In a a previous blog we learned how we could add Groovy code to a Maven based Java project. In this blog post we see how we can use the GMaven plugin to add Groovy code to the Maven build process itself. Sometimes we want a bit more functionality than Maven supports by default. We could write our own Maven plugin to get this extra functionality, but sometimes we can get away with a little scripting. And for this scripting we use Groovy of course.

To execute Groovy code from Maven we only have to define the GMaven plugin in our POM. And then we write Groovy code in the POM itself to be executed, or from a local file, of from an URL. The Groovy code can access a couple of variables:

• project: points to the Maven project and we can access everything.
• pom: just another name for the project variable.
• session: the current Maven session.
• settings: the user's global settings.
• log: the SLF4J logger instance to log messages to the console.
• ant: a reference to an AntBuilder instance to execute Ant tasks.
• fail(): a helper method to throw a MojoExecutionException.

Let's add the Maven plugin to our POM with a reference to a local Groovy script file:

<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.mrhaki.sample</groupId>
    <artifactId>sample-app</artifactId>
    <packaging>jar</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>sample-app</name>
    <url>http://maven.apache.org</url>
    <build>
        <plugins>
            <!-- GMaven plugin so we can run Groovy scripts. -->
            <plugin>
                <groupId>org.codehaus.groovy.maven</groupId>
                <artifactId>gmaven-plugin</artifactId>
                <version>1.0</version>
                <executions>
                    <execution>
                        <!-- Script will be executed in 
                                the generate-resources phase. -->
                        <phase>generate-resources</phase>
                        <goals>
                            <goal>execute</goal>
                        </goals>
                        <configuration>
                            <source>${pom.basedir}/src/main/scripts/MavenHelper.groovy</source>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
    <profiles>
        <!-- Extra profile to be used in Groovy script as an example. -->
        <profile>
            <id>dev</id>
        </profile>
    </profiles>
</project>

Next we create the Groovy script file in src/main/scripts. In this script we look for active profiles, and if we have a match for a profile with id dev we log some extra info and use the Antbuilder to copy a properties file:

// Get list of active profiles.
def profiles = project.getActiveProfiles()

profiles.each{ profile ->
    // Show off Groovy's switch for matching on strings.
    switch(profile.id) {
        case 'dev':
            // Show some extra logging info.
            log.info "*" * 70
            log.info "* You are running in DEVELOPMENT mode."
            log.info "* Version: ${pom.version}, Finalname: ${pom.build.finalName}"
            log.info "*" * 70
            
            // Using Antbuilder to copy properties file.
            ant.copy(file: 'src/main/resources/dev.properties', tofile: project.build.outputDirectory + '/app.properties')
            break;
        default:
            break;
    }
}

To execute Maven with the dev profile we run the following command: $ mvn install -Pdev. In the logging generated by Maven we get the following output:

[INFO] [groovy:execute {execution: default}]
[INFO]  **********************************************************************
[INFO]  * You are running in DEVELOPMENT mode.
[INFO]  * Version: 1.0-SNAPSHOT, Finalname: sample-app-1.0-SNAPSHOT
[INFO]  **********************************************************************
Copying 1 file to P:\samples\sample-app\target\classes

So we learned how to add Groovy scripts to our Maven build. The Groovy website contains more information about all the configuration we can do to use the GMaven plugin like this.

Grassroots Groovy: the Maven Route (part 1)

We can introduce Groovy into our projects at grassroots level. In a series of blog posts we learn different routes we can follow to get our Java projects more Groovy. First off we look at how we can add Groovy sources to our Maven based Java project. But we will also see how XML handling, unit testing, Spring integration and other routes allow us to add Groovyness to a Java project in other posts.

The GMaven plugin adds Groovy support to a Maven based Java project. We configure the plugin to allow Groovy classes to be compiled just like normal Java classes. The compiled Groovy classes can than be integrated with the Java classes.

Let's start with the Maven quickstart archetype to create a simple Java Maven project:

$ mvn archetype:generate -DgroupId=com.mrhaki.sample -DartifactId=sample-app -DinteractiveMode=false
$ cd sample-app

Maven creates a new directory sample-app with a basic Java application at src/main/java/com/mrhaki/sample/App.java. We open pom.xml to add the GMaven plugin so we can compile and use our Groovy code:

<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.mrhaki.sample</groupId>
    <artifactId>sample-app</artifactId>
    <packaging>jar</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>sample-app</name>
    <url>http://maven.apache.org</url>
    <build>
        <plugins>
            <!-- GMaven plugin so we can compile Groovy sources. -->
            <plugin>
                <groupId>org.codehaus.groovy.maven</groupId>
                <artifactId>gmaven-plugin</artifactId>
                <version>1.0</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>generateStubs</goal>
                            <goal>compile</goal>
                            <goal>generateTestStubs</goal>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <!-- Extra dependency for Groovy libraries for runtime. -->
        <dependency>
            <groupId>org.codehaus.groovy.maven.runtime</groupId>
            <artifactId>gmaven-runtime-1.6</artifactId>
            <version>1.0</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

Now our Groovy sources ending with .groovy in the directory src/main/groovy are compiled into target/classes and Groovy sources in the directory src/test/groovy are compiled into the target/test-classes directory when we run mvn install. We create a Groovy source file at src/main/groovy/com/mrhaki/sample/MyGroovy.groovy:

package com.mrhaki.sample

class MyGroovy {

    String username
    
    def reverseName() {
        username.reverse()
    }
    
    def calculateBigDecimals() {
        40G + 2G
    }
}

For this example we add a new test method to the existing AppTest class in the directory src/test/java/com/mrhaki/sample. In this test method we create an instance of the Groovy class and invoke the methods to test them:

...
public void testGroovy() {
    MyGroovy groovy = new MyGroovy();
    groovy.setUsername("mrhaki");
    assertEquals("mrhaki", groovy.getUsername());
    assertEquals("ikahrm", groovy.reverseName());
    assertEquals(new java.math.BigInteger("42"), groovy.calculateBigDecimals());
}
...

Now we run $ mvn test and our Groovy class is compiled and the Java test class will be executed without errors. So we see integrating Groovy with Java is not that difficult when we already have a Maven based Java project. In a following post we learn how we can use Groovy in the Maven POM as another route to get Groovy into our Java project.

Groovy Goodness: Tell the Groovy Truth

Groovy has a broader sence of what is true or false than the boolean expressions. For example an empty list or string is false in Groovy. This makes that we can write less code in conditional expressions to evaluate true or false:

// Booleans:
def b1 = false
def b2 = true
assert !b1
assert b2

// Strings:
def s1 = ''
def s2 = 'abc'
assert !s1
assert s2

// Numbers:
def n1 = 0
def n2 = 42
assert !n1
assert n2

// Lists, iterators:
def l1 = []
def l2 = [1, 3, 6]
assert !l1
assert l2
def i1 = l1.iterator()
def i2 = l2.iterator()
assert !i1
assert i2

// Maps:
def m1 = [:]
def m2 = ['key': 'value']
assert !m1
assert m2

// Objects:
def o1
def o2 = new Expando(name: 'groovy')
assert !o1
assert o2

// Matchers:
def ma1 = "groovy" =~ /java/
def ma2 = "groovy" =~ /groovy/
assert !ma1
assert ma2

Groovy Goodness: 'String', "Strings", /Strings/ and GStrings

Compared to Java we have a lot of ways to represent strings. We can use single quotes, double quotes, slashes and dollar slashes. So the following statements all are string literals:

// Single quotes:
def s1 = 'Yep this is a string, and we can use "double quotes" without escaping them.'
def m1 = '''With three quotes
we can write multi-line strings, without all
the hassle of concatenation.'''

// Double quotes:
def s2 = "Also a string, and of course we can use 'single quotes' without escaping them."
def m2 = """\
With three double quotes we can have a multi-line string,
and we don't need to add extra + signs. Very easy for 
copy-paste."""

// Slashy (also multi-line possible):
def s3 = /Well almost everything goes in a "slashy" 'string' without escaping. Good for readable regular expressions!/

// Dollar slashy:
def s4 = $/
Like a 'slashy' string, but with different escaping rules
for slashes ("/")
/$

If we use a double quoted string and it contains an Groovy expression with the ${expression} syntax we have a GString. This is a different kind of class. So a GString is not the same as a String. GStrings are very handy if we want some templating done without using a complete template engine:

// Simple Groovy expressions can be used:
def user = new Expando(name: 'mrhaki', email: 'mail@email.com')
def gs1 = "Hi, your name is ${user?.name}. If I shout I will call you ${user?.name?.toUpperCase()}!" 
println gs1 // Output: Hi, your name is mrhaki. If I shout I will call you MRHAKI!

// Works also in triple double quoted multi-line strings:
def mailMessage = """
Hi ${user?.name},

thank you for signing up.
You signed up wit the following e-mail address:

${user?.email}

Kind regards,

the support team.
"""
println mailMessage 

// We can use closures in GStrings to do lazy evaluation. 
// The closure is evaluated when the toString() method on the
// GString is invoked instead of when the GString is created.
// Closures can have zero or one arguments. With one argument we get a writer
// object we can write to, with zero arguments the toString() is called on the
// closure.
def directEval = "Current name value is ${user.name} and email is ${user.email}."
def lazyEval = "Current name value is ${ -> user.name } and email is ${ out -> out << user.email }."
user.name = 'changed username'
user.email = 'changed email'
println directEval // Output: Current name value is mrhaki and email is mail@email.com.
println lazyEval // Output: Current name value is changed username and email is changed email.

Groovy Goodness: Turn Methods into Closures

Groovy supports closures and they are very useful when we create Groovy applications. For example we can pass closures as arguments to methods to execute them. We can create closures ourselves, but we can also convert a method to a closure with the .& operator. And we can use the converted method just like a normal closure. Because Groovy can use Java objects we can also convert a Java method into a closure.

Let's start with a simple Java class:

public class JavaObject {
 public static void javaSays(final String s) {
  System.out.println("Java says: Hello " + s + "!");
 }
}

With the following script we use this Java class and convert the javaSays method to a closure:

// Simple list with names.
def names = ['groovy', 'grails', 'mrhaki']

// Simple closure.
names.each { println 'Normal closure says: Hello ' + it + '!' }

// Groovy method to convert to closure.
def groovySays(s) {
 "Groovy says: Hello ${s}!"
}
// Use .& syntax to convert method to closure.
names.each(this.&groovySays)

// Convert Java method to closure and use it.
def javaSays = JavaObject.&javaSays
names.each javaSays

If we run this script we get the following output:

Normal closure says: Hello groovy!
Normal closure says: Hello grails!
Normal closure says: Hello mrhaki!
Groovy says: Hello groovy!
Groovy says: Hello grails!
Groovy says: Hello mrhaki!
Java says: Hello groovy!
Java says: Hello grails!
Java says: Hello mrhaki!

Groovy Goodness: Bound and Constrained Properties in GroovyBeans

In a previous post we learned some basic GroovyBeans syntax. With less code we could write a complete JavaBean. In this post we see how we can implement bound and constrained properties as defined in the JavaBeans specification. A bound property is a bean property for which a change to the property results in a notification being sent to some other bean. A constrained property is a bean property for which a change to the property results in validation by another bean. The other bean may reject the change if it is not appropriate.

Implementing these properties is of course easy in Groovy! Groovy supports the @Bindable and @Vetoable annotations (extra info on Groovy site) to implement bound and constrained properties. The following code shows a simple bean:

import groovy.beans.*

class Car {
   int numberOfDoors
   @Vetoable String model
   @Vetoable String brand
   boolean automatic
   @Bindable double price
 
   String toString() {
     "[Car details => brand: '${brand}', model: '${model}', #doors: '${numberOfDoors}', automatic: '${automatic}', price: '${price}']"
   }
}

And that is it! When we compile the Car class Groovy adds all necessary addXXXListener methods with implementations. Just for excercise we see the equivalent in Java:

import java.beans.*;

public class Car {
 private int numberOfDoors;
 private String model;
 private String brand;
 private boolean automatic;
 private double price;
 
 private final PropertyChangeSupport pcs = new PropertyChangeSupport(this);
 private final VetoableChangeSupport vcs = new VetoableChangeSupport(this);

 public void addPropertyChangeListener(PropertyChangeListener listener) {
  pcs.addPropertyChangeListener(listener);
 }

 public void removePropertyChangeListener(PropertyChangeListener listener) {
  pcs.removePropertyChangeListener(listener);
 }
 
 public void addVetoableChangeListener(VetoableChangeListener listener) {
  vcs.addVetoableChangeListener(listener);
 }

 public void removeVetoableChangeListener(VetoableChangeListener listener) {
  vcs.removeVetoableChangeListener(listener);
 }

 public void setPrice(double price) {
  double oldPrice = this.price;
  this.price = price;
  pcs.firePropertyChange("price", oldPrice, price);
    }
 
 public double getPrice() {
  return this.price;
 }
 
 public void setModel(String model) throws PropertyVetoException {
  String oldModel = this.model;
  vcs.fireVetoableChange("model", oldModel, model);
  this.model = model;
  pcs.firePropertyChange("model", oldModel, model);
 }
 
 public String getModel() {
  return this.model;
 }
 
 public void setBrand(String model) throws PropertyVetoException {
  String oldBrand = this.brand;
  vcs.fireVetoableChange("model", oldBrand, brand);
  this.brand = brand;
  pcs.firePropertyChange("model", oldBrand, brand);
 }
 
 public String getBrand() {
  return this.brand;
 }
 
 public void setNumberOfDoors(int numberOfDoors) {
  this.numberOfDoors = numberOfDoors;
 }
 
 public int getNumberOfDoors() {
  return numberOfDoors;
 }
 
 public void setAutomatic(boolean automatic) {
  this.automatic = automatic;
 }
 
 public boolean isAutomatic() {
  return this.automatic;
 }
 
 public String toString() {
  final StringBuilder builder = new StringBuilder();
  builder.append("[Car details => brand: '");
  builder.append(brand);
  builder.append("', model: '");
  builder.append(model);
  builder.append("', #doors: '");
  builder.append(numberOfDoors);
  builder.append("', automatic: '");
  builder.append(automatic);
  builder.append("', price: '");
  builder.append(price);
  builder.append("']");
  return builder.toString();
 }
}

Well I think we see the difference in lines of code...

Let's write a simple Groovy script to listen for the propertychange and vetoablechange events. We can also write less code than the Java equivalent here. In Groovy we can use closures to implement listener interfaces. The following Groovy script listens for property changes and rejects certain values:

import groovy.beans.*
import java.beans.*

def toyota = new Car(brand: 'Toyota', model: 'Verso', price: 28919, numberOfDoors: 5)
toyota.propertyChange = {
 if (it.propertyName == 'price') {
  println "The price has changed. Inform sales the new price is '${it.newValue}'." 
 }
}
toyota.vetoableChange = { PropertyChangeEvent pce ->
 if (pce.propertyName == "brand") {
  if (!(pce.newValue in ['Toyota', 'Lexus'])) {
   throw new PropertyVetoException('New value is not Toyota or Lexus', pce)
  }
 }
 if (pce.propertyName == "model") {
  if (pce.newValue ==~ /.*\d+.*/) {
   throw new PropertyVetoException('No numbers in model names allowed.', pce)
  }
 }
}

toyota.price = 30995
assert 30995 == toyota.price

toyota.brand = 'Lexus'
assert 'Lexus' == toyota.brand

try {
 toyota.brand = 'AUDI'
 assert false: 'We should not be able to set this value.'
} catch (PropertyVetoException e) {
 assert true
}

try {
 toyota.model = 'A5'
 assert false: 'We should not be able to set this value.'
} catch (PropertyVetoException e) {
 assert true
}

Our Groovy Car class is compiled to Java byte code. So we must be able to use this class in our normal Java applications. So here is a Java application which implements the appropriate listener interface. Notice we must compile this Java application with groovyc and we cannot use anonymous inner classes as interface implementations.

import java.beans.*;
import java.util.regex.*;

public class CarApp implements PropertyChangeListener, VetoableChangeListener {
    public static void main(String[] args) {
        Car toyota = new Car();
        toyota.setModel("Verso");
        toyota.setBrand("Toyota");
        toyota.setNumberOfDoors(5);
        toyota.setPrice(28919);

        CarApp app = new CarApp();
        toyota.addPropertyChangeListener(app);
        toyota.addVetoableChangeListener(app);

        toyota.setPrice(30995);
        toyota.setBrand("Lexus");
        try {
            toyota.setBrand("AUDI");
        } catch (PropertyVetoException e) {
            System.out.println("Brand is not changed.");
        }
        try {
            toyota.setModel("A5");
        } catch (PropertyVetoException e) {
            System.out.println("Model is not changed.");
        }
    }

    public void propertyChange(PropertyChangeEvent evt) {
       if (evt.getPropertyName().equals("price")) {
           System.out.println("The price has changed. Inform sales the new price is '" + evt.getNewValue() + "'.");
       }
    }

    public void vetoableChange(PropertyChangeEvent evt) {
       if (evt.getPropertyName().equals("brand")) {
           if (!isValidBrand(evt.getNewValue())) {
               throw new PropertyVetoException("New value is not Toyota or Lexus", evt)
           }
       }
       if (evt.getPropertyName().equals("model")) {
           if (!isValidModel(evt.getNewValue())) {
               throw new PropertyVetoException("No numbers in model names allowed.", evt)       
           }
       }
    }

    private boolean isValidBrand(String newValue) {
        final String[] names = new String[2];
        names[0] = "Toyota";
        names[1] = "Lexus";
        for (String name: names) {
            if (newValue.equals(name)) {
                return true;
            }
        }
        return false;
    }
 
    private boolean isValidModel(String model) {
        return !Pattern.matches(".*\\d+.*", model);
    }
}

The @Bindable and @Vetoable really make for readable Groovy code and still implement all the functionality we expect.

Groovy Goodness: GroovyBeans the Simpler JavaBeans

We use JavaBeans a lot in our day to day Java programming. GroovyBeans use a much simpler syntax to get the same result. If we define a property in a JavaBean we must provide the corresponding get/set methods. Most IDEs can generate these methods for us, but still the resulting code of a JavaBeans is less readable, because we have a lot of these methods.

Let's see some code first before we continue to look into GroovyBeans in more detail:

class Car {
 int numberOfDoors
 String model
 String brand
 boolean automatic
 double price
 
 String toString() {
     "[Car details => brand: '${brand}', model: '${model}', #doors: '${numberOfDoors}', automatic: '${automatic}', price: '${price}']"
 }
}

We can use the Car GroovyBean from a Groovy class or script:

Car ford = new Car(brand: 'Ford', model: 'Focus', numberOfDoors: 4, automatic: false, price: 24995)
Car toyota =  new Car(brand: 'Toyota', model: 'Verso')
toyota.automatic = true  // Yes, this invokes setAutomatic.
toyota.setPrice(28919)  // Normal set invocation.
toyota.setNumberOfDoors 5  // In Groovy we can omit the parentheses.
println ford  // Output: [Car details => brand: 'Ford', model: 'Focus', #doors: '4', automatic: 'false', price: '24995.0']
println toyota // Output: [Car details => brand: 'Toyota', model: 'Verso', #doors: '5', automatic: 'true', price: '28919.0']

The class Car is our GroovyBean. It looks like we defined only some fields, but because we haven't use any of the access modifiers (public, protected or private) these fields are actually properties when the code is compiled. Groovy will generate the get/set methods so we can use the properties (see a complete list of rules the compiler uses to generate properties). To prove this is true we write a simple Java application to use the Car class:

public class CarApp {
 public static void main(String[] args) {
     Car car = new Car();
  car.setNumberOfDoors(3);
  car.setModel("A3");
  car.setBrand("AUDI");
  car.setPrice(32010);
  System.out.println(car); // Output: [Car details => brand: 'AUDI', model: 'A3', #doors: '3', automatic: 'false', price: '32010.0']
 }
}

GroovyBeans are flexible. To make a property read-only when designate a field final. Let's make the brand final and run our Groovy script again. We now get the following exception:

Caught: groovy.lang.ReadOnlyPropertyException: Cannot set readonly property: brand for class: Car

We can add our own get/set methods if we like. For example in the following code we have added our own protected set method for the model property:

class Car {
 int numberOfDoors
 String model
 final String brand  // Readonly, public getter is generated, no setter.
 boolean automatic
 double price
 
 // Protected setter, public getter is still generated.
 protected void setModel(model) {
     this.model = modelName
    }
 
 String toString() {
     "[Car details => brand: '${brand}', model: '${model}', #doors: '${numberOfDoors}', automatic: '${automatic}', price: '${price}']"
 }
}

To access the fields in GroovyBeans directly we can use the .@ operator. With this operator we bypass the get/set methods. Of course we shouldn't be doing this, because that is why we have properties, but still the following code shows how it works:

class Car {
 int numberOfDoors
 String model
 String brand
 boolean automatic
 double price
 
 public void setBrand(brand) {
     this.brand = brand + ' (set via setter method)'
 }
 
 String toString() {
     "[Car details => brand: '${brand}', model: '${model}', #doors: '${numberOfDoors}', automatic: '${automatic}', price: '${price}']"
 }
}

Car ford = new Car(brand: 'Ford', model: 'Focus', numberOfDoors: 4, automatic: false, price: 24995)
println ford.brand  // Output: Ford (set via setter method)

ford.@brand = 'Overrule brand'
println ford.brand  // Output: Overrule brand
println ford.@brand  // Output: Overrule brand

GroovyBeans provide a clean way to define beans with properties usable in Groovy and Java code.

Groovy Goodness: Safe Navigation to Avoid NullPointerException

Little things in live can be such a joy. The safe navigation operator in Groovy is such a little thing. Suppose we have modelled a simple domain like this:

class Company {
    Address address
    String name
}

class Address {
    Street street
    String postalCode
    String city
}

class Street { 
    String name
    String number
    String additionalInfo
}

We want to display the streetname, but we don't know if all object instances are available. To avoid a NullPointerException we write the following code:

// company can be null.
if (company != null && company.getAddress() != null && company.getAddress().getStreet() != null) {
    println company.address.street.name
}

Groovy adds the safe navigation operator to shorten all this to:

// company can be null.
println company?.address?.street?.name

If one of the objects was null, the output of the total statement is null. We will not get any NullPointerExceptions.

Groovy Goodness: the Elvis Operator ?:

Groovy is good at shortening normal Java statements. The Elvis operator is to shorten the ternary operator. If we have a sensible default when the value is null or false (following Groovy truth rules) we can use the Elvis operator. And why is it called the Elvis operator? Turn your head to the left and you will know.

def sampleText

// Normal ternary operator.
def ternaryOutput = (sampleText != null) ? sampleText : 'Hello Groovy!'

// The Elvis operator in action. We must read: 'If sampleText is not null assign
// sampleText to elvisOuput, otherwise assign 'Viva Las Vegas!' to elvisOutput.
def elvisOutput = sampleText ?: 'Viva Las Vegas!'

Serving JSON data the JSONP way with Groovlets

If we want to serve JSON data and want it to be cross-domain accessible we can implement JSONP. This means that if we have a Groovlet with JSON output and want users to be able to access it with an AJAX request from the web browser we must implment JSONP. JSONP isn't difficult at all, but if we implement it our Groovlet is much more useful, because AJAX requests can be made from the web browser to our Groovlet.

The normal browser security model only allows calls to be made to the same domain as the web page from which the calls are made. One of the solutions to overcome this is the use of the script tag to load data. JSONP uses this method and basically let's the client decide on a bit of text to prepend the JSON data and enclose it in parentheses. This way our JSON data is encapsulated in a Javascript method and is valid to be loaded by the script element!

The following code shows a simple JSON data structure:

{ "title" : "Simple JSON Data", 
  "items" : [ 
    { "source" : "document", "author" : "mrhaki" }
    { "source" : "web", "author" : "unknown"}
  ]
}

If the client decides to use the text jsontest19201 to make it JSONP we get:

jsontest19201({ "title" : "Simple JSON Data", 
  "items" : [ 
    { "source" : "document", "author" : "mrhaki" }
    { "source" : "web", "author" : "unknown"}
  ]
})

Okay, so what do we need to have this in our Groovlet? The request for the Groovlet needs to be extended with a query parameter. The value of this query parameter is the text the user decided on to encapsulate the JSON data in. We will use the query parameter callback or jsonp to get the text and prepend it to the JSON data (notice we use Json-lib to create the JSON data):

import net.sf.json.JSONObject

response.setContentType('application/json')

def jsonOutput = JSONObject.fromObject([title: 'Simple JSON data'])
jsonOutput.accumulate('items', [source: 'document', author: 'mrhaki'])
jsonOutput.accumulate('items', [source: 'web', author: 'unknown'])

// Check query parameters callback or jsonp (just wanted to show off
// the Elvis operator - so we have two query parameters)
def jsonp = params.callback ?: params.jsonp
if (jsonp) print jsonp + '('
jsonOutput.write(out)
if (jsonp) print ')'

We deploy this Groovlet to our server. For this blog post I've uploaded the Groovlet to Google App Engine. The complete URL is http://mrhakis.appspot.com/jsonpsample.groovy. So if we get this URL without any query parameters we get:

{"title":"Simple JSON data","items":[{"source":"document","author":"mrhaki"},{"source":"web","author":"unknown"}]}

Now we get this URL again but append the query parameter callback=jsontest90210 (http://mrhakis.appspot.com/jsonpsample.groovy?callback=jsontest90210) and get the following output:

jsontest90210({"title":"Simple JSON data","items":[{"source":"document","author":"mrhaki"},{"source":"web","author":"unknown"}]})

We would have gotten the same result if we used http://mrhakis.appspot.com/jsonpsample.groovy?jsonp=jsontest90210. The good thing is user's can now use for example jQuery's getJSON() method to get the results from our Groovlet from any web page served on any domain.

The following is generated with jQuery.getJSON() and the following code:

$(document).ready(function() {
  $.getJSON('http://mrhakis.appspot.com/jsonpsample.groovy?callback=?', function(data) {
    $.each(data.items, function(i, item) {
      $("<p/>").text("json says: " + item.source + " - " + item.author).appendTo("#jsonsampleitems");
    });
  });
});

JSONP output:

Groovy Goodness: Use a Map as Interface Implementation

To implement an interface we can use a Groovy map and with the as keyword we can pass it on to a method. Here is a simple example where we define an implementation of the java.io.FileFilter as a map. We pass the implementation on to the java.io.File.listFiles() method to display all files with extensions .css and .png:

map = [ 
    // Implement FileFilter.accept(File) method.
    accept: { file -> file.path ==~ /.*\.(css|png)$/ } 
] as FileFilter
new File('c:/temp').listFiles(map).each { 
    println it.path 
}

If the interface we want to implement has more than one method we can only use a map for the implementation. But if the interface only contains one method (as in this case) we can also use a closure for the implementation. In the following example we implement the java.io.FileFilter interface with a closure to filter all files with the extension .jpg:

filter = { it.path ==~ /.*\.jpg$/ }
new File('c:/temp').listFiles(filter as FileFilter).each { file ->
    println file.path
}

Add Client Side Caching to Groovlets and Separate Logic and View

We are going to see how we can implement client side caching for our Groovlet we made in the previous blog. Client side caching decreases bandwidth usage and also lessens the load on our application. Besides a faster response for the users we can even save costs if we have to pay for bandwidth usage if the appliation is running in a cloud for example. We will also refactor the Groovlet and use a view template for displaying the data and remove all HTML generation from the Groovlet. This way we have a better separation between the code and the HTML.

The current Groovlet display a list of activities and each activity has a certain date. We use this date to print out nice human readable, relative timestamps like we see on Twitter. For example 2 days ago, just now, 1 week ago. Each time we refresh the Groovlet we want to update this information. Because the time is relative compared to the current date and time we cannot cache this data. Even if all activity information stays the same we still have to update the human readable timestamps, otherwise the timestamps don't make sense. To solve this and make the data cachable we move the code to generate the human readable timestamps from the server to the client. On the client side we use Javascript to generate the relative date and time values. Now we can implement client side caching for our Groovlet output, because the uncachable date and time values are generated on the client side and not on the server.

Here is the code for the Groovlet. We place it in a file war/WEB-INF/groovy/viewactivities.groovy:

import net.sf.json.groovy.JsonSlurper
import net.sf.json.JSONException
import org.apache.commons.codec.digest.DigestUtils

// We keep track of the last time the content of the Yahoo! Pipe
// has changed in the application context.
now = new Date()
lastUpdated = application.getAttribute('lastUpdated')
if (!lastUpdated) {
    lastUpdated = now
    application.setAttribute('lastUpdated', lastUpdated)
}

// Get the contents of the Yahoo! Pipe.
pipeUrl = 'http://pipes.yahoo.com/pipes/pipe.run?_id=UtVVPkx83hGZ2wKUKX1_0w&_render=json'
result = urlFetchService.fetch(new URL(pipeUrl))
def resultString = new String(result.content)

// The Yahoo! Pipe contains a pubDate element, which contains the date and time
// the Pipe is requested. This changes each time we make a request, even all other
// data stays the same. We filter out this date, so we can make a MD5 hash without
// the dynamic date and time value for better comparison.
resultStringNoPubDate = resultString.replaceFirst(/"pubDate":"\w{3}, \d{2,} \w{3} \d{4} \d{2}:\d{2}:\d{2} [+-]\d{4}",/, '')

// Determine MD5 hash for the Pipe's content.
etag = DigestUtils.md5Hex(resultStringNoPubDate)

// If the request header contains the 'If-None-Match' header we must see if the 
// value is the same as we calculated above. If so the content of the Pipe has not changed
// and we can return a 304 Not Modified response.
// Or if the request header contains a 'If-Modified-Since' header we see if the value
// is the same as the last time the content has changed. We return a 304 Not Modified
// if this is the case.
// The response is send directly to the client without any further content.
if (headers['If-None-Match'] == etag || headers['If-Modified-Since'] == lastUpdated.time) {
    response.sendError javax.servlet.http.HttpServletResponse.SC_NOT_MODIFIED
    return
}

// The content has changed so we update the lastUpdated value.
lastUpdated = now
application.setAttribute('lastUpdated', lastUpdated)

// We set the response headers 'ETag', 'Last-modified'
// and 'Cache-Control'. Cache-control is set to 5 minutes.
response.setHeader('ETag', etag)
response.setDateHeader('Last-modified', lastUpdated.time)
response.setHeader('Cache-Control', 'max-age=' + 5 * 60)

// Set request attributes with values we can use in the 'activities.gtpl' view.
if (resultString) {
    try {
        def jsonReader = new JsonSlurper()
        def json = jsonReader.parseText(resultString)
        request.setAttribute('json', json)

    } catch (JSONException jsonException) {
        request.setAttribute('error', jsonException.getMessage() + "
${resultString}
")
    }
} else {
    request.setAttribute('error', 'No results found. Try again later.')
}

// Go to the view 'activities.gtpl'.
forward 'activities.gtpl' 

At line 22 we remove the value for the pubDate element from the Yahoo! Pipe. This value is updated each time we make a request. If we want to be able to compare the MD5 hash of the content from a previous request with the current result we must exclude the dynamic date and time value. If we don't remove it we always get a different MD5 hash value and we wouldn't be able to send back a 304 Not Modified response.

In line 25 we calculate the MD5 hash with the Apache Commons Codec library. We need to place the JAR file in the war/WEB-INF/lib directory of our Google App Engine application.

At line 34 we see if the request header If-None-Match is available. The value of this header is the MD5 hash of the content from a previous request. We compare this value to the freshly calculated value in line 25. If they are equal we know the content of the Pipe hasn't changed, so the client can use a cached version of the page. As a fallback we also check the If-Modified-Since header in case the client doesn't support If-None-Match.

We set the response header Cache-Control to five minutes at line 47. This means the client can use a cached copy of the page for five minutes before it needs to check the server again to see if something has changed. This means if the content has changed in five minutes after the last request, the client will not see it. For this application that is acceptable, but we always need to check if such a period is acceptable or not for the users.

At lines 50-61 we populate the request with attributes we can use in the view template. The JSON results is stored in a request variable named json. If any error has occured we save it in the request variable error.

At the last line we leave the Groovlet and continue on to the view activites.gtpl.

We create a view activites.gtpl in the directory war to display the results:

<%
import java.text.SimpleDateFormat
import java.text.ParseException

// Closure to parse the date string from the activities.
def parseActivityDate = { dateString ->
    def result
    dateString = dateString.replaceAll(/(\d{2}):(\d{2})$/, '$1$2')
    def parsers = [ 
        new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss Z", Locale.US),
        new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'", Locale.US),
        new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ", Locale.US)
    ]
    for (SimpleDateFormat parser : parsers) {
        try {
            result = parser.parse(dateString)
            break
        } catch (ParseException e) {
            continue
        }
    }
    result        
}
%>

<html>
<head>
    <title>Social Activities</title>
    <link type='text/css' rel='stylesheet' href='css/activities.css'/>
</head>
<body>
    <h1>Activities</h1>

    <% if (request.getAttribute('error')) { %>
      ${request.getAttribute('error')
    <% } else { %>
      <div id="activities">

      <% request.getAttribute('json').value.items.each { item -> %>
        <div class="item">
          <img src="images/socialicons/32x32/${item.source}.png" title="${item.source}" alt="${item.source}"/>
          <a href="${item.link}">${item.title}</a>
          <br />
          <span class="date" title="${parseActivityDate(item.pubDate).format("yyyy-MM-dd'T'HH:mm:ss")}"> </span>
      </div>
    <% } %>
    </div>
  
    <script src='js/jquery.js' type='text/javascript'> </script>
    <script src='js/humane.js' type='text/javascript'> </script>
    <script type='text/javascript'>
     \$(document).ready(function() {
       \$("span.date").humane_dates();
       setInterval(function() {
        \$("span.date").humane_dates();
       }, 5000);
    });
    </script>
   <% } %>

</body>
</html>

At the top we define a closure to parse the date strings from the JSON results. We use it at line 43 to format the date for the title attribute of the span element. At line 50 we import the Yet Another Pretty Date Javascript library. This library will use the value of the title attribute of an element and create a human readable timestamp. At lines 52-57 we define Javascript to calculate the timestamp values and display them on the page. We also define an interval of five seconds to update the information. This way the information is always up-to-date without refreshing the page. Notice how we must escape the $ sign to have it placed literally in the output, otherwise Groovy thinks it is an Groovy expression.

The rest of the code is straightforward. At line 38 we loop through the JSON results and display a div element with the information for each item.

We have seen how we can add reponse headers to our Groovlet to support client side caching. We set the Cache-Control header to minimize the number of requests to the server. And if a request hits our Groovlet we check if the contents has changed. If there is no change we return a 304 Not Modified response, so a cached version on the client side can be used. Furthermore we refactored the previous Groovlet to separate the logic from the view. A more spiced up example of the application is available on Google App Engine.

Groovy Goodness: String.multiply()

I really like the Groovy extensions to the String class. Today I discoverd the multiply() method. We want to repeat a dash 80 times? Solution:

'-'.multiply(80)

Simple and readable. But it can even be shorter. As John Flinchbaugh mentioned in the comments, the multiply method is nothing more than an overload for the operator *. So we get the same result with the following statement:

'-' * 80

Using more Google App Engine SDK services with a Groovlet and Gaelyk

In this blog post we learn how we can use the Google App Engine SDK API with Gaelyk and on the side we learn some Groovy syntax suggar. In a previous blog post we learned how easy it is to write and deploy a Groovlet with Gaelyk. We are going to extend this Groovlet with caching of the Yahoo! Pipe results. Each time a user requests the Groovlet, also the Yahoo! Pipe is requested. The results are transformed to a HTML page. Since we are using the URLFetchService from the Google App Engine, each request has a cost. Luckely we can make a lot of invocations on the URLFetchService, but still we have to take into account the number of invocations is limited.

To reduce the number of invocations on the URLFetchService and to speed things up, we will cache the Yahoo! Pipe results with the Google App Engine MemCacheService. Because the MemCacheService is part of the Google App Engine SDK API we are also limited to a number of invocations. But this number is much bigger than for the URLFetchService, so we can serve much more requests before we hit the limit.

Using the MemCacheService is very easy: the service is already injected into the Groovlet by Gaelyk and is available as memcacheService. The service itself is basically a hashmap we can store data in and retrieve data from. To save the Yahoo! Pipe results we only have to invoke the put(key, value) method. A downside of the Yahoo! Pipe request is that we don't get any response headers like Last-Modified or ETag. Normally we could have used this headers to determine if the results have changed and if we had to make a new request, or that we could use the cached value. But because we don't get this information we use a different caching mechanism: we store the results for five minutes in the cache. When a user requests the Groovlet again after five minutes we refresh the cache and make a new request for the Yahoo! Pipe. In a next blog post we see how we can add response headers Last-Modified and ETag to our Groovlet. Clients can use this information to use cached results if necessary. This will lessen the number of invocations on the URLFetchService and MemCacheService and increase the response time of the Groovlet.

Here is the code of the rewritten Groovlet with caching of the Yahoo! Pipe results:

import java.net.URL
import java.text.SimpleDateFormat
import java.text.ParseException
import org.codehaus.groovy.runtime.TimeCategory
import net.sf.json.groovy.JsonSlurper
import net.sf.json.JSONException
import com.ocpsoft.pretty.time.PrettyTime

/*
 * Groovy Category to extend the Date and String class with
 * the methods prettyTime and parseActivityDate.
 * We use the <a href="http://ocpsoft.com/prettytime/">PrettyTime</a>
 * library to get results like 9 minutes ago, 3 days ago.
 */
class ActivityDateTimeCategory {
    static String prettyTime(Date date) {
        new PrettyTime().format(date)
    }
    
    static Date parseActivityDate(String dateString) {        
        def result
        
        // Remove the colon from the timezone, because otherwise we can't
        // parse the date.
        dateString = dateString.replaceAll(/(\d{2}):(\d{2})$/, '$1$2')
        
        // The date formats of the different sources. We try each one
        // to parse a date.
        def parsers = [ 
            new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss Z", Locale.US),
            new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'", Locale.US),
            new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ", Locale.US)
        ]
        for (SimpleDateFormat parser : parsers) {
            try {
                result = parser.parse(dateString)
                break
            } catch (ParseException e) {
                continue
            }
        }
        result        
    }
}

// If the request parameter refresh is set we always
// refresh the cache with the latest Yahoo! Pipe result.
refresh = params.refresh

// The number of minutes before we refresh the cache.
period = params.period ? Integer.valueOf(params.period) : 5

now = new Date()
lastUpdated = application.getAttribute('lastUpdated')
if (!lastUpdated) {
    lastUpdated = now
    application.setAttribute('lastUpdated', lastUpdated)
    refresh = true
}

// Using the TimeCategory so we can use the cool ?.minutes syntax.
use (TimeCategory) {
    checkDate = lastUpdated + period.minutes
    if (now.after(checkDate)) {
        lastUpdated = now
        application.setAttribute('lastUpdated', lastUpdated)
        refresh = true
    }
}

def resultString
if (!refresh) {
    resultString = memcacheService.get('result')
}

// If we have to refresh, or the resultString from the cache is null
// we make a new request.
if (!resultString) {
    def pipeUrl = 'http://pipes.yahoo.com/pipes/pipe.run?_id=UtVVPkx83hGZ2wKUKX1_0w&_render=json'
    def result = urlFetchService.fetch(new URL(pipeUrl))
    resultString = new String(result.content)
    memcacheService.put('result', resultString)
}

html.html { 
    head {
        style (
            """
            img { 
                width: 32px; height: 32px; float: left; padding: 4px 4px 0 0;
            }
            .item {
                border-bottom: 1px solid #999;
                padding: 5px;
            }
            #activities {
                border-top: 1px solid #999;
            }
            """
        )
    }
    body { 
        h1 "Activities" 
        
        if (resultString) {
            try {
                def jsonReader = new JsonSlurper()
                def json = jsonReader.parseText(resultString)

                div(id: 'activities') {
                    json.value.items.each { item ->
                        div(class: 'item') {
                            img(src: 'http://localhost/images/socialicons/32x32/' + item.source + '.png', width: 32, height: 32, alt: item.source, title: item.source)
                            a(href: item.link, item.title)
                            br()
                            // Using our own Category and showing of ?. Groovy safe operator.
                            use (ActivityDateTimeCategory) {
                                span(class: 'date', item.pubDate?.parseActivityDate()?.prettyTime());
                            }
                        }
                    }
                } 
            } catch (JSONException jsonException) {
                p jsonException.getMessage()
                pre resultString
            }
        } else {
            p 'No results found. Try again later.'
        }
    }
}

A simple Groovlet to read and display JSON data on Google App Engine with Gaelyk

It's time to use our Yahoo! Pipe to display the JSON results on a Groovy generated page running on Google App Engine. Why? Because we can! In this blog post we learn how easy it is to deploy a simple Groovlet to the Google Appengine and how easy it is to parse JSON results.

We start with Gaelyk, the lightweight Groovy toolkit for Google App Engine. With Gaelyk we can build Groovlets and use the Google App Engine SDK APIs in our Groovy code. To get started with Gaelyk we download the template project which contains almost everything we need. After downloading the ZIP file we must expand it to a directory on our computer. Next we open the file war/WEB-INF/appengine-web.xml. In the application element we fill in our own Google App Engine application identifier. Our project is now configured for Google App Engine, that is all we need to do!

Next we create a new Groovlet at war/WEB-INF/groovy/pipes.groovy:

import java.net.URL
import net.sf.json.groovy.JsonSlurper

def pipeURL = 'http://pipes.yahoo.com/pipes/pipe.run?_id=UtVVPkx83hGZ2wKUKX1_0w&_render=json'
def pipeResult = urlFetchService.fetch(new URL(pipeURL))
def json = new JsonSlurper().parseText(new String(pipeResult.content))

html.html {
 head {
   style("""
     .item {
       border-bottom: 1px solid #eee;
       padding: 5px;
     }
     .date {
       font-style: italic;
     }
     #activities {
       border-top: 1px solid #eee;
     }
   """)
 }
 body {
  div(id: 'activities') {
   json.value.items.each { item -> 
    div(class: 'item') {
     a(href: item.link, item.title)
     br()
     span(class: 'date', item.pubDate)
    }
   }
  }
 }
}

In this Groovlet we use the URLFetchService from the Google App Engine SDK API. With this service we invoke our Yahoo! Pipe and get the results using Google's infrastructure. The service is automatically injected by the Gaelyk toolkit so we can use the service immediately. At line 4 we see the call to get the Yahoo! Pipe results.

The result of the pipe is then parsed by the JsonSlurper from the JSON-lib library (line 5). The JsonSlurper is modeled after the XmlSlurper to read in JSON data instead of XML data.

At line 7 we start with building the HTML output. A Groovlet has the implicit variable html, which is actually a MarkupBuilder connected to the servlet output. The HTML is straight forward. In the head section we define some CSS styles, in the body we have a div block with id activities. At line 24 we use the each closure to walk throught the Yahoo! Pipe results and create a new div element with information from the Pipe. We use the item's link and title field to create an HTML anchor and at a new line we print the item's pubDate field enclosed by a span element.

We need to copy the JSON-lib JAR files (including dependencies) to the war/WEB-INF/lib directory, so the Groovlet can use the JsonSlurper class.

To test the Groovlet we run the Google App Engine development server from the command-line:

$ dev_appserver war

In our browser we open http://localhost:8080/pipes.groovy and we get an HTML formatted list of the Yahoo! Pipe JSON output. To deploy the application to the Google App Engine we only have to execute one command:

$ appcfg update war

Well that is it! A simple Groovlet using the Google App Engine SKD API to read a JSON source and display the results as HTML. In a next blog post we will learn how to pretty print the date (using PrettyTime) and how we can use a separate view to display the results. Further we will optimize the Yahoo! Pipe request by using the MemCache service from Google App Engine SDK API.

Weekly sightings

BeoTime

This looks like a flute, but it is an amazing alarm clock. With a movement sensor the display adepts to how the alarm clock is held. And of course we can use any B&O appliance to wake up to. Read more about the design.

'Invalid label' when using jQuery.getJSON() method

I was working on a small project and I wanted to test some Javascript code that read data formatted as JSON. I setup a small HTML file, sample.html, with a reference to the jQuery library and invoke $.getJSON("http://localhost/example.json", function(data) { alert(data); });. In Firefox I opened http://localhost/sample.html and suspected to see some data. But instead I got an Invalid label in Firebug. What happened? After reading this Stackoverflow answer I knew why. I invoked the $.getJSON() method with a URL starting with http://, and jQuery automatically assumes the request is a JSONP request. But in this case that is not true. Replacing the URL with the relative example.json is the solution.

Reuse Yahoo! Pipes with parameters

Yahoo! Pipes are useful to get and manipulate data from the web. We can provide user defined values to a pipe to configure modules in the pipe. The user input adds flexibility to a pipe at runtime. The user value can be passed to the pipe URL (really useful) or asked when we use the Run Pipe link.

We create a new pipe where we want to do a Twitter search and convert the results to a RSS feed. The parameters for the Twitter search are provided by the user and can change. The User inputs section contains several input modules. We use the Text Input module to get the search query. This module has 5 fields. The Name field is important now, because that is the name of the parameter we can use in the pipe URL. We can even provide a default value if the input is not given with the Default field. The Debug field is for testing the pipe: the value we fill in is used in the Pipe editor and the Debugger. The Prompt and Position fields are used when we use the Run Pipe link, but we don't use them for our example. We use search for the Name field and type Yahoo Pipes as a Debug value.

Next we use this input module to build a correct Twitter search string. We drop the URL Builder builder module in the editor. The Base URL is http://search.twitter.com, the Path element is search.json and we have one query parameter named q. The value of the q parameter is the value of our input module. So we connect the input module to the grey dot to the right of the value field of the q parameter. When the pipe is executed the value of the q parameter will be the value of the pipe's input module. We can check the value of the complete Twitter search URL in the Debugger window: http://search.twitter.com/search.json?q=Yahoo+Pipes

We have our Twitter search URL and pass it on to a Fetch Data module. The results of the Twitter search are in JSON format. To convert these into a RSS feed we use the Create RSS module from the Sources section. We simply map fields from JSON to RSS fields.

Here is the complete pipe:

To search for "Yahoo Pipes" we invoke the following URL: http://pipes.yahoo.com/mrhaki/uBirAfB_3hGNHsvYwmH_9A?search=Yahoo+Pipes. But we can also use this pipe to search for "Grails": http://pipes.yahoo.com/mrhaki/uBirAfB_3hGNHsvYwmH_9A?search=Grails, or we search for Java: http://pipes.yahoo.com/mrhaki/uBirAfB_3hGNHsvYwmH_9A?search=Java.

Using jQuery's JSONP support to get data from Yahoo! Pipe

In a previous blog entry we learned how to create a Yahoo! Pipe. In this blog entry we will see how we can use jQuery to parse the Yahoo! Pipe's output.

When we run our pipe we have a link Get as JSON. We need the URL of this link so we can use it with our jQuery script. Basically it is the URL of the pipe appended with &_render=json.

Normally we cannot get data from just any domain on the internet with Javascript. But with JSONP we can get data from other domains. jQuery and Yahoo! Pipes supports JSONP, we only have to add &_callback=? to our Yahoo! Pipe JSON URL and we are in business. We use the $.getJSON method to get the data from our pipe. Once we get hold of the data we can use it any way we want. The following code snippet loads the data and then creates a list with the title, link, date and source of the items:

$(document).ready(function() {
    $.getJSON("http://pipes.yahoo.com/pipes/pipe.run?_id=UtVVPkx83hGZ2wKUKX1_0w&_render=json&_callback=?", 
    function(data) {
        $.each(data.value.items, function(idx, item) {
            var listitem = $("<li/>");
            $("<a/>").attr("href", item.link).attr("class", item.source).text(item.title).appendTo(listitem);
            $("<br />").appendTo(listitem);
            $("<span/>").attr("class", "source").text("from " + item.source).appendTo(listitem);
            $("<span/>").text(", " + item.pubDate).appendTo(listitem);
            listitem.appendTo("#activities");  // append to UL element with id activities
        });
    });
});

All put together in a HTML page and a little styling we get the following output:

And here is the complete HTML source:

<html>
<head>
  <script src="http://code.jquery.com/jquery-latest.js"></script>

  <script>
  $(document).ready(function() {
    $.getJSON("http://pipes.yahoo.com/pipes/pipe.run?_id=UtVVPkx83hGZ2wKUKX1_0w&_render=json&_callback=?",
        function(data) {
          $.each(data.value.items, function(idx,item) {
            var listitem = $("<li/>");
            $("<a/>").attr("href", item.link).attr("class", item.source).text(item.title).appendTo(listitem);
            $("<br />").appendTo(listitem);
            $("<span/>").attr("class", "source").text("from " + item.source).appendTo(listitem);
            $("<span/>").text(", " + item.pubDate).appendTo(listitem);
            listitem.appendTo("#activities");
          });
        });
  });
  </script>
  <style>
    .source { font-style: italic; }
  </style>
</head>
<body>
  <h1>Yahoo! Pipes</h1>
  <ul id="activities">
  </ul>
</body>
</html>

Using Yahoo! Pipes to combine Blogger, Delicious, Twitter and Google Reader data

Yahoo! Pipes make it possible to aggregate data from different sources, change the data and combine it to a single output. In this post we will see how we can use output from our Blogger, Delicious, Twitter and Google Reader accounts and display them in one list.

We create Yahoo! Pipes using the Editor tool in our web browser. We go to Yahoo! Pipes. If we don't already have an account we must create one. Once we are logged in we click on the Create a pipe button. We get a big clean sheet to drop modules on. These modules are located on the left side of the screen. The modules can be used to define sources, operations and different manipulations. At the bottom of the screen we have the Debugger window. The Debugger window shows the contents of the pipe.

Let's first create a complete pipe to get the contents from our Twitter account, add some field to identify that the source is Twitter. We go to our Twitter page and look for the RSS feed link at the right and select the link. For my Twitter account the link is http://twitter.com/statuses/user_timeline/19233075.rss. In the Yahoo! Pipes Editor we select the module Fetch Feed from the Sources section and drop it on the canvas. Now we fill in the RSS feed link in the URL input field:

If we click on the module (we know it is active when the title bar turns orange) and look at the Debugger window we see the results of the feed:

Well, that is a start. Now we are going to add a new field to the output. The easiest way to do this is to copy an existing field. We need the Rename module from the Operators section to achieve this. With this module we can rename an existing field, but also copy a field. We drag the Rename module on the canvas. We connect the Fetch Feed module to the Rename module by dragging our mouse from the connector dot at the bottom of the Fetch Feed module to the connector dot at the top of the Rename module. Both modules are connected and the fields from the feed are now available in a drop-down list in the Mappings of the Rename module. For our example we copy the item.guid field to a source field. When we refresh the Debugger window we see the new field source in the output:

The contents of the source field must be changed to the value twitter, so we know from wich source the contents is acquired. We drop the RegEx module on the canvas and connect the Rename module to the Regex module. In the Rules section we select from the drop-down list item.source. We want to replace the whole text so we apply the regular expression .* to replace it with the text twitter. We click on the refresh link in the Debugger window we see that the source field now has the value twitter:

These steps need to repeated for every feed we want in our final output. For example if we want to include the output of our Delicous account we have to visit our Delicous page. Here we select the RSS link from the bottom of the page (mine is http://feeds.delicious.com/v2/rss/mrhaki?count=15. We use this URL for another Fetch Feed module. Then we add a Rename and Regex module to create a new source field with the value delicious. We can lookup the RSS links for our Blogger account and Google Reader account as well and add them to the canvas.

To connect both feeds into one output we use the Union module from the Operators section. We connect both Regexp modules to the top of the Union module:

We are almost done. For good measure we will sort the aggregated feed by date, so the newest items from the feeds are displayed at the top of the output. We drop the Sort module from the Operators section. We select item.pubDate and sort in descending order:

Finally we can join the Sort module to the Pipe Output to finish the pipe. To really use our pipe we must first save it. We click on the Save button and fill in a name for the pipe. After we have saved the pipe we get a link Run pipe... at the top of our window. We click on this link and a new window opens with our pipe output:

That's it. We can even subscribe to our newly created feed in a RSS reader, or we can create a badge for our website to display the results. In a next blog entry we will use the Get as JSON link to get the results in JSON format and use it with jQuery to display the results.