December 12, 2008

Using Groovy to invoke cURL and use a date range

I want to learn more about Groovy, so I keep looking for ways to use Groovy in our project. The best way to learn something new is to use it in a real live situation.

For our project we have a Unix shell script to access a URL with a variable date. The URL is http://servername/import-data/{yyyy-MM-dd}, where {yyyy-MM-dd} is a date. So if we want to import data for the first of December 2008, we use http://servername/import-data/2008-12-01. In the shell script we access this URL for all dates from now to the next month. So let's say today is December 1st 2008 then the script invokes http://servername/import-data/2008-12-01, next the scripts invokes http://servername/import-data/2008-12-02 and so on.

The URL is accessed with the program cURL. This way we can save the output and have some nice feedback on the console. The complete command for a single day is curl -o log/2008-12-01.log http://servername/import-data/2008-12-01.

I was able to rewrite the script into the following Groovy script:

// Start from today.
def today = new Date()
// Determine date value for the next month.
// The TimeCategory makes it possible to use the 1.month syntax.
def nextMonth
use (org.codehaus.groovy.runtime.TimeCategory) {
  nextMonth = today + 1.month
// Loop through every day from now to the next month.
for (day in today..nextMonth) {
  println "Starting import for ${day.format('dd MMMM yyyy')}."
  // Define cURL process with correct arguments.
  def proc = "curl -o log/${day.format('yyyy-MM-dd')}.log "
           + "http://servername/import-data/${day.format('yyyy-MM-dd')}"
  // cURL uses error output stream for progress output.
  Thread.start { System.err << proc.err } 
  // Wait until cURL process finished and continue with the loop.

I am impressed by the power of Groovy to write small applications. The language contains nice features like date ranges, date related DSL and simple execution of external processes as demonstrated in this small application.