Sending Asynchronous HTML Email in Grails with ActiveMQ, JMS and Gmail

Sending email asynchronously is an essential part of most Web applications and there are many ways in which to implement it. In this post I’ve chosen to demonstrate how to set up JMS, ActiveMQ and Gmail with the Grails Mail plugin to provide asynchronous email capabilities while only having to write a few lines of code, all thanks to some great plugins. I’ll also show the fantastic Grails Mail templating feature which uses GSP views to generate HTML emails along with a solution for one of the big bugs with it too!

Design

Below is an overview of the system I’ll be building.

Design Overview

Design Overview

The EmailService handles the synchronous part of sending (queueing, adapting, formatting), the SendMailQueue is ActiveMQ via JMS and the SendMailService handles the background sending of the email using the Grails Mail plugin connecting to Gmail over SMTPS/SSL.

Setup

You will need Grails 1.2.x or newer and these plugins:

You will also need a Gmail account if you want to reproduce this entire tutorial. I set up a premium account but the configuration should be the same for a free account.

Configuration

Most of the effort is in getting the configuration right, and it’s not really any work at all as I’ve worked it out for you! We just need to set up an ActiveMQ Spring bean as our JMS provider and configure the mail plugin to use a Gmail account.

Spring Configuration

If you don’t have an external ActiveMQ server you’ll need to set up a local one by adding a connection factory bean. In the following example I also set up a connection pool to improve performance (otherwise there’s a performance hit on every message sent). Just add this to Resources.groovy:

beans = {
    jmsConnectionFactory(org.apache.activemq.pool.PooledConnectionFactory){bean ->
        bean.destroyMethod ="stop"
        connectionFactory(org.apache.activemq.ActiveMQConnectionFactory) {
            brokerURL = "tcp://localhost:61616"
        }
    }
}

JavaMail settings for Gmail

There is a huge amount of conflicting information about how JavaMail should be configured for Gmail, it’s an interesting case as it requires an SSL connection, rather than a plain SMTP one. However, I can guarantee that the following settings work for the Grails Mail plugin (and therefore also for Spring’s SimpleMailMessage) with a Gmail account.

In Config.groovy add the following (and replace the username and password with your own!):

grails.mail.host = "smtp.gmail.com"
grails.mail.from = "address@mydomain.com"
grails.mail.port = "465"
grails.mail.username = "user@mydomain.com"
grails.mail.password = "mypassword"
grails.mail.props = ["mail.smtp.auth": "true",
        "mail.smtp.socketFactory.port": "465",
        "mail.smtp.socketFactory.class": "javax.net.ssl.SSLSocketFactory",
        "mail.smtp.socketFactory.fallback": "false",
        "mail.smtp.starttls.enable": "true",
        "mail.debug": "true"]

The last item in the props setting – mail.debug – provides some useful logging, more on that below, but you can remove it when you have it working.

Of course, it’s unlikely that you’ll have the same mail server for dev, test and prod so you can set up different environment configurations:

environments {
    production {
        grails.mail.host = "smtp.gmail.com"
        grails.mail.port = "465"
        grails.mail.username = "user@mydomain.com"
        grails.mail.password = "mypassword"
        grails.mail.props = ["mail.smtp.auth": "true",
                "mail.smtp.socketFactory.port": "465",
                "mail.smtp.socketFactory.class": "javax.net.ssl.SSLSocketFactory",
                "mail.smtp.socketFactory.fallback": "false",
                "mail.smtp.starttls.enable": "true",
                "mail.debug": "true"]

    }
    development {
        grails.mail.host = "smtp.gmail.com"
        grails.mail.port = "465"
        grails.mail.username = "devuser@mydevdomain.com"
        grails.mail.password = "mydevpassword"
        grails.mail.props = ["mail.smtp.auth": "true",
                "mail.smtp.socketFactory.port": "465",
                "mail.smtp.socketFactory.class": "javax.net.ssl.SSLSocketFactory",
                "mail.smtp.socketFactory.fallback": "false",
                "mail.smtp.starttls.enable": "true",
                "mail.debug": "true"]
    }
}

You may be tempted to add the config like this:

development {
        grails {
            mail {
                host = "smtp.gmail.com"
                port = ...

Don’t! By defining a “grails” closure you will clear all previously defined settings for the “grails” key!

Implementing the Services

We’re almost finished and we haven’t yet written a line of real code! Let’s start with the EmailService which will help us queue the messages:

EmailService

Let’s create a new non-transactional service named EmailService, in this service we could add any request processing, security checks and adapt the request for queuing.

package lourish

class EmailService {

    boolean transactional = false

    def sendEmail(attrs) {
        sendJMSMessage("sendMail",
                [to: attrs.to,
                 from: attrs.from,
                 subject: attrs.subject,
                 view: attrs.view,
                 model: attrs.model])
    }
}

This service method is just one call to sendJMSMessage (a meta-method added to all services and controllers by the JMS plugin). The first argument is the queue name we’re sending to and the second is the JMS message itself, in this case a map defining the email to send.

Make sure that everything you send over an ActiveMQ queue is a primitive type, map or list – it ought to be possible to send serializable objects but in my experience it simply doesn’t work!

SendMailService

The JMS plugin is set up to deliver messages to any service “exposed” to JMS which has the same name as the queue the message is on. So in the EmailService we send the message on the sendMail queue so the JMS plugin will try to find a service named SendMailService and call its onMessage method:

package lourish

class SendMailService {
    boolean transactional = false

    //Tell the JMS plugin that this is a message endpoint
    static expose = ['jms']

    //The Mail plugin service
    def mailService

    def onMessage(emailMessage) {
        try {
            mailService.sendMail {
                to emailMessage.to
                from emailMessage.from
                subject emailMessage.subject
                body(view: emailMessage.view, model: emailMessage.model)

            }
        } catch (Exception e) {
            log.error("Failed to send email ${emailMessage}", e)
        }
        //Return null to avoid poison messages
        return null
    }
}

Notice that the onMessage method is returning null – this is important as it tells the MessageListenerAdapter that the method has run successfully. If you return anything else it will be assumed it’s a retry message and you could end up with a poison messaged in your queue.

Also note that the “mailService” bean is set up by the Mail plugin and there are more details in the plugin documentation about how it works.

HTML Templates

As we’re sending HTML emails we need to create a template GSP for the message body as defined in the “view” parameter of the JMS message. Create the following helloWorld.gsp template in grails-app/views/emails:

<%@ page contentType="text/html" %>
<html>
<body><h1>${greeting}</h1></body>
</html>

I’d recommend a good read of the Mail plugin and Spring JMS docs for full details. Bear in mind that the email templates don’t support sitemesh layouts.

Sending a Mail

That’s the services created and the system configured so we can try it out.

An Example Controller

Here’s an example of a controller to send a test email to demonstrate how the whole thing works:

package lourish

class SendMailController {
    //Our email queuing service
    def emailService

    def index = {
        try {
            emailService.send(to: "myemail@emailaddress.com",
                from: "me@gmail.com",
                subject: "Test HTML mail using JMS, ActiveMQ, and the Grails Mail plugin",
                model: [greeting: "Hello World!"],
                view: "/emails/helloWorld")

        } catch (Exception e) {
            log.error("${e}", e)
            flash.message = "Mail not sent: ${e}"
            return
        }
        flash.message = "Mail sent"
    }
}

and a corresponding GSP in views/sendMail/index.gsp:

<%@ page contentType="text/html;charset=UTF-8" %>
<html>
<head><title>Mail Test</title></head>
<body>${flash.message}</body>
</html>

You can now run the app and browse to /sendMail to send an email.

Debug

When you run the app, you should not only receive an email but you should also see some logging from Java mail if you have the debug in. It will look a bit like this:

DEBUG: JavaMail version 1.4.1
DEBUG: not loading file: C:\java\jdk1.6.0_16\jre\lib\javamail.providers
DEBUG: java.io.FileNotFoundException: C:\java\jdk1.6.0_16\jre\lib\javamail.providers (The system cannot find the file specified)
DEBUG: !anyLoaded
DEBUG: not loading resource: /META-INF/javamail.providers
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Sun Microsystems, Inc], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Sun Microsystems, Inc], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Sun Microsystems, Inc], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Sun Microsystems, Inc], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Sun Microsystems, Inc], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Sun Microsystems, Inc]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Sun Microsystems, Inc], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Sun Microsystems, Inc], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Sun Microsystems, Inc], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Sun Microsystems, Inc], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Sun Microsystems, Inc], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Sun Microsystems, Inc]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: !anyLoaded
DEBUG: not loading resource: /META-INF/javamail.address.map
DEBUG: not loading file: C:\java\jdk1.6.0_16\jre\lib\javamail.address.map
DEBUG: java.io.FileNotFoundException: C:\java\jdk1.6.0_16\jre\lib\javamail.address.map (The system cannot find the file specified)

If it gets to this point and stops, it means that the mailService bean has initialised but sending the mail has failed because of some application issue before connecting to the remote mail server. Note that the exceptions in the logging above are benign if they’re at DEBUG level so don’t be confused by them, you’ll need to check your application logging for the real problem.

Running in a WAR

There is a bug in the Mail plugin which means that sending mail using templates fails when an application is packaged as a WAR and deployed in a container. The exception you’ll see is:

org.springframework.jms.listener.adapter.ListenerExecutionFailedException:
 Listener method 'onMessage' threw exception; nested exception is
  java.lang.NoClassDefFoundError: org/springframework/mock/web/MockHttpServletRequest

The problem is that the Mail plugin relies on the MockHttpServletRequest class in the spring test jar which is unavailable when running in a WAR, but present when using run-app. To resolve this issue it’s necessary to add the Spring Framework test jar, in my case $GRAILS_HOME/lib/org.springframework.test-3.0.0.RELEASE.jar, to your lib directory. It’s an ugly fix, but hopefully it will only be temporary.

Further Reading

I hope that I’ve provided a useful overview of queued email which is suitable for production Web applications. In most large companies the mail server is likely to be local so you probably won’t have to deal with Gmail, however the concepts remain the same.

For more information on the topics in this post also see:

About these ads

22 thoughts on “Sending Asynchronous HTML Email in Grails with ActiveMQ, JMS and Gmail

  1. Hi,

    Nice work. Indeed, helped me alot.

    It worked at my end but i need to tweak the code a bit in SendMailService like:

    static destination = “My Queue Name”

    Because, my queue did’t send mail until unless i declared the “hardcoded” destination.

    Now, i need to achieve following two objectives:

    1. Send JMS messages to non-hardcoded queue like X queue (read from db) and then consume the message from the same.

    2. An UI interface like ActiveMQ admin panel. In order to check-out the status of my N queues.

    It would be great, if you could help me somehow?

    Thanks alot in advance.

    -Vikas

  2. Pingback: Configure Grails AcegiSecurity Plugin with GMail | David Gustafsson
  3. Hi!

    Thanks for a couple of good posts. I am new to Grails but have worked with Hibernate and Spring before. So far I find grails very interesting!

    I am planning to take up a major project soon, what is your arguments for using Grails instead of Ruby on Rails? Is there any solution to make Grails compile faster or to not having to recompile every time you change a controller?

    -david@techonomics.se

    • Hi David, the why Grails question keeps coming up and it probably deserves a proper post, however I’ll try to answer your 2 questions briefly:

      “What are your arguments for using Grails instead of Ruby on Rails?”

      I’ve never used RoR and that was a big reason for me to pick up Grails – I know Java, Spring and Hibernate so the learning curve isn’t so steep. Another reason was that if you find a certain part of the system doesn’t perform you can rewrite in Java. And my final reason was that I’m not a fan of the Active Record pattern as I find it leads to code which is difficult to maintain. Since starting to use grails I’ve found many more good reasons to choose it such as groovy, ease of testing, and the many great plugins, however I understand that RoR is much more mature, if less performant.

      “Is there any solution to make Grails compile faster or to not having to recompile every time you change a controller”

      Giving Grails more memory, specifically increasing the permgen space in the JVM (I use 250m), can help. Also not going overboard on plugins helps – only keep the ones you use. As for recompiling when changing a controller, that shouldn’t be necessary. I can go hours without recompiling as long as I don’t touch the domain or service classes.

      Hope that helps

      Dave

  4. Great. Thanks for your reply!

    Another question; which web-hotel/hosting do you use/recommend?

    Also; how do you increase the permgen space in the JVM? I saw someone doing it just for the web container (tomcat) but how can you do it for the default container for grails? Are you changing the setting for the entire jvm by changing an environment variable (JAVA_OPTS=%JAVA_OPTS% -XX:PermSize=64m -XX:MaxPermSize=256m )?

  5. I use CloudFoundry at the moment, you can set the JVM options on launch using their console but when running from a WAR there should be no need to increase the permgen space as everything’s precompiled.

    For development I use Idea and that has a place to add JVM options too.

  6. I got it to send domain object in the model! You just have to send an object instead of a map. I just changed EmailService to take a EmailRequest object. Make sure all of the objects you send implement Serializable.

    class EmailRequest implements Serializable {
    String to
    String from
    String subject
    String view
    def model
    }

    class EmailService {

    boolean transactional = false

    def send(EmailRequest emailRequest) {
    sendJMSMessage(“queue.sendMail”, emailRequest)
    }
    }

    class SendMailService {

    boolean transactional = false

    //Tell the JMS plugin that this is a message endpoint
    static exposes = ['jms']
    static destination = “queue.sendMail”

    def onMessage(emailRequest) {
    try {
    sendMail {
    to emailRequest.to
    from emailRequest.from
    subject emailRequest.subject
    body(view: emailRequest.view, model: emailRequest.model)

    }
    } catch (Exception e) {
    log.error(“Failed to send email ${emailRequest}”, e)
    }

    //Return null to avoid poison messages
    return null
    }
    }

  7. Also with the new activeMQ plugin, I didn’t have to add the following to my resources.groovy.

    beans = {
    jmsConnectionFactory(org.apache.activemq.pool.PooledConnectionFactory){bean ->
    bean.destroyMethod =”stop”
    connectionFactory(org.apache.activemq.ActiveMQConnectionFactory) {
    brokerURL = “tcp://localhost:61616″
    }
    }
    }

  8. >If it gets to this point and stops, it means that the
    >mailService bean has initialised but sending the mail
    >has failed because of some application issue before
    >connecting to the remote mail server

    In the connection to the mail server fails, will the system automatically retry every so often?

    • No, there’s no auto-retry by default, but you could implement it quite easily by re-adding the message to the queue, counting the number of tries made. (I’d be careful of using a transactional queue as you’ll almost certainly end up with a poison message unless you carefully consider your redelivery policy).

  9. I was adapting your example into my implementation and I ran across the following error:
    ‘…Property org not defined in resources.groovy’.

    After investigation, I realized that with Grails 1.4.M1 release, you need to add the following to your BuildConfig.groovy under dependencies {…}

    runtime ‘org.apache.activemq:activemq-core:5.5.0′
    runtime ‘org.apache.activemq:activemq-pool:5.5.0′

    Sharing this info for others. Someone else out there might run into the same issue.

    Thanks

    Venkatt

  10. One more tip that might save some time (especially for users migrating to Grails 1.4.M1).

    In your example, your SendMailService class is inside a package called “lourish”. So you will need to modify your EmailService class that current invokes:
    sendJMSMessage(“sendMail”, [to: attrs.to, …)
    to
    sendJMSMessage(“lourish.sendMail”, [to: attrs.to, …)

    If you don’t, you might be spinning your wheels wondering why your messages aren’t being picked up from the queue.

    Cheers.

    Venkatt

  11. What do you do when you have to give a gift to a person whom you don’t know much about.
    This helps in adding the personal touch in the gifts.

    Get ideas for items to make for children, teens and adults.

  12. However, it is absolutely essential to understand that the term HEPA should not equate to
    immediate trust of the purifier using it. If you have the opportunity, you are
    advised to read online reviews of multiple air purifiers.
    HEPA is an abbreviation to describe a ‘High Efficiency Particulate Arresting’ filter, originally designed in the 1940s by
    the US military to prevent the spread of airborne radioactive particles.

  13. Factor in the cost of everything before you head to the store.

    Instead, she might appreciate a $5 gift card to the local coffee shop inside a refillable coffee mug – something she will enjoy on cold winter mornings driving a bus full of children around town.
    or before you even think about applying for another credit card.

  14. A fabulous feature for the kitchen is a wall oven, and ought to be considered by every homeowner.

    Crab and Cheese Rounds – Cocktail-size bread rounds make a handy base
    for many appetizer offerings. If you are in a similar situation, read on and I will explain the process I went through to wire my new oven.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s