Premium Hosted Website & Server Monitoring Tool.

(Sysadmin / Devops blog)

visit our website

Blog   >   Android   >   Android push notifications (tutorial)

Android push notifications (tutorial)

android-push-notifications-tutorial

C2DM

Perhaps the most important feature of our server monitoring iPhone and Android apps is the ability to receive alerts via push notification directly to your device.

On the iPhone, notifications are easy to implement and included since iPhone OS 3 but on Android it’s a little more complex, and the Google provided “Android Cloud to Device Messaging” is only built into the OS as of Android 2.2. That said, it’s not just designed to send a simple message as it can include any kind of payload and used for any kind of updates, not just text alerts:

Android Cloud to Device Messaging (C2DM) is a service that helps developers send data from servers to their applications on Android devices. The service provides a simple, lightweight mechanism that servers can use to tell mobile applications to contact the server directly, to fetch updated application or user data. The C2DM service handles all aspects of queueing of messages and delivery to the target application running on the target device.

But since we wanted to support Android 1.6+ (1.6 and 2.1 making up 56.8% of deployed versions) then we needed an alternative to C2DM.

C2DM Alternatives

A blog post on tokudu.com was amongst the top results in Google and provides a detailed walkthrough of the various options.

  • Polling. The application itself would periodically poll your servers to check for new messages. You would need to implement everything from queing messages to writing the polling code. Alerts are no good if they’re delayed due to a low polling period but the more frequently you poll, the more the battery is going to die.
  • SMS. Android can intercept SMS messages and you could include a payload to tell the app what to do. But then why not just use SMS in the first place?
  • Persistent connection. This would solve the problem of periodic polling but would destroy the battery life. Apple’s push notifications work well because there’s only ever 1 service connecting to Apple’s servers. This is also how the C2DM service works and means you don’t have every app leaving open connections.

A solution is proposed based on the persistent connection idea:

mqtt is a publish/subscribe messaging protocol intended that is designed to be lightweight. It is useful for use with low power sensors, but is applicable to many scenarios…Did you see the part about ‘low power’? So did I. Basically, the reason why one might consider using MQTT is that it was designed to be very lightweight, so that it doesn’t consume much power. This is ideal for a mobile push solution as it addresses many battery life related concerns about persistent TCP/IP connections. Obviously, MQTT also has some disadvantages such as privacy, but we can talk about that later. So, my idea consists of taking a KeepAliveService and replacing the raw TCP/IP connection with an MQTT connection.

This is a fairly nice solution, and Java sample code is provided, along with the PHP server side code. Yet it still suffers from a few key problems which stopped us from using it:

mqtt Diagram

  • Your app still has to maintain an open connection and that uses up a lot of battery if other apps are doing the same.
  • It requires the app to maintain connections to your servers on a custom port with a custom server. This makes it difficult to load balance and scale. It also requires opening a custom port and handling that kind of traffic. Our service currently operates entirely over HTTP/HTTPS and we like it that way. Communication with custom services like Apple is outgoing only.
  • The protocol is open by default. This means “anyone who knows the IP and the PORT at which the broker is running can connect and intercept your Push messages”, unless you write your own authentication.

There is an open source project being developed to create a proper client/server solution so you can run your own push service. But with that service not yet finished and considering these points, we decided to go for a different solution.

Urban Airship Push for Android

Urban Airship Logo

This is a commercial service that works similarly to Apple’s Push Notification service. You include their library in your app and send messages to their web service, which then handles delivering the messages to user devices. The big disadvantage is that it requires the user install the AirMail app onto their device. This app maintains a connection to their servers but only requires you install it once for any app that uses the service. Therefore only 1 connection to the Urban Airship servers is needed, thus saving battery life.

We spend a lot of time on the user experience for our products and we had long discussions about whether we wanted to force the user to install a 3rd party app. Their app has additional features like setting quiet periods and on balance, we ultimately decided that it was worth it to offer push notifications.

But we’re working on using C2DM. Android 2.2 is gaining in popularity (33.4%) and we want to support it in the future.

Working with C2DM

Google provides detailed documentation and sample projects about how to implement C2DM into your Android Java project. They also have a good mailing list with Google employees who actually reply in just a few hours. However, like Apple their documentation is lacking on how to implement the server side aspects – the bit that actually sends the message from you to Google, and then onto the user.

The complex part of Apple’s service is setting up the SSL certificates. This isn’t necessary for Google but they do require a ClientLogin token, which is equally complex:

ClientLogin Diagram

With programmatic authorization implemented, your users can log into their Google account and grant access to their Google service data from inside your application. The application then contacts Google with the login data and requests access to a specified Google service. Once Google authorizes access, your application can access the Google service data, allowing the user to create, read, update, or delete service data as needed using your application interface.

Authorization with the ClientLogin API is a step up from the “low-tech” approach, which required applications to include the user’s login name and password in every request to a Google service. With programmatic login, Google issues a token that can be referenced in all subsequent requests.

Once you have that token you can then communicate with the C2DM service. How this might work in practice with a real app is as follows.

  1. The user launches your app on their Android device. You call the C2DM registration method.
  2. After the user has logged in, Google provides a registration ID to uniquely identify the device. You must send that ID to your servers and store it so you can communicate with the user.
  3. You send a message to the user using their registration ID. This consists of a POST to the Google C2DM service.
  4. The user’s device receives the message and you handle it.

Step 3 is the server side code that we’re going to deal with here. The other steps are all implemented in Java in the app itself, and are outside the scope of this tutorial.

Sending messages to Google C2DM – ClientLogin authentication

You first need to register to use C2DM. It’s recommended that you set up a separate Google account just for this. If you use a Google Account that is both a Google Account and a Hosted Google Account (through Google Apps) then you may have a conflict. In this case, the Hosted account will always get registered (source).

You can also try to request “HOSTED” instead of “HOSTED_OR_GOOGLE” in ClientLogin – the hosted account is enabled for C2DM if there are both a hosted and a consumer ( google ) account. If your domain is converted to use the new ‘consumer’ account style – the problem may go away as well, since the old consumer account will be renamed.

You then need to obtain the ClientLogin token for the account you registered with C2DM. The process is well documented and there are plenty of things you need to handle (like CAPTCHA and expiring tokens) but this Python sample code will get a token for you:

import urllib, urllib2

class ClientLoginTokenFactory(): 
	_token = None 
	
	def __init__(self):
		self.url = 'https://www.google.com/accounts/ClientLogin'
		self.accountType = 'GOOGLE'
		self.email = 'example@gmail.com'
		self.password = 'password'
		self.source = 'Domokun'
		self.service = 'ac2dm'
	
	def getToken(self):	
		if self._token is None:
			
			# Build payload
			values = {'accountType' : self.accountType,
					  'Email' : self.email,
					  'Passwd' : self.password, 
					  'source' : self.source, 
					  'service' : self.service}
			
			# Build request
			data = urllib.urlencode(values)
			request = urllib2.Request(self.url, data)
			
			# Post
			response = urllib2.urlopen(request)
			responseAsString = response.read()
			
			# Format response
			responseAsList = responseAsString.split('n')
			
			self._token = responseAsList[2].split('=')[1]
			
		return self._token

(Code taken and modified from sample posted here).

You need to replace self.email, self.password and self.source with your own credentials. The email and password are for the account you signed up to C2DM with and source can be anything identifying who you are to Google – see here for details. If you’re using a hosted Google account then you need to change self.accountType too – see here for details.

To execute this code you’d use:

clientAuthFactory = ClientLoginTokenFactory()
clientAuth = clientAuthFactory.getToken()

Sending messages to Google C2DM – cd2m/send

The payload you send to Google has a number of requirements, as detailed in their documentation. You need to specify the registration ID the user’s device has sent you and also provide a collapse key:

An arbitrary string that is used to collapse a group of like messages when the device is offline, so that only the last message gets sent to the client. This is intended to avoid sending too many messages to the phone when it comes back online. Note that since there is no guarantee of the order in which messages get sent, the “last” message may not actually be the last message sent by the application server.

You’ll also notice that there is no required “message” field. This is because the payload can contain many “data” fields and it is up to your app to handle them. This might be just a single field called data.message that is displayed by your app, or you could use it to send any other kind of data just to update the app. For example we could use it to send the latest data from a server as soon as we receive it to automatically refresh the values in the app.

This sample Python code will build the request, get your ClientAuth token (using the class posted above) and then send it.

class C2DM():
	
	def __init__(self):
		self.url = 'https://android.apis.google.com/c2dm/send'
		self.clientAuth = None
		self.registrationId = None
		self.collapseKey = None
		self.data = {}
		
	def sendMessage(self):
		if self.registrationId == None or self.collapseKey == None:
			return False
		
		clientAuthFactory = ClientLoginTokenFactory()
		self.clientAuth = clientAuthFactory.getToken()
		
		# Loop over any data we want to send
		for k, v in self.data.iteritems():			
			self.data['data.' + k] = v
			
		# Build payload
		values = {'registration_id' : self.registrationId,
				  'collapse_key' : self.collapseKey}		
		
		# Build request
		headers = {'Authorization': 'GoogleLogin auth=' + self.clientAuth}		
		data = urllib.urlencode(values)
		request = urllib2.Request(self.url, data, headers)
		
		# Post
		try:
			response = urllib2.urlopen(request)
			responseAsString = response.read()
			
			return responseAsString
		except urllib2.HTTPError, e:
			print 'HTTPError ' + str(e)

(Note this code is designed to be used in the same file as the previous sample code. If you separate them it requires import urllib, urllib2)

To use this code you need to call it like so:

sender = C2DM()
sender.registrationId = 'hats'
sender.collapseKey = 1
sender.data = {'message' : 'Hello there', 'hats' : 54}
response = sender.sendMessage()

sender.data takes a Python dictionary, which will be converted to the right format to send to C2DM. Note that the sender.registrationId is not set – it should be the registration ID for the user you want to send to.

What else you need to do

This sample code is useful for development but doesn’t do some things you’ll need for production:

  • Error handling – you’ll need to do more advanced handling of any HTTP errors. If the Google service is down it’ll return a 503 HTTP code and a retry value, which you must honour.
  • You must queue messages if the service returns a quota error. This needs to use Exponential backoff.
  • The ClientLogin code does not handle CAPTCHA or resetting the auth token. See this post for how this might be presented.

Final words

Push notifications on Android are fairly fragmented right now. The Urban Airship solution we use on our server monitoring apps works well but we do want to use C2DM in the future so we can avoid requiring users to install a separate, 3rd party app.

  • http://blog.automated.it ScaredyCat

    Nice article…

    For those that might just want to send the odd message to their device try*:

    Auth

    curl https://www.google.com/accounts/ClientLogin -d Email=$1 -d "Passwd=$2" -d accountType=GOOGLE -d source=My-Server-Event-Alerter -d service=ac2dm

    Send:

    curl --header "Authorization: GoogleLogin auth=$1" "https://android.apis.google.com/c2dm/send" -d registration_id=$2 -d "data.payload=$3" -d collapse_key=myappalert09

    it works really well and can be converted to use php libcurl (or python equivalent) very quickly to deal with 503′s etc.

    I guess if you’re throwing lots and lots of messages at different devices you need to do it a bit better than my 1 liners, as you have, but it’s still nice to be able to spit out your own messages and have your phone respond.

    *obviously you need to have a suitable app on your device.

    SC

    • http://odanchuk.blogspot.com oleksandr

      all this is nice. I just need to clarify about google accounts
      what account are you using when you autorize

      >>curl https://www.google.com/accounts/ClientLogin -d Email=$1 -d “Passwd=$2″ -d >>accountType=GOOGLE -d source=My-Server-Event-Alerter -d service=ac2dm

      I assume that this is a different google account (not the same as on device side and not c2dm account.. Am I right?

      • http://www.serverdensity.com David Mytton

        You authorise using your c2dm Google Account.

      • http://odanchuk.blogspot.com oleksandr

        Thanks a lot for the reply. It is really useful
        I will be so pleased, If you could answer one more question..
        I have send a POST request from my server to c2dm server and receive
        id0 in response (looks like a successful send)
        but on device side nothing happen I set up a break point in my c2dm broadcast receiver with no results. I cannot see my messages on device.

        could you suggest what to check in this case?

  • http://andypiper.co.uk Andy Piper

    Hi David, nice post! I just wanted to clarify something though:

    The protocol is open by default. This means “anyone who knows the IP and the PORT at which the broker is running can connect and intercept your Push messages”, unless you write your own authentication.

    Actually, version 3.1 of the MQTT protocol (supported in RSMB and documented in the specification which is openly available, royalty-free, on the IBM developerWorks site), supports username/password on connect. I’m not going to claim that this is necessarily “strong” authentication but it would enable you to apply a modicum of security to the solution you suggest.

    (note: I work for IBM, and I just wanted to clear this point up – hope this helps)

  • http://www.josht.com/ JT

    Is there any reason you didn’t go ahead and use Google’s service if the device is 2.2? The reason this is preferred is it uses the already established connection the device has open for Gmail, Google contacts, etc and doesn’t have to open and keep another connection, each connection that has to stay open means less battery life for the end-user. Since 2.2 is now 1/3 of the market, and possibly an even larger share of the kind of user’s who would be using a service like Server Density, it seems like it’d make sense to only fall back if needed.

    • http://www.serverdensity.com David Mytton

      We decided not to use C2DM for the initial release it because it would add extra time to development and 2.2 isn’t as widely used as other versions yet. We do plan to adopt it for a future release though because of benefits you mentioned.

      • Dustin

        What push-notification approach did you use? I’ve looked at xtify and urban airship and am trying to weigh the option of building our own infrastructure (that scales) or just biting the bullet and using one of these services.

        • http://www.serverdensity.com David Mytton

          We used Urban Airship.

  • http://www.codepress.com Anthony Webb

    I wonder if perhaps the urban airship guys will start using C2DM themselves for message delivery in the future. I love their API, nice and clean. But cant require a separate download and background process for my users.

  • http://www.deaconproject.org/ Dave

    There is an open source project being developed to create a proper client/server solution so you can run your own push service. But with that service not yet finished and considering these points, we decided to go for a different solution.

    Just a quick update – The Deacon Project moved to Beta status this past weekend.

  • John Monroe

    I’m trying to convert your Python into Java, and I’m having problems right at the start with the ClientLogin. My code is returning a 403 / Forbidden when run. I’ve received access for my application to C2DM via the signup page (http://code.google.com/android/c2dm/) and an email confirmation, so I should have access. Here is my code if anyone has any ideas, and thanks in advance:

    log.info(“Obtaining the Google C2DM Client Login token.”);

    // Make POST request
    HttpResponse res = null;
    try {
    DefaultHttpClient client = new DefaultHttpClient();
    URI uri = new URI(“https://www.google.com/accounts/ClientLogin”);

    List params = new ArrayList();
    params.add(new BasicNameValuePair(“accountType”, “HOSTED_OR_GOOGLE”));
    params.add(new BasicNameValuePair(“Email”, “MY_ACCOUNT@gmail.com”));
    params.add(new BasicNameValuePair(“Password”, “MY_PASSWD”));
    params.add(new BasicNameValuePair(“service”, “ac2dm”));
    params.add(new BasicNameValuePair(“source”, “ACCOUNT_NAME-V0.1″));

    HttpPost post = new HttpPost(uri);
    UrlEncodedFormEntity entity = new UrlEncodedFormEntity(params, “UTF-8″);
    post.setEntity(entity);
    res = client.execute(post);
    } catch (Exception e) {
    log.error(“Error obtaining the Google C2DM Client Login token.”, e);
    }

    log.debug(“response=”+res);
    if (res != null) {
    log.debug(“Response status code = “+res.getStatusLine().getStatusCode());
    log.debug(“Response status = “+res.getStatusLine().getReasonPhrase());
    }

    • Dannon D’Mello

      Change the password param to Passwd. Works for me
      new BasicNameValuePair(“Passwd”, “Your_Password”));

      • John Monroe

        Good catch. Works like a charm now.
        Thank you very much.

  • Suresh Nariya

    Hi David, Nice Post!!!

    I have many android apps on android market, and i am thinking to implement Push solution for my applications, I am exactly not sure which solution i should implement, yourExperts commentsinput will be help me to take better decision.
    1. Use any third party Push Service like Urban AirshipXtify(Not sure which one is better)
    2. Go with Android’s C2DM Service.
    3. Go with own Messaging solution using XMPP.

    Thanks,
    Suresh Nariya.

    • http://www.serverdensity.com David Mytton

      Really depends on which Android versions you want to support.

    • jonny

      Hi,
      I’ve been trying the XMPP method with the asmack library – polls rather than pushes hence is a battery drain. There are some intents the look at GTALK connections but I haven’t managed to find any info on them.

  • http://blog.lytsing.org liqing huang

    hi,
    17 # Loop over any data we want to send
    18 for k, v in self.data.iteritems():
    19 self.data['data.' + k] = v

    this code has problem: RuntimeError: dictionary changed size during iteration .

    and sender.data = {‘message’ : ‘Hello there’, ‘hats’ : 54}
    the data is not seed, it should be input into the values.

  • jonny

    not being of the OO ilk I’ve simplified the code somewhat. outMessage is the data you want to communicate. I’ve had this working from a GAE web app.

    clientAuthFactory = ClientLoginTokenFactory()
    request = urllib2.Request( url = ‘https://android.apis.google.com/c2dm/send’,
    data = urllib.urlencode({‘registration_id’ : l.clientAuth, ‘collapse_key’ : 1, ‘data.payload’ : outMessage)),
    headers = {‘Authorization’: ‘GoogleLogin auth=’ + clientAuthFactory.getToken()})
    try:
    response = urllib2.urlopen(request)
    responseAsString = response.read()
    print response
    except urllib2.HTTPError, e:
    print ‘HTTPError ‘ + str(e))

  • http://haploid.fr Guillaume

    hi,
    I think you forgot to add your data when you are building the playload. Maybe you should do something like this :

    # Build payload
    values = {‘registration_id’ : self.registrationId,
    ‘collapse_key’ : self.collapseKey}
    # adding data
    for k, v in self.data.iteritems():
    values['data.' + k] = v

    no?

    • http://twitter.com/ovidiup ovidiup

      I agree, the sample code for sending a message is messed up. I modified the code exactly as you suggest.

  • androindian

    can anyone zip me a complete sample @ arfaoui88@gmail.com

  • Ahmad Shaman

    my app needs to be notified every minute but also sometimes after every 10 to 30 to 1 hr. C2DM is not suited for my App as it will black list the server for notifying frequently, did MQTT solution will support me for this problem?