Category Archives: Tech

Spring-Session with a relational database(Part 2)

by

Preface:

I recently had to work on a project where we had to turn an existing Java (Springboot) application whom was served by multiple web servers sitting behind a load balancer and required sticky sessions to an application that could be deployed to an environment using non-sticky sessions.

I wrote a blog article about how we setup our proof of concept using Spring-Session with our application here.  We initially wrote our tutorial using Redis as our session store, but it was decided to use a relational DB instead(Oracle).  Documentation was pretty sparse about how to exactly configure Spring-Session to use a different session store so I decided to write up a little blog post outlining what we did.

Spring-Session

Spring-Session provides a container portable method of persisting Http session data.   I won’t go into great detail on how to set it up in this post,  but full instructions to setup Spring-session can be found on their official website here.   For us,  we are going to configure Spring-session to use our own custom session store back by a MySQL database!  We do this by:

  1. Implementing our own SessionRepository and
  2. Injecting the custom SessionRepository to SessionRepositoryFilter

 

Create the SessionRepository

We first have to implement the SessionRepository interface.   It has four methods and they are all pretty self explanatory on what they do.   It’s worthwhile noting that the Session type here is not the container provided javax.servlet.http.HttpSession, but a org.springframework.session.Session instance instead.   In the context of a web application,  it does still represent a HttpSession in that the getId() method will return a jSessionId and attributes will be its session data.

  
public interface SessionRepository<S extends Session> {
           S createSession();
           void save(S session);
           S getSession(String id);
           void delete(String id);
}

For our “MySQL” storage sample,  the repository makes use of a “SessionStoreDao”.   This is simply our DAO object responsible doing CRUD operations on a table called “SessionStore”, which will store our user session data.  I won’t provide code for the DAO (as that depends on if you are using a ORM or stright JDBC-SQL) or the table.   We also have a SessionStore  “domain object”, which is just used to pass around information in our application.   This POJO contains information such as the jsessionId, timestamp and the serialized session data.  Let’s start:

public class MysqlSessionRepository implements SessionRepository {
	   @Autowired
	    private SessionStoreDao sessionStoreDao;

	    /**
	     * @return A new instance of MapSession
	     */
	    @Override
	    public Session createSession() {
	        Session s = new MapSession();
	        return s;
	    }

createSession() simply creates a new MapSession instance.  Spring-session will invoke this method whenever it needs to create a brand new session, for example, the first time the application invokes request.getSession().   You can do any additional initialization steps here, but in our simple example we simply return a empty map.

  


	    /**
	     * Update or create a new SessionStore to save the session to the DB.
	     * @param session This will be serialized and stored in the DB.
	     */
	    @Override
	    public void save(Session session) {
	        String id = session.getId();
	        Timestamp currentTime = new Timestamp(System.currentTimeMillis());
	        SessionStore sessionStore = sessionStoreDao.findOneBySessionId(id);
	        if (session != null && sessionStore == null ) {
	            sessionStore = new SessionStore();
	            sessionStore.setSessionId(session.getId());
	            sessionStore.setCreatedDate(currentTime);
	        }

	        if (session != null){
	        	// searlize the object
	            byte[] bytes = Util.objectToByteArray(session);
	            sessionStore.setData(bytes);
	            sessionStore.setLastModified(currentTime);
	        }

	        // save
	        sessionStoreDao.save(sessionStore);
	    }

The save() method is where you will have to do any work required to persist your session data to the database.   You have to first check if the user session already exists in the database.  For web applications, a natural key for the session record (in the database) is the jsessionId.     In our example, we first try to obtain the existing data (domain object) from the database.   If it exists we then  serialize the data contained in the provided in the Session object.   The session data can be represented in any way you want it in the database, but for simplicity sake, we just serialized the entire Session object.  This of course assumes everything stored in the session serializable.  Here, we also save additional information along with the persisted session (such as last modified date).

The SessionStoreDao.save() method simply takes in our domain object and persists it to the MySQL database.  Of course, if this is the first time the session is being persisted to the DB, we’ll have to create a brand new row to save the user data (hence the null check).

  

	    /**
	     * Load the seralizied session from the DB.
	     * @param id The session id (jsessionId)
	     * @return the Session instance correspoinding to the jsessionid
	     */
	    @Override
	    public Session getSession(String id) {
	        SessionStore sessionStore = sessionStoreDao.findOneBySessionId(id);
	        if (sessionStore == null ) {
	            return null;
	        }
	        byte[] blobData = sessionStore.getData();
	        MapSession s = (MapSession)Util.byteArrayToObject(blobData);
	        return s;
	    }

getSession() is responsible for deserializing/re-creating a the org.springframework.session.Session if it exists in the database.  Our SessionStoreDao is used to query the database based on the “session id” provided by Spring-Session.  In the context of a web application, the id parameter will always be the jsessionId.

In our save() method example above, we simply serialized the entire Session object and stored it into the database so to recreate it,  we simply deserialize it back into the Session object.  If session is brand new, there should be no such record in the database yet.  For brand new user sessions, Spring-session will determine on your behalf when to call load() and create() .  To signify to Spring-session that the user session is new (or not found) and a brand new session needs to be created, getSession() should return null.

 

  

	    /**
	     * Handles requests to invalidate the session (ie logout).
	     * Delete the session from the DB.
	     * @param id The session id (jsessionId)
	     */
	    @Override
	    public void delete(String id) {
	        SessionStore sessionStore = sessionStoreDao.findOneBySessionId(id);
	        if (sessionStore != null ) {
	            sessionStoreDao.delete(sessionStore);
	        }
	    }
}

delete()  The final method that you need to implement is the delete() method.  A example of when Spring-session will invoke this method is if you invalidate the HttpSession.  In our simple example, we simply have to delete the SessionStore record from the MySQL database.

Inject it into SessionRepositoryFilter

After creating our MysqlSessionRepository implementation, we need to inject into SessionRepositoryFilter so that it will be used by Spring-session.  There many ways to do this, I will present one method here.

@Configuration
public class MysqlSessionStoreConfiguration {	

    @Bean
    public MysqlSessionRepository sessionRepository() {
        return new MysqlSessionRepository();
    }
    
    @Bean
    public <S extends ExpiringSession> SessionRepositoryFilter<? extends ExpiringSession> springSessionRepositoryFilter(SessionRepository<S> sessionRepository) {
        SessionRepositoryFilter<S> sessionRepositoryFilter = new SessionRepositoryFilter<S>(sessionRepository);     
        return sessionRepositoryFilter;
    }
}

The code is a bear-bones implementation of a configuration class that creates a MySqlSessionRepository bean and injects into a SpringRepsoitoryFilter.  Note the @Configuration annotation indicating this is a configuration class.

And that is it.  We’ve created a MysqlSessionRepository that implements SessionRepository.  This custom repository stores user session data into a MySQL database.  As your application handles http requests while storing data into the HttpSession object, Spring-session will invoke create/save/delete/load methods as required which will save the user session data to the MySQL database without the application knowing.  The changes on application developers is very minimal as Spring-session provides an almost seamless layer to route the session data anywhere.  Developers can more or less continue to use the application HttpSession as normal, but we do make an assumption here that the data stored in the session is serializable.

 

Non-sticky sessions with Spring-Session (Part 1)

by

Preface:

I recently had to work on a project where we had to turn an existing Java (Springboot) application whom was served by multiple web servers sitting behind a load balancer and required sticky sessions to an application that could be deployed to an environment using non-sticky sessions.

The application was not stateless and was using the session to store data between requests.  After doing some analysis, changing the application to become stateless was not an option for us.  So we needed to save the session data somewhere, on disk, using Redis or a database.

Spring-Session

Spring-Session provided an API layer to persist session data without having to significantly change application code.  It basically provided an HttpSession implementation that saved data to disk and allowed application developers to continue freely using the Session object as before (as long as the data was serializable).

I won’t go into huge detail on how Spring-Session works, but it basically replaces the container provided HttpSession object with its own (via a Filter) and at some point, Spring-session will persist the data to someplace (as configured by us) without the application itself needing to know what is going on behind the scenes.

When the user’s request ends, the session data is persisted and when that same user makes another request, as expected, Spring-session will fetch the user’s session data and desearialize it.  The data will then be available using the HttpSession’s getParameter() methods as a conventional HttpSession object.

Proof of Concept and Session Store
Spring-Session comes with support using Redis as the session store (using Jedis).   So our first step in evaluating Spring-Session is to get a our application integrated with Spring-session,  setup a Redis and Redis backup (Redis slave) servers. This way we can determine if Redis is a good choice for our session store and if not, can we still use Spring-session with another session store (such as a relational DB).

One can argue if the redundancy of session data is actually required here.  That is, the data we are using Redis to store is only session data as the user navigates though our application and not actual user data that requires persistence.   I’ll leave that as a argument for a later time, but for our sake, we needed to see some redundancy features of Redis as a matter of “policy”.

Our PoC arcitecture looked like the following:

Setting Up With Maven
Spring-Session with Redis requires 2 dependencies,  i) the Spring-Session dependency and   ii) the Redis connector.   If you are are using a custom session store, you do not need to include the Spring-Session data-redis dependency.

  
      <dependency>
         <groupId>org.springframework.session</groupId>
         <artifactId>spring-session</artifactId>
         <version>1.0.2.RELEASE</version>
      </dependency>
      <dependency>
         <groupId>org.springframework.session</groupId>
         <artifactId>spring-session-data-redis</artifactId>
         <version>1.0.2.RELEASE</version>
         <type>pom</type>
      </dependency>

Springboot
If you are using Springboot, you have to create the JedisConnectionFactory bean that will be injected into the SessionRepositoryFilter.  Many of the configuration properties of a JedisConnectionFactory have default values, so the example below is pretty sparse.  I only specify the URL of my Redis master  instance, but it is in here where you can specify other properties such as password, etc.

  
    @Bean
    public JedisConnectionFactory connectionFactory() {
    	JedisConnectionFactory redisConnectionFactory= new JedisConnectionFactory();
        redisConnectionFactory.setHostName("redis.masterserver.com");
        redisConnectionFactory.afterPropertiesSet();
        return redisConnectionFactory;
    }

Testing the Setup
To test our Poc, we wanted to see that 1) our setup is truly session-sticky-less and 2) Redis data redundancy was working properly.

1) Session Stickiness
To test session non-stickiness, we send a HTTP request from a computer though the load balancer.   We find out it is being handled by application server A and the application puts some data into the session.  We then shut down server A and send another request from the same browser.  The request is now handled by server B, but because we’ve externally stored the session data, the user’s session (and data) are still alive.

2) Redis Redundancy
To test Redis redundancy, we sent a HTTP request to our app though the load balancer, but we don’t care which server handles the request.  We use the Redis client (redis-cli) check that the session data being stored in the master Redis instance.  We then connected the client to the slave instance and check that the same data is also in the slave.

 

Conclusion and More Work
It turned out that integrating our application with Spring-Session was very straight forward. Our PoC Redis and replication process was also easy to get going. A architectural decision was made on not to use Redis as the session store but we determined that we still wanted to use Spring-Session with a relational database as our storage. This leads us to part 2 of our Spring-Session introduction, using Spring-Session with a relational DB (Oracle or MySql).

References and other tutorials
If you are interested in more details on how to setup Spring-Session with Redis, you can look at these sites below. They helped us a lot.

  1. Spring-Session Project Page
  2. Video: Spring-Session by Josh Long
  3. Basic Tutorial
  4. Spring-Data: Jedis
  5. Managing Users Data with Spring Session
  6. Another Redis Replication Tutorial

 

Obtaining Facebook page access tokens: The 4 step program

by


More and more businesses are creating Facebook pages to promote their products and services.    To keep content fresh, many websites and applications are automatically posting content to their pages.     They allow an application to post content on behalf of the user.   But how do these applications post content on a page, as the page?   That’s where page access tokens come into play.   I wrote a detailed article for The Jonah Group on how to do this.   You can view it on their blog here.

Grails Tip # 6: Service layer transitions

by

On my last Grails project, as good Grails developers, we faithfully put core application logic inside various service methods. Delightfully, the benefits of doing so paid off and we began re-using services though out the application.

One day, while using our application on a fairly complex set of test data (both in volume and the fact our domain model graph is complex), we stumbled on a huge performance problem. After profiling the controller-action, the problem was traced down to a call to a service method. Simplified, it look like:

class ArticlesController {
   def service

   def handleAction = {
      // stuff ...
      // more stuff ...
      for(int i=0;i < loopLimit;i++){
         service.serviceMethod(someParameters)
      }
      render(view: "article"...)
   }
}

This was a commonly re-used service method, but the interesting part of this method was the fact it was being called within a loop, a loop that could potentially be called several hundred times. The looping itself is not a problem (nor could it be avoided).

By trial and error, we discovered that the actual call to the serviceMethod() was causing the performance issue. If we
1) Simply removed the service method and embedded the application logic in the service method into the loop or
2) Made the service class non-transactional

The performance issue disappeared.  We realized that the under-the-hood work that goes on to setup the transactional service boundry, when there is a complex or large volume of data in the Hibernate session, causes a dramatic perfomrance impact when we repeadly cross the controller- service layer boundry.

The simplest and cleanest solution was to loop inside the service method, thus the application cross the boundary only once. This has the side effect that any errors will roll-back the entire set of data and not just the single record being updated but we were ok with this.

class ArticleService {
   static transactional = true
   def serviceMethod(def params){
      // do stuff..
   }

   def serviceMethodMultiple(def params){
      for(int i=0; i < loopLimit; i++){
         serviceMethod(params)
      }
   }
}

And in the controller:

class ArticlesController {
   def service

   def handleAction = {
      // stuff ...
      // more stuff ...
      service.serviceMethodMultiple(someParameters)
      render(view: "article"...)
   }
}

From the standpoint of a intermediate Grails developer, this isn’t a problem that could have been detected beforehand since it only manifests itself when there is a complex domain model graph already in the session.

We chalked this up as a life lesson. The final solution, avoid any code that may repeatedly crosses into the service layer, especially for services marked as transactional. Instead, create a new method that does the looping inside the service boundary. Identification and avoidance was the best action forward as we were in no position to fix any potential underlying issue with the framework (if there even was any).

Just Add Shoyu

by




Being Japanese (Canadian), people often ask me what I eat at home and if it is similar to those served at Japanese restaurants. I always have to explain, that Japanese food at a restaurant is quite different then the food prepared at home. For the record, very few Japanese people make their own sushi at home. It is just too time consuming and expensive to gather and prepare all the ingredients (analogous to homemade Dim Sum).

I would say “Japanese Canadian” food is a mixture between traditional “Japanese food”, Chinese, and western foods. Many of the food we eat could be technically be classified as Korean or Chinese, but has become such a staple in Japanese Canadian households that we do consider it part of our culture as well.

In addition, much of the food recipes have been modified to use western ingredients. Take for example, Spam sushi and Japadogs sandwiches.

The Japanese Canadian Cultural Center(JCCC) recently published a cookbook, a project that focuses specifically on Japanese Canadian food. Not only will you find traditional sounding recipes, but also foods from other cultures have become a part of the foods Japanese Canadians eat.

The book was produced using recipes submitted by and produced by countless volunteers within the Japanese community. For example, my aunt was the project manager, my cousin was a principle photographer, and I made their website. This book is more than a regular cookbook as it also describes the personal stores behind the recipes.

The book is now out; you can find it at Chapters and other cookbook stores or directly though the JCCC. The book looks awesome and has some great recipes (including one for my grandfather’s roasted chicken).

Click here to view the official Just Add Shoyu website: Just Add Shoyu

Various reviews of the cookbook:

Grails Tip # 5: Apache POI

by

Apache POI with Grails

Many people have asked how to use Apache POI with Grails. A quick search on the Internet
reviews bits and pieces but no official documentation or tutorial. So I decided two quick ways of incorporating POI
with your Grails applications.

In a nutshell, POI provides a Java API for reading and writing MS documents such as Word and Excel documents.
You can find the library and full details here.

Write directly out to the output stream

The first method involves obtaining a handle to the response output stream and writing directly to it. You must
remember to set the content response header and content type appropriately or the user will see garbage. 

        package com.ex
        import javax.servlet.ServletOutputStream
        import org.apache.poi.hssf.usermodel.*

        class ShelfController {

            def index = {

                HSSFWorkbook wb = generateWorkbook();
                response.setHeader("Content-disposition", "attachment; filename=serverList.xls");
                response.setContentType("application/excel");

                // Write the output to the OutputStream
                ServletOutputStream f = response.getOutputStream();
                wb.write(f);
                f.close();
            }

            private HSSFWorkbook generateWorkbook() {
                HSSFWorkbook wb = new HSSFWorkbook();

                HSSFSheet sheet = wb.createSheet("new sheet");

                // populate the spreadsheet
                HSSFRow row = sheet.createRow((short)0);
                    .
                    .
                    .
                return wb
            }

    }

Use AbstractExcelView

Because Grails is built on Spring and your Grails controllers can return Spring’s ModelAndView objects, you can
extend Spring’s AbstractExcelView and implement its buildExcelDocument method. Your Grails controller will simply
return this Excel view as the one to render. It will set the header and content type for you. 

        package com.ex
        import javax.servlet.ServletOutputStream
        import org.apache.poi.hssf.usermodel.*

        class ShelfController {

            def index = {
                // forward to the list view to show them
                return new ModelAndView(new SheflExcelView(), [ bookList : "abc" ])
            }


        }

Implement the buildExcelDocument method in your Excel view: 

        package com.ex

        import javax.servlet.http.HttpServletRequest
        import javax.servlet.http.HttpServletResponse
        import org.apache.poi.hssf.usermodel.*
        import org.springframework.web.servlet.view.document.AbstractExcelView

        class SheflExcelView extends AbstractExcelView{

            protected void buildExcelDocument(Map<String, Object> model,
                HSSFWorkbook workbook, HttpServletRequest request,
                HttpServletResponse response) throws Exception {

                HSSFWorkbook wb = generateWorkbook(workbook);
            }

            private HSSFWorkbook generateWorkbook(HSSFWorkbook wb) {
                HSSFSheet sheet = wb.createSheet("new sheet");
                .
                .
                .
            }
       }

I personally prepare the second method. The Excel document is a “view” so I don’t want to pollute the controller
with a whole lot of code generate the Excel report. In addition, AbstractExcelView also sets your response and content
type headers on your behalf.

Grails Tip # 4: java.lang.Boolean.call()

by

No signature of method: java.lang.Boolean.call()

As specified by , Grails provides convenience method to check if a object contains errors and set the CSS class on the element.

I came across this problem where suddenly, under some conditions, I was seeing the following error:

No signature of method: java.lang.Boolean.call()

Further down the stacktrace, I notice something more interesting.

Caused by: groovy.lang.MissingMethodException: No signature of method: java.lang.Boolean.call() is applicable for argument types: (java.util.LinkedHashMap, java.lang.String) values: [[bean:{}, field:asc], input-error]Possible solutions: wait(), any(), dump(), wait(long), wait(long, int), and(java.lang.Boolean)

At first glance, the stacktrace looks like complicated garbage. However, after a few minutes of looking though the stacktrace, the parameter’s in the exception cause looked suspiciously like those specified in a invocation of the GSP’s hasErrors method.

I soon realized that this exception only occurs during situations where errors would be displayed on the GSP. Looking though the code flow, I soon realized something suspicious. 

		def save = {
			.
			.
			.
			
			command.validate()
			if (command.hasErrors()){
				render (view:input, model:[command:command, hasErrors:true])
				return
			}
		}

Notice that the if the command object has errors after validation, the model being returned to generate the GSP contains both the command and a variable hasError which is assigned to true.

I ran a quit test, what happens if try and execute the value of a boolean as a method. 

		true()

And volila, a No signature of method: java.lang.Boolean.call() error. It appears, when there is a variable in the model with the same name as a GSP method, under some conditions, Grails replaces the method invocation with the value of the variable and then it executes the method.

In this example, it appears that hasErrors(…) was being replaced by the value of the hasError variable, true, and then being executed. Thus the final statement looked like the following: true(bean:command, …) This of course, would result in a no method signature found error.

After changing the name of the hasError variable to something else, say, showErrors, the GSP rendered as expected. By reading this article, I hope that you can save yourself a couple of hours of struggle.

Grails Tip # 3: refresh()

by

refresh() is not an undo()

In your application, you may come across a time when you need to revert or undo
changes done on a domain object. discard() comes to mind, but there may be instances where
that is not an option. refresh() also looks like an tantalizing option, but there is one critical
feature of refresh() you need to be aware:

If you have a trasient object associated to the domain object you are calling
refresh() on, you will get a
org.hibernate.AssertionFailure: null identifier
error.

Consider the following example to see why you might need to use refresh
and what you have to watch out for.

The Domain

Consider the following domain, a Car has several models
(for each year) and most of the details of a “car” is
in the model domain. There are a few description fields on
the Car domain object however.

The Code

The Car domain object:

		package com.ex
			
		class Car {
			String name
			String description
			static hasMany = [models:Model]
		}

The Model domain object:

		package com.ex
			
		class Model {
			String name
			int year
			Engine engine
			. (other properties)
			.
			.
			static belongsTo = [car:Car]
		}

Users are able to modify car details and model details at the same time.
They can also add new models to the car as well. All changes to the
Car (and it’s models) must be atomic. That is, all changes
must be committed as one transaction. Lets take a
look of the UI:

<%= image_tag "articles/grailstip3/flow.png" %>

When the user decides to change their changes,
they exectue the saveChanges action on CarController.

Because of the belongsTo relationship on Models to Cars, we
can save any user changes to Car model and it’s models by
calling save() on the Car object. 

		package com.ex
			
		class CarController {
			def saveChanges = {
				.
				.
				.
				// save car under edit
				car.save(failOnError:true)
				.
				.
				.
			}
		}

The Problem

Due to a requirement, the users have the ability
to revert changes on a particular Model, but
still save their changes on the Car. 

		package com.ex
			
		class CarController {
			def saveChanges = {
				.
				.
				.
				// model to revert
				if (some condition)
					// rever model
				.
				.
				.
				// save car under edit
				car.save(failOnError:true)
				.
				.
				.
			}
		}

One might think we can call discard(), but this will
not work in this situation because further down
the code, we end up calling save on Model’s parent,
the Car. Even if we discard a particular Model of a Car,
it will still be saved when Car is saved due to the
cascade behaviour brought on by belongsTo relationship.

We also are unable to change the cascade settings between
Car and Model. In this particular instance, we could
prevent cascade on save, but in other cases, we do need it.

The other other option is to call refresh() on the Model
being reverted. And we do this. 

		package com.ex
			
		class CarController {
			def saveChanges = {
				.
				.
				.
				// model to revert
				if (some condition)
					model.refresh()
				.
				.
				.
				// save car under edit
				car.save(failOnError:true)
				.
				.
				.
			}
		}

Done right? well, not quite. Under some use cases, we
discover this exception being thrown:
org.hibernate.AssertionFailure: null identifier

The error only occurs when new objects are associated to
a Model. For example, if the user adds a new Engine to
the reverted Model. What’s going on?

When you call refresh on the Model, Hibernate will traverse
though the object’s object graph and overwrite its data
with what is in the database.

If there is a transient object associated to the Model,
Hibernate will not find an id associated with this entity
and thus will throw a null identifier assertion failure.

The Solution

To be able to use the refresh() method, you must break all
associations with transient objects in the graph in the
current Hibernate session.

This can be done by calling clear() on any hasMany associations
and re-assigning any variables assigned to transient objects to null. 

		package com.ex
			
		class CarController {
			def saveChanges = {
				.
				.
				.
				// model to revert
				if (some condition){
					model.engine = null
					model.tries.clear()
					model.refresh()
				}
				.
				.
				.
				// save car under edit
				car.save(failOnError:true)
				.
				.
				.
			}
		}

Many people believe this to be hackish — a workaround
and, well, though this might be true, Hibernate’s API is being
used correctly and instances such as this, transient objects
are known and can be properly be removed from the session.

Though Grails itself does not offer an “undo” method, I
would argue the solution above deals with the problem at hand
using the API as it was designed. However, one of the reasons
I’ve posted this is to see if there are any others who have faced
similar problems have found a different way of solving it.

GO Mobile

by

GO Transit just released their official mobile app and it is long over due. Better yet, its free (much to the demise of the the 3rd party apps on iTunes).

The menu is clean and simple – the way it should be. It has basic functionality such as viewing and favoriting train/bus schedules, being notified of departures, and viewing the Union Station departure board. Filtering of the departure board was a nice touch that no other 3rd party app seems to have done yet.

<%= image_tag "articles/gotransit/screenshot1.jpg", :class=>‘image-right’ %>

The app is very basic, so I expect it to work perfectly. It does not. Sometimes, just minutes into using the app, the main menu icons stopped working. You click on the icon, the icon reacts, but nothing happens. Sometimes, the expected feature opens after a while, but other times, nothing happens and I’m required to close the app and restart it. This is totally unacceptable. …even if the app was built by students.

I’m greatly disappointed that there is no way to view GO Transit train/bus statuses from the app. For most users, being notified of delays is the primary reason they want a mobile app!

The launch of the official app pretty much kills off all the $.99 3rd-party applications out there. These apps do not have access to any GO transit API so they screen scrape the content off the GO transit webpage and repackage it into a little apps. Because of this, whenever GO transit changes their webpage styling, the apps stop working. Often, it is no longer worthwhile for these companies to push out an update so user simply have to go off looking for a substitute app. The best apps can usually push out updates quickly, but even then; their app is not working for a day or two.

This has left riders unhappy with GO transit apps. They’ve gotten used to knowing that their app, which they paid for, will eventually stop working. This means you can never fully trust what the app is saying. If the app says there are no delays, is this because there are actually no delays or because the app cannot parse the website contents? Users end up double checking the official website.

I also developed a GO Transit app, and though I did not release it to the app store, I did try it for a bit. Like others, it scrapped the data from the main GO transit website. My app’s main purpose was to communicate delays on a user’s particular train.

Though I could easily parse the entire status section and place it on my app (which many 3rd party apps do), I wanted the ability to parse delays by train-line so users could filter it. However, every few weeks, the app would stop showing delays. I suspect that the status section does not use any template, but rather, the entire status section is just free from text-field (in addition, the entire status section is surrounded by <pre> tags to further prove my suspicion). In the end, I couldn’t find a reasonable delimiter between statuses on different lines and scrapped the app.

The image to the left is a screenshot of the status section on the GO transit website. Trying to parse it by line proved difficult because no standard template was used. In this particular example, notice how the first delay effects one train-line and the 2nd delay effects two train-lines even though it is displayed in the same fashion as the one-train line status. This inconsistent use of headers is one example of the difficulties of parsing this page.

Because of this, I’m pleased that there is an app that is based on some official API. Though, I do not really expect myself to use this app too much do the lack of status updates, I think the general public will be please to have a GO Transit app they can use with confidence.