Spring integration makes it easy to monitor an sftp server for new files and inject those files into your application for processing.  I like to configure my spring integration applications with annotations and @Bean configurations.  I found the documentation, and online examples for doing this annotations a bit lacking.


This post demonstrates the basics on how to connect get a spring-boot (1.3.x or 1.4.x) application to monitor a sftp server.

Read the rest of this entry »

Sometimes the behaviour of an application is controlled through properties
and the application needs to detect changes to the property file so it can switch to the new configuration. You also want to ensure that a particular request either uses the old configuration or the new configuration but not a mixture of old and new. Think of this as ACID like isolation for properties to ensure that your requests don’t get processed using an inconsistent configuration.

We accomplish this by using a property lookup bean (PropertyBean) that returns property lookups. The PropertyLookupBean is a request scoped bean meaning that a new instance is created for each request in a spring MVC application. The PropertyLookup bean has an init method that runs post construction and calls the PropertyCache bean to get the current set of properties. Each instance of the PropertyLookup bean will return the property values from the Properties object it gets at initialization.


The property lookup bean has a reference to a singleton scoped PropertyCache. The PropertyCache maintains a reference to the current Properties object. When the PropertyCache bean is asked to return the current properties it checks to see if they have changed on disk and if so reloads the properties.

class PropertyBean {
    private Properties currentProperties;

    PropertyCache propertyCache;

    void doInit() {
        currentProperties = propertyCache.getProperties();
    public String getProperty(String propertyName) {
        return currentProperties.getProperty(propertyName);


The PropertyBean would look similar to the above sample class. This class simply caches an instance of a Properties object post-construction and delegates property requests to this instance.

The PropertyBean needs to be created as a request scoped bean. This can be done with a Spring annotation configuration class as follows.

class MyConfig {
    @Scope(value="request", proxyMode=ScopedProxyMode.TARGET_CLASS)
    public PropertyBean propertyBean() {
        return new PropertyBean();


Managing the loading of properties falls to the PropertyCache class which would look similar to the below class

public class PropertyCache {
    private Properties currentProperties;
    private Long lastReloadTime=0;
    private String propertyFileName;

    public Properties getProperties()  {
         synchronized(this) {
             return currentProperties;

    private reloadPropertiesIfRequired() {
         FileInputStream stream = null;
         try {
             File f = new File(propertyFileName);
             if(f.lastModified() > lastReloadTime) {
                 Properties newProperties = new Properties();
                 stream = new FileInputStream(f);
                 synchronized(this) {
                     lastReloadTime = System.getCurrentMillis();
                     currentProperties = newProperties;
    catch(IOException exp) {
        //do something intelligent on the error
    finally {
        if (stream != null)


All of the business logic code that needs properties to process a request will use the request scoped PropertyBean instance bean. This ensures that the entire request uses the same set of properties even if the underlying property file changes.

class SomeBusinessService {
    private PropertyBean propertyBean;

   public void someOperation() {
      String v = propertyBean.getProperty("some.property");


When a new request is started a new instance of the PropertyBean is created. The doInit() method invokes the PropertyCache getProperties() method. The Property Cache checks the timestamp of the file and looks for modifications, if it detects one it reloads the property file. Any PropertyBean instances that have already been initialized will continue to use the old properties instance.

I recently gave a talk at PuppetCamp Toronto 2015 focusing on how we use puppet at work. Slides are available here.

One of the more interesting topics I covered was how to integrate the building of puppet manifests into the software development life-cycle.

Read the rest of this entry »

Tonight I presented a talk on using JSON in Postgres at the Toronto Postgres users group. Pivotal hosted the talk at their lovely downtown Toronto office. Turnout was good with a little over 15 people attending (not including the construction workers banging against some nearby windows).

I talked about the JSON and JSONB datatypes in Postgres and some idea for appropriate uses of NoSQL features in a SQL database like Postgres.

My slides are available for download

We are thinking of having lighting and ignite talks for the next meetup. If anyone is in the Toronto area and wants to give a short (5 minute) talk on a Postgres related topic let me know.

Last week I attended the second Canadian DevOps Days event which was the first to be held in Toronto. DevOps days is a conference dedicated to DevOps. I had a great time and would like to thank the conference organizers and sponsors for making this happen.

DevOps Days was different than most I.T. conferences that I attend because none of the talks were technical in nature. There are lots software products and technologies used in the practice of DevOps such as Puppet, Chef, various CI servers , test frameworks, and logging frameworks but the conference wasn’t really about those. Talks referenced these technologies but that wasn’t the focus of the talks. The key challenges in DevOps today are softer issues. What is How do you go about introducing and promoting DevOps was the key themes of the conference.

One of the most memorable stories from the conference was told during an open-spaces session. The woman telling the story was a manager of a team responsible for some I.T service at a government ministry. She told us how formal initiatives required approvals and budgets that were difficult and time consuming to put in place. Instead of trying to get a “Devops Program” approved or budget to hire “Devops engineers” she took a different approach. Each time one of her reports came to her asking if they could do something such as make a change, fix an issue or propose a solution to a problem she would ask one question.

Will this make us suck less?

If the answer was yes, even if it made them suck just a tiny bit less she often went along with the request. If the answer was no she asked them to rethink the request. This approach allowed her team to make small incremental improvements to the systems they run. Over time their service became much more reliable meaning her team was spending less time fighting fires and had more time to implement bigger improvements within their existing budget. Running a more reliable service also relived some of the pressure on her managers and business partners earning them goodwill. She knew that she couldn’t fix all of the problems with her system in one go and she didn’t sugar coat the reality of their system. The motto wasn’t about being being great, or striving for excellence which would have seemed like a far off goal. The motto was about sucking less something they could archive in the short term while still leaving more room for improvement . The lesson to take away from this story is

Make small incremental improvements that are focused on reducing the pain your co-workers are feeling

Another story from the conference that stood out for me was from Telus. Telus started introducing DevOps practices into various teams a while ago. They started slowly but soon saw dramatic decreases in the number of production issues on products that they had tried DevOps practices on. These success where noticed and helped them roll DevOps practices out to more projects leading to more success. The CEO of Telus (a large Canadian telco) is now giving executive level support and encouragement to the DevOps initiative. The lesson to take away from this story is

Start small but in a position prepared to measure your success. Use those measured success to make a business case for expanding the effort

Firefox 29 was released in early May and contained a re-design of the user-interface called “Australis”. Unfortunately Firefox 29 crashes on startup for 32 bit powerpc users on Linux, and BSD along with most SPARC platforms.

This is because of Bug 961488 which I encountered in early January. I was able to isolate the commit that introduced this issue but figuring out why was more difficult.

It had to do with differences in the calling convention with how C structures are passed to functions. Firefox allows javascript code to call functions that are actually implemented in C++. Different plaforms use different conventions for passing data between functions if the caller and callee don’t agree on how function arguments will be passed then you have problems. The 32 bit ppc platform on Linux and the BSD platforms use a different calling convention than ppc64, AIX and ppc OSX.

The good news is that this bug is now fixed and firefox 30 should work on the powerpc. (note ppc64 builds were uneffected by this issue).

I want to thank Ginn Chen of Oracle, Gustavo Luiz Duarte of IBM for their work in coming up with the fix and Landry Breuil for shepherding the patch though the commit process.

I was recently working on a project where we had about half a dozen developers working on an established code base. All of the developers were new to the code base and I knew that we were going to be making a fair number of database schema and data-seeding changes in a short period of time. Each developer had their own development environment with a dedicated database (PostgreSQL). The developers on the project had their hands full learning about the code base and I didn’t want to distract them by having to take a lot of their time managing their development database instances.

I decided to try using Alembic to manage the database schema migrations.
Read the rest of this entry »