The @Bean annotation is a great way to declare Spring beans when writing code.  This allows the programmer to control which beans get instantiated and with what arguments.   Some situations call for deciding if beans should be instantiated at runtime based on configuration this is often done with the @Conditional annotation on @Configuration classes.   The @Conditional annotation works well if you are enabling or disabling a particular bean through configuration but sometimes the number of beans needs to be completely dynamic.

This often comes up if your application needs to connect to a bunch of external services such as a database.  You want the number of  services or databases that the application connects to be configurable.

Read the rest of this entry »

Spring integration makes it easy to monitor an sftp server for new files and inject those files into your application for processing.  I like to configure my spring integration applications with annotations and @Bean configurations.  I found the documentation, and online examples for doing this annotations a bit lacking.

 

This post demonstrates the basics on how to connect get a spring-boot (1.3.x or 1.4.x) application to monitor a sftp server.

Read the rest of this entry »

Sometimes the behaviour of an application is controlled through properties
and the application needs to detect changes to the property file so it can switch to the new configuration. You also want to ensure that a particular request either uses the old configuration or the new configuration but not a mixture of old and new. Think of this as ACID like isolation for properties to ensure that your requests don’t get processed using an inconsistent configuration.

We accomplish this by using a property lookup bean (PropertyBean) that returns property lookups. The PropertyLookupBean is a request scoped bean meaning that a new instance is created for each request in a spring MVC application. The PropertyLookup bean has an init method that runs post construction and calls the PropertyCache bean to get the current set of properties. Each instance of the PropertyLookup bean will return the property values from the Properties object it gets at initialization.

properties

The property lookup bean has a reference to a singleton scoped PropertyCache. The PropertyCache maintains a reference to the current Properties object. When the PropertyCache bean is asked to return the current properties it checks to see if they have changed on disk and if so reloads the properties.

class PropertyBean {
    private Properties currentProperties;

    @Autowired
    PropertyCache propertyCache;

    @PostConstruct
    void doInit() {
        currentProperties = propertyCache.getProperties();
    }
    public String getProperty(String propertyName) {
        return currentProperties.getProperty(propertyName);
    }
}

 

The PropertyBean would look similar to the above sample class. This class simply caches an instance of a Properties object post-construction and delegates property requests to this instance.

The PropertyBean needs to be created as a request scoped bean. This can be done with a Spring annotation configuration class as follows.

@Configuration
class MyConfig {
    @Bean
    @Scope(value="request", proxyMode=ScopedProxyMode.TARGET_CLASS)
    public PropertyBean propertyBean() {
        return new PropertyBean();
    }
}

 

Managing the loading of properties falls to the PropertyCache class which would look similar to the below class

@Component
public class PropertyCache {
    private Properties currentProperties;
    private Long lastReloadTime=0;
    @Value("${dynamic.property.file.path")
    private String propertyFileName;


    public Properties getProperties()  {
         reloadIfRequiredProperties();   
         synchronized(this) {
             return currentProperties;
         }
    }

    private reloadPropertiesIfRequired() {
         FileInputStream stream = null;
         try {
             File f = new File(propertyFileName);
             if(f.lastModified() > lastReloadTime) {
                 Properties newProperties = new Properties();
                 stream = new FileInputStream(f);
                 newProperties.load(stream);
                 synchronized(this) {
                     lastReloadTime = System.getCurrentMillis();
                     currentProperties = newProperties;
                 }
             } 
          } 
    }
    catch(IOException exp) {
        //do something intelligent on the error
    }
    finally {
        if (stream != null)
            stream.close();
    }
}

 

All of the business logic code that needs properties to process a request will use the request scoped PropertyBean instance bean. This ensures that the entire request uses the same set of properties even if the underlying property file changes.

class SomeBusinessService {
    @Autowired
    private PropertyBean propertyBean;

   public void someOperation() {
      String v = propertyBean.getProperty("some.property");
       .
       .
    }

}

When a new request is started a new instance of the PropertyBean is created. The doInit() method invokes the PropertyCache getProperties() method. The Property Cache checks the timestamp of the file and looks for modifications, if it detects one it reloads the property file. Any PropertyBean instances that have already been initialized will continue to use the old properties instance.

I recently gave a talk at PuppetCamp Toronto 2015 focusing on how we use puppet at work. Slides are available here.

One of the more interesting topics I covered was how to integrate the building of puppet manifests into the software development life-cycle.

Read the rest of this entry »

Tonight I presented a talk on using JSON in Postgres at the Toronto Postgres users group. Pivotal hosted the talk at their lovely downtown Toronto office. Turnout was good with a little over 15 people attending (not including the construction workers banging against some nearby windows).

I talked about the JSON and JSONB datatypes in Postgres and some idea for appropriate uses of NoSQL features in a SQL database like Postgres.

My slides are available for download

We are thinking of having lighting and ignite talks for the next meetup. If anyone is in the Toronto area and wants to give a short (5 minute) talk on a Postgres related topic let me know.

Last week I attended the second Canadian DevOps Days event which was the first to be held in Toronto. DevOps days is a conference dedicated to DevOps. I had a great time and would like to thank the conference organizers and sponsors for making this happen.

DevOps Days was different than most I.T. conferences that I attend because none of the talks were technical in nature. There are lots software products and technologies used in the practice of DevOps such as Puppet, Chef, various CI servers , test frameworks, and logging frameworks but the conference wasn’t really about those. Talks referenced these technologies but that wasn’t the focus of the talks. The key challenges in DevOps today are softer issues. What is How do you go about introducing and promoting DevOps was the key themes of the conference.

One of the most memorable stories from the conference was told during an open-spaces session. The woman telling the story was a manager of a team responsible for some I.T service at a government ministry. She told us how formal initiatives required approvals and budgets that were difficult and time consuming to put in place. Instead of trying to get a “Devops Program” approved or budget to hire “Devops engineers” she took a different approach. Each time one of her reports came to her asking if they could do something such as make a change, fix an issue or propose a solution to a problem she would ask one question.

Will this make us suck less?

If the answer was yes, even if it made them suck just a tiny bit less she often went along with the request. If the answer was no she asked them to rethink the request. This approach allowed her team to make small incremental improvements to the systems they run. Over time their service became much more reliable meaning her team was spending less time fighting fires and had more time to implement bigger improvements within their existing budget. Running a more reliable service also relived some of the pressure on her managers and business partners earning them goodwill. She knew that she couldn’t fix all of the problems with her system in one go and she didn’t sugar coat the reality of their system. The motto wasn’t about being being great, or striving for excellence which would have seemed like a far off goal. The motto was about sucking less something they could archive in the short term while still leaving more room for improvement . The lesson to take away from this story is

Make small incremental improvements that are focused on reducing the pain your co-workers are feeling

Another story from the conference that stood out for me was from Telus. Telus started introducing DevOps practices into various teams a while ago. They started slowly but soon saw dramatic decreases in the number of production issues on products that they had tried DevOps practices on. These success where noticed and helped them roll DevOps practices out to more projects leading to more success. The CEO of Telus (a large Canadian telco) is now giving executive level support and encouragement to the DevOps initiative. The lesson to take away from this story is

Start small but in a position prepared to measure your success. Use those measured success to make a business case for expanding the effort

Firefox 29 was released in early May and contained a re-design of the user-interface called “Australis”. Unfortunately Firefox 29 crashes on startup for 32 bit powerpc users on Linux, and BSD along with most SPARC platforms.

This is because of Bug 961488 which I encountered in early January. I was able to isolate the commit that introduced this issue but figuring out why was more difficult.

It had to do with differences in the calling convention with how C structures are passed to functions. Firefox allows javascript code to call functions that are actually implemented in C++. Different plaforms use different conventions for passing data between functions if the caller and callee don’t agree on how function arguments will be passed then you have problems. The 32 bit ppc platform on Linux and the BSD platforms use a different calling convention than ppc64, AIX and ppc OSX.

The good news is that this bug is now fixed and firefox 30 should work on the powerpc. (note ppc64 builds were uneffected by this issue).

I want to thank Ginn Chen of Oracle, Gustavo Luiz Duarte of IBM for their work in coming up with the fix and Landry Breuil for shepherding the patch though the commit process.