Well, it is time now. We are moving office in CE to a new custom built office with writing space EVERYWHERE. It is a fairly exciting time as it will completely change the way we can work.

Continue reading "Time for a change...." »

Its been a wee while since the last post, but it has been an eventful period with a number of changes, big and small. We have adjusted our internal development team as you may have noticed with the CE announcements, we are also in the process of expanding the time zone coverage of our support teams. And while this is all very good and exciting, for me it's always back to the software. We have made a few major changes to the framework we are using for the software and it has just shown us that we really have got it right. The framework is extremely flexible and resilient, and responds very well to client demand, change, and growth. I'm all about the growth and nurturing of software, where specifications can bend to demand rather than the strict architectural approach where every detail is planned to the nines, and there is no room for change. Over the past couple of weeks we have had a few requests which without the framework would have resulted in a plumbers quote, slow intake of breath, "oooohhh, that's a big un", as well as a few minor but still fairly significant changes that in other companies would have resulted in a few sleepless nights and at least one coke and coffee filled weekend.

I give some paraphrased examples of the requests along with actual development times (testing included):

- "Your software does not support MSSQL, we can not get it through procurement without full support for MSSQL". 1.5 days later the products now supports MSSQL databases with zero change to current code, business logic, or data layers. Only a data adapter was required.

- "We need to run your server as a passthrough server forwarding to another instance of your application behind our DMZ". 1 day later, partial proxy and full pass through services implemented into the software. Only one class required changing. All web services, and server side pages, and javascript worked without modification.

- "I only run linux at the moment". 1 hour later, one line of code added to support the case sensitive nature of the linux file system.

- "Flash is not acceptable, we can not install in our environment". 3 weeks later, HTML Collector is up and running and live. Granted 3 weeks is much longer than the other times given, but we did have to rewrite all of the flash code in javascript and html, only the web services were reusable.

Most of the early projects I worked on, change was seen as the enemy of progress. Specifications were locked down for months so developers could get out deliveries, months were spent even before development started trying to think of every possible situation that could occur within the problem space and trying to ensure they were all included in the specification, missing a few here and there which was then used as defense when the situation did occur "well it wasn't in the spec". This type of process is great for rigid structures where the rules will not change by the time the development will be complete. In the world we live in, we can't even predict the price of petrol from day to day, never mind attempting to define all of the problems spaces in which we will be working 2 years from now. In the world of problem solving, rigid solutions are often not workable, and the time and resources will have to be invested after the fact to adjust the solution, usually resulting in a complete teardown and rebuild. Growing software takes the time up front, it is important to get the roots right, but once they are, there is no need for a teardown. Adjustment is all that is needed, and this takes much less effort and resources than rebuilding. It's always the roots that are important and they must be carefully planned and built.

Seeing the major changes we have been able to cope with in the framework, and how easy it has been to implement them, I am confident the roots are solid. With each change the software becomes more resilient, more flexible, and more suitable to different situations, it is not a "square peg, square hole" solution, it is more like lego bricks, build what you need.

As the development framework that we use at Cognitive Edge includes some very core functionality such as web server and smtp communication, you get to see some interesting things when you are playing around and watching how internet browsers communicate with the servers.

Over the last few days, I've been watching some different browsers and what caught my eye was the Blackberry default internet browser. From what I can see, it is set up so that it will not ask the web server that it is communicating with to compress the data that will come down to the phone when you are browsing using the data services, but it does allow for the compression when you are browsing on the wireless (i.e. the hotspot browser).

Basically the compression works like this, the browser asks the server for a web page and while doing so notifies the server that it is able to receive compressed files. The Server will then send a compressed version of the file, often more than 50% smaller than the original, down to the client. This practice had been put in place to combat slower connection speeds and lower bandwidth connections, less data traveling across the wire means faster download speeds.

Now given that the Blackberry can definitely support compression which we know because the hotspot browser does exactly that, it leads me to believe the only reason the default browser does not support compression is because a lot of BB customers pay by the amount of data they are actually downloading. They have made the decision to ensure you have compressed data for faster downloads while you are connected to a wi-fi connection, but when you are using the data services and actually paying, they force you to download more than 50% more data than you actually need to be getting.

Admittedly I am without any doubt a "desk jockey". I spend the day sitting at a desk, and sometimes get so lost in my work that I even end up at the desk in the wee hours of the morning wondering where the day went. Having said that, all throughout my working life I have felt that it is important to have a physical outlet for releasing tension and expending energy. The forms this has taken over the years has varied but usually has been a fairly aggressive activity, military, boxing, rugby, or something along those lines. Unfortunately since coming out to Singapore, the rugby experience isn't quite as good as it was in England with the team I had joined struggling even to find grounds to train on, and the season being so short that the frustration just builds rather than getting released. There is nothing better after a rough day at work than getting knocked around a bit to let you know that however bad the day was, it just got worse.

The other reason behind the activities is that I have felt that being physically fit also promotes mental fitness and energy. When energy levels are high, you think faster, clearer and (in my case) seem to be more creative with the solutions to some problems. With that clear headed mentality it also makes it easier to interact with others as you have less of those grumpy cranky days where anything and everything annoys you to the point where even the green charging button on your laptop gets an evil look for being too green.

So now that the rugby season is once again over and now that there is no area to release, I've decided to make a larger commitment than usual. I've found a fitness program online that consists of one entire year of exercising. Having just started the program, and with the way I am currently feeling, I'm now unsure as to my soundness of mind. I think that most people who were in any way sane and have seen this routine would have laughed it off and gone on to something much more realistic but never the less, it's started, and I'm going to do my best to complete it, no matter how sore and physically tired I feel. It will be interesting to see how it changes sleep patterns, working patterns, productivity and output. Only time will tell.

To a software developer, everything is a problem needing to be solved. Once a solution has been found, there is no longer a problem. This can make it very difficult to keep developers interested in a project. On a developers first day of work, moving buttons around on the screen is exciting, by the 3rd button, they are so sick of moving buttons they can not stand it anymore. They look at their watch, 30minutes into the day, only 7 and a half hours left of moving buttons, that may as well be a lifetime. At that point the developer uses 1% of their brain on the moving buttons task, and the rest gets used up on thinking about what new tool they are going to go home and code. Instead of buttons being moved to the bottom left of the screen, they get moved to the bottom leftish of the screen, boredom sets in and details are blurred.

Repetitive tasks are the developer equivalent of a lobotomy. Brain function stops and interest in the project and the job is lost. If you are working on a task for a second time, then it is a perfect candidate for automation. If you did it once, and now you are working on it the second time, most likely you will need to do it a third and fourth time. By the second time, the template for the solution is already there, there isn't time used up researching how to solve the problem, so instead spend the same amount of time looking into automating the solution. If the problem is the exact same one each time, then hard code the solution, if the problem has variables, then allow for the input of parameters to your solution. If you are not able to push your code back in to the project for everyone to benefit from, then create your own dashboard of buttons that run each automated task.

If you are managing developers, then you should be watching for repetitive tasks, you should be looking for ways to automate, and you should be encouraging your developers to do the same. If you ask a developer to complete a task and they pull up a dashboard of tools they have created, you should be asking yourself why that code hasn't been integrated into the project. Automation is full of advantages, which by far outweigh any disadvantages. Here are some of the benefits:

- The automation of a problem includes a complete analysis of a problem in order to code a solution. During that analysis a developer may see that during the manual process is inefficient in the number of steps, or is error prone because of the order of steps.

- Automating allows for simple monitoring of the task including how long the task takes and how many times it has been run. This can allow for patterns to be analyzed such as why so many refunds need to be manually issued at the end of December.

- Automation of a task give the person tasked with automation a deep insight into the task

- Increasing the code ownership for a developer giving them more stake in the project, getting their heart into it.

- Refactoring of the code can take place, applying the solution to other tasks or problems

- What was originally a task assigned to the same person can now be delegated to any individual who can press the go button, again removing the burden from the developers and moving it over to the operations side.

The main benefits to a developer is the workload of repetitive tasks are reduced, meaning the developer can work on more interesting and challenging tasks.

So while the saying goes, if at first you don't succeed, try, try again. I propose an alternate, if at first you do succeed, automate!

Building software is often compared side to side with erecting skyscrapers and bridges. The thoughts being that if a civil engineer can write up the timelines and plans so detailed that they include details on not only the position of each nut and bolt, but also the materials that those nuts and bolts must be made of in order to maintain a tolerance for an event that may or may not happen in the history of the structure being erected, then why should it not be possible for software architect to do the same. Why shouldn't software architects be held to the same exacting standards. Why shouldn't software architects be forced to provide detailed plans with every milestone, every component, and every technique and technology that will be used in the lifetime of the software. Why shouldn't software architects future proof their software for every possibility?

The answer is actually fairly simple. Buildings and constructs are not scaleable. You can only build a skyscraper so big before the patterns fail and collapse. You can only build a bridge so far before the techniques must be altered and different load bearing methods must be employed. A building will not move. When a site is chosen, that site is examined and will not change. Software does not exist, software is build for ever changing requirements. If you plan a detailed software build now that is to complete in two years, the day you release your software it is already two years out of date. In the technological landscape, two years may as well be eternity.

So how does one build software that will stand the test of time, conform to the rigors of an environment that changes as much as it does within Cognitive Edge, keep up to date with the latest research and techniques so each release includes the ability to incorporate the latest research on the Cynefin model, or leverage the latest techniques in semantic analysis or knowledge management?

Software must no longer be considered a build, but rather a growth. Software architects must become gardeners balancing the rate of growth and death with the aesthetics and functionality of all the components withing the it. If you picture a problem space as a piece of property that is relatively static, but can change with major events, earthquake, flood, etc. Then the software is a garden on that piece of land. Each plant, tree, and weed is a component of the software built and placed to enhance or support the function of each component around it. Making the garden more than the sum of it's parts. Each component separately and over different time spans goes through its cycle of germination, maturation, and death, but the garden itself lives on. As a component gets old and stops functioning, or doesn't function as efficiently as it could be, it is replace with another component that is just as beautiful and supports the ecosystem around it. If tastes (or methodologies) change and the new fashion for the year is blue flowers, then all of the red ones can simply be removed and replaced. Again the change means the garden is still the garden, the software is still the software, so long as the gardener plans the changes correctly and pays attention to the ecosystem watching for signals from the environment inside and out, that change can be made without disruption of the garden. Growing software in this method means that the software can easily outlast the gardener, there is no longer a need to destroy what was built, to tear down the building to replace with a more futuristic version of the same thing, instead it is tending the garden, as areas become overgrown and weedy, you trim them back and clean them up. When components become outdated, you upgrade only those components, not the system as a whole.

Currently I am the gardener for Cognitive Edge, I have the privilege of preparing the roots for all who come after me, we are at the stage now where the roots are down. We have gone through a lot of planning and many iterations at Cognitive Edge to get to this stage, the plants are starting to hit their first maturation cycle and we are getting ready to open the gates to the garden. We are very happy with our garden, we know there are a couple of weeds hidden here and there, and some of the flowers may not have been the most aesthetically pleasing choices, but when we open the gates we will not do so without leaving a comment box for you. When we find the weeds we will quickly route them out, when the vines get tangled we will spend the time cleaning them up. The software is the software, the components are changeable. As we are feeling out the garden and getting used to it we will become more and more open to your direct feedback.

We thank you for waiting, we welcome you to our garden.