How to measure the effectiveness of IS?

That is the main question that I would like to address in this blog.

As we may have known, IS/IT (for some reasons, both terms will be use intertwined, as the main focus is not the difference between them) is a crucial factor determines a success of institutions. No doubt that a successful institution must have well established IS/IT inside. That’s why institutions do not hesitate to spend huge amount of fund to buy and implementing grand new IS. Billions of dollars had been spend, and is going to be spending, by both private and public institutions in order to get the advantages of it.

As it was cited by ZDnet;

Government spending on technology will continue to grow for at least the next four years, with local authority spending outpacing that of central government.

Western European governments will invest $49bn (£27bn) in IT in 2009, compared to $38bn last year, predicts analyst firm IDC.

Government IT spending will grow at an average 5.3 percent until 2009, compared with 4.5 percent growth for the wider IT market

However, the result of the spending is still difficult to be measured. It is elusive to evaluate the fruits of IT, although many efforts have been done. There is no single approach to know the effectiveness of new IT investment right away. Why?

According Remenyi (Remenyi et al. 2000), there are 4 major areas contributed to the problems of identifying and measuring the benefits of IT. They are;

1. Identification of benefits and performance improvements
2. Issues of IT reach
3. Forms of benefits (tangible and intangible)
4. The evolution of the benefits

Lets talk each of them.

In order to be successful, IS projects need to be started by identification of potential benefits. What kind of benefit are expected by implementing new IS in a company? This question should be answered and the beginning stage of the project. However, it is difficult to give definitive statements of all benefits a project might be given. The benefits often come not solely from the project itself, but from the combination with others.

Information system’s benefit cannot be separated with other participants in making improvements. Other participants here could refer to human resources, existing Information Systems, existing equipments, timing and many others. It is difficult to identify the effect of IS itself without interacting it with others, since IS cannot play by itself. A simple example could be implementation of new attendance system. When it was implemented, the system will interact with other systems, for instance payroll system, rewards and punishments, taxation system and others, immediately. It will be difficult to just evaluate the effect of new attendance system if other systems did not exist. it may not give anything to the company, but costs.

Another difficulty is because the benefit of IT, sometime does not closely related to what it was expected. The “side” effects here, sometimes encourage the all components of an institution to provide better performance, thus, it seems that the implementation of it, has brought more benefit than it was intended.

So, how can we measure the effectiveness of implementation of particular IS/IT? The answer is difficult. But it is not impossible. Many works still needs to be done to give better guidance to asses the performance of IT. Is it importance? Yes of course. Huge amount of money had been spent, and still, will be spent in the future for IT. If the tool to measure them is not ready in the very near days, we would never know how we focus our expenses on.

The question how to measure the effectiveness of IT/IS should have been answered by IS/IT sellers. Don’t they try to convince their prospect customers to buy their product, do they?