TECHNOLOGY ASSESSMENT AND EVALUATION

TECHNOLOGY ASSESSMENT

Technology Assessment (TA) and Technology Evaluation (TE) are very closely related terminologies and sometimes even overlap. Technology Forecasting (TF) is also generally dealt with in relation to TA and TE. TA deals with assessment of a technology on a wider canvass than the immediate concerns of a firm. It covers in its ambit the direct and indirect effects and consequences of introduction and use of technology on diverse parties and in specific environments.

Historically, every society in some way assesses the introduction of new technology in an implicit way, e.g. Tipu Sultan while-introducing rocketry in warfare made an, implicit assessment of the havoc the novelty of it would cause amongst the British soldiers, rather than its actual damage, distinctive effect. However, the distinguishing feature of present day technology assessment (TA) is that it is an explicit attempt to assess and consciously select the implementation of a new or expansion of an existing technology.

The birth of TA in its formal sense originated in early sixties when the Congress of USA, concerned with the pervasive and often undesirable effects of introduction of new technologies, especially on the environment, called for a new form of policy research and analysis to deal with the subject. TA thus has been largely applied in USA, and to some extent in France, Germany and other E.E.C. countries but very little in developing countries, including India. This is not to give the impression that since most new technologies originate in the developed countries, TA is undertaken in such countries and is of relevance to them. TA is also relevant to the developing countries. Consider, for example, the introduction of computerisation in banking and insurance, farm mechanisation, robotics in assembly line manufacturing etc. Obviously their impact would be vastly different in the Indian context than say in USA or Japan. This would indicate that TA needs to consider the social, cultural, political, economic, and industrial environment not only of the country but even of different target groups within it.

Government departments, depending upon their responsibilities, have to form opinions about technologies say construction of a major dam, extending telecom facilities to rural areas, location of a ‘shooting range’ for defence purposes, introduction of a new contraceptive, etc. In these tasks, technology may play from a major to a subsidiary role. Traditionally, for TA the government has relied on advice of civil servants, advisory committees, experts, commissioned consultancy reports etc. But these advisory arrangements are generally a far cry from the full-fledged formal TA that should be undertaken to help the government arrive at meaningful decisions for public good/interest.

Definition

There are several definitions of Technology Assessment and all of them focus on the advantages of TA and are more or less similar in content. Emilo Daddario, the US Congressman, who is credited with coining the term TA in 1967, defined it as:

“Technology Assessment is a form of policy research which provides a balanced appraisal to the policymaker. Ideally, it is a system to ask the right questions and obtain correct and timely answers. It identifies policy issues, assesses the impact of alternative courses of action and presents findings. It is a method of analysis that systematically appraises the nature, significance, status, and merit of technological progress.”

The definition used by the US Congressional Research Service, although slightly lengthy is more expressive and explicit, it reads as:

“Technology Assessment is the process of taking a purposeful look at the consequences of technological change. It includes the primary cost/benefit balance of short-term localised market place economics, but particularly goes beyond these to identify affected parties and unanticipated impacts in as broad and long range fashion as is possible. It is neutral and objective, seeking to enrich the information for management decisions. Both ‘good’ and ‘bad’ side effects are investigated since a missed opportunity for benefit may be detrimental to society just as is an unexpected hazard.” 

(Source: (for above two quotations) Wad A., and Radnor M., Technology Assessment: Review and Implications for Developing Countries,. UNESCO-Science Policy Studies and Documents No. 61, 1984).

It is thus obvious that TA attempts to derive a cause and effect relationship between the policy options to be pursued and their effects, both beneficial and harmful. It not only considers those effects that are apparently direct, but also those that are unforeseen, delayed or even indirect. The definition thus implies that if such and such technology is adopted, such and such is likely to happen and such and such groups in certain ways are likely to be affected leading to certain possible outcomes. It is then for the decision-maker to decide which effects are acceptable/ permissible and which are not, and hence which policies need to be pursued or not pursued.

TA can be either problem oriented or technology oriented. Examples of problem oriented TA in the Indian context are: reducing imports of crude, optimisation of energy use, abolition of illiteracy, etc. The examples of technology oriented TA are: introducing bio-pesticides in agriculture, computerisation of railway operations etc. The problem related TA must obviously list all the available solutions of the problem and analyse their impact. The technology oriented TA must not only analyse the impact of a given technology but also study the rival/alternate technologies and their impact as well. Thus, basically both the types of TA deal with/involve the same procedures/methodology.

METHODOLOGY OF TECHNOLOGY ASSESSMENT

Due to the vastness and the inherent interdisciplinary nature of the subject and the complexities of the issues involved, there has often been ambiguity in defining what TA methodology should include. Here, the TA methodology is defined to include not only the listing and application of major stages in the TA process itself, but also the organisation and.the management of TA exercise.

Stages in the TA process

Several well-known authors’ on the subject have listed what are considered as the key stages in the TA process. Some of these are given in Table-1.

 Stages in technology accessment

Table-1: Stages in Technology Assessment

A brief explanation of the various stages is given below.

Problem Definition: This involves the proper specification of the problem to be studied and establishing its limiting (bounding) parameters. Automatically the questions that need to be answered are:

  1. a) For whom or what purpose is the study being done?
  2. b) Who are the affected groups/targets?
  3. c) Over what time horizon is the problem to be studied?
  4. d) What is the spatial/geographical coverage?
  5. e) The extent/range of technological options to be covered?
  6. f) What is the choice of projected societal values and structure?

It should be recognised that this stage is not a one time exercise, but a continuous process that permits the problem to be refined and redefined as the study progresses.

Technology description/forecast: The three main elements of technology description are

(a) establishing the boundary of the technology per se

(b) the data pertaining to the technology to be acquired and

(c) technology forecast. Thus, this basically involves defining the current state of the art of the technology and projecting it into the future along feasibly attainable alternative paths.

Prior to using a technology forecast we need to decide:

  1. i) the extent of projecting past trends as compared to defining future objectives;
  2. ii) the extent of considering technological alternatives; and 

Technology forecast: Study more clicking on Technology Forecast

Social descriptions/social forecast: Since the core purpose of TA is to examine the effects of technology on society it is appropriate and necessary to describe the society in which the technology exists or will exist. There are various ways of describing society, viz.:

a) The state and stability of society (war, no political upheavals etc.),

b) macro indicators such as characteristic of the economy in terms of industrial/agricultural/services shares, income and its growth, population as characterised by male; age and educational profile etc.

c) specific descriptions of aspects such as percentage of children in the population, percentage of people in a certain income group, literacy, etc.

Some experts suggest that there should also be a description of the symbolic elements of society that the policy makers are trying to achieve or goals they have set for themselves e.g. decentralization, panchayat level planning etc. Social forecasting is extremely difficult and complex and there are very few models that could be advantageously used. The two most used approaches are cross-impact analysis and scenario construction, though they have their limitations.

Impact identification, analysis and evaluation: It mainly consists in identifying, analysing and evaluating the impacts of the specific technology. It is thus concerned with producing knowledge to assess the range of consequences that will result from particular technology development. It also involves performing a comparative evaluation of the technological alternatives using broad-based criteria such as social, cultural, political and environmental concerns along with more conventional concerns such as technical performance, economic, legal and institutional considerations. The selection of criteria by which the impact of technology is assessed is a critical step in performing TA since, in a sense, it pre-focuses the entire impact assessment effort. The impacts identified are then rated according to their importance so that the impact field could be within manageable levels. The importance of impact and the criteria followed could vary in terms of significance for the policy makers, the affected groups, the political system etc. This should be clearly specified. Each of short-listed impacts including the second order impacts are analysed in terms of their significance, probability, timing, costs, affected parties etc.

This could be, carried out in the following ways:

a) Scientific analysis: Subject wise experts, are employed to conduct and analyse the impacts in their respective fields e.g. economic impact, environmental impact, psychological impact

b) Interdisciplinary and futuristic analysis: This covers a large range of fields and involves systems analysis and futures research. Some of the common techniques used are; expert opinion, cross-impact analysis, scenario writing

c) Social impacts analysis: The emphasis is on a broad set of social impacts identified for each of the short-listed impacts. The common techniques used are expert opinion, polling, morphological analysis.

The last stage is evaluation of impacts. This basically entails the assigning of values to specific impacts, e.g. degree of environment non-pollution for a fertilizer plant project could be assigned a higher value than the profitability, etc. The values assigned are largely dictated by the bias of the evaluator, and/or the sponsor or the social group it represents. But the primary concern is to enhance the objectivity of the TA. One of the ways to achieve this is by involvement of the stakeholders (interested parties) in the evaluation through conferences, polls, public reviews, interviews, direct participation in the evaluation team, etc. To some extent impact evaluation is implicit in initial stages of TA as well. At this initial stage itself the underlying assumptions/biases are made explicit. Several techniques are available for evaluation of impacts, e.g. dimensionless scaling, decision analyses and policy capture. 

Policy Analysis: The policy analysis consists of two levels: the first level deals with specific policy options and is carried out in four steps:

a) formulating feasible policy options through which to implement each of the technological options,

(b) comparative analysis of the policy options by using the impact assessment and evaluation, also listing therein second order and perceived impacts,

(c) synthesizing the best or optimal policy or combination of policies and the strategies for implementing each technological alternative and

(d) presentation of a summary comparison of the selected optimal policy options and a comparative evaluation of the advantages and disadvantages of each policy option.

It may not always be possible to follow these four steps as application of one or more technological alternatives may extend beyond the scope of policy thrust itself.

The second level policy analysis deals with identifying and assessing general circumstances, obstacles, concerns and conflicts that might be associated with the technology alternatives. However, the outcome of policy analysis should be to provide the decision maker with fairly substantial, realistic and objective description of the various available alternatives, their implications, and their feasibilities. 

Communication of Results: Effective communication of the conclusions/ recommendations/results of the study is essential to the success of TA itself. The objective of the study is to present the results in a manner that is comprehensible to the diverse constituencies, the decision makers, the stakeholders and the public. The decision-makers have to be informed of the implicit trade-offs between future and present costs/benefits/impact etc. A communication barrier may nevertheless arise with the decision-maker as he may be unwilling to assume the risk which may be assigned to various policy options. Communication to public may pose problems as most TA studies deal with potentially controversial issues. Lastly communication within the study team on the revision of definition of the task etc. is also important. 

ORGANISATION AND MANAGEMENT OF TA

Given the complexity of the TA and its broad interdisciplinary coverage the organisation and management of the TA study itself is included as a part of the TA activity. Two features identified with respect to TA study are structural and process details. The structural features include project team, its composition, and leadership, selection of techniques, hierarchical set up and whether the study is carried out as interdisciplinary or interdepartmental work, the budgetary resources etc. The process features include project scheduling with milestones, communication pattern among project team and its integration, especially among the members from different disciplines. Although these features are well stated in theory the TA in practice is often undertaken on an ad hoc basis.

TA IMPERATIVES

TA attempts to assess the effects in future of technology development and its introduction on society are inextricably linked to forecasting. TA thus attempts to obtain scenarios of future developments of both technology and society and their mutual interactions. The key words for success of TA are neutrality, unbiasedness and objective analysis on the part of the study group, and their presentation and interpretation of results without any fear or favour. But it is not easy to confirm these qualifications of TA as the collection of real life database for TA is very difficult. Braun has described why this is so:

“The database for Technology Assessment does not consist of laboratory observations obtained from immutable nature according to strict rules. Instead, it consists of a complex web of fact, conjecture and opinion obtained from a wide range of written and oral sources. Different assessors have access to different sources and often, alas all too often, the data available are but the tip of the iceberg of secret information; secret because of commercial or political or personal confidentiality or simply out of the convenience of the cabal. Such information is largely inaccessible to the analyst and often even the knowledge of its existence remains hidden. We must accept that the database for a TA will be incomplete and variable according to the methods and possibilities of collection and selection by different groups of analyst.”

Due to these difficulties a certain amount of subjectivity is inevitable in any TA study. TA is an ideal: strive as we must, the best we can achieve is a gradual improvement in our understanding of the effects of technology and thus better control and mastery over it. TA has now reached a level where it is an accepted technique to arrive at meaningful and useful policy analysis. Thus TA needs to be used increasingly by decision-makers in developing countries to choose appropriate forms of technology keeping in view their national/organisational objectives/goals, e.g. selfreliance, fulfillment of basic needs, poverty alleviation, enhanced employment generation, environmental soundness etc.

In forward looking very large corporate organisations in developed countries, TA is sometimes undertaken as a continuous exercise in order to remain competitive. The findings of this exercise are used as inputs to the planning process and decision making in general. At the national level, in India, there have been sporadic attempts at TA from time to time; it is only recently that the Technology Information, Forecasting and Assessment Council (TIFAC) has been set up in the Department of Science and Technology. It has initiated studies related to TA in India. It is hoped that these efforts would be useful to the policy makers in the newly emerging policy environment. 

TECHNOLOGY EVALUATION

In India the formal acquisition of technology, through licensing or purchase, has been rather small. Indigenous technology sources such as NRDC, CSIR, ISRO, DRDO, AEC, DOE, DST, collectively license no more than 400-500 parties in a year and import of technology accounts for a similar figure. In most cases technology is acquired through informal mechanisms such as mobility of persons, reverse engineering etc.

Technology evaluation as a formal tool has not been applied or used in most cases of technology acquisition in India. But with the integration of the Indian economy with the global economic and trade systems, industrial units in India would need to be globally competitive. For exports not only product specifications arid quality but also manufacturing practices and facilities would need certification and accreditation. This would require that the entrepreneurs carefully select the technology to be adopted. The first step in this direction is technology evaluation (TE). TE is a firm/ organisation level exercise of choosing a technology from amongst a set of available technologies, the adoption or use of which will optimise on a set of defined parameters. These technologies may be available from abroad, within the country, or even developed in-house by the firm/organisation. TE thus differs from TA in that:

i) TA is a macro level exercise and seeks to look at the effects of a technological development on society and more specifically on special groups etc., whereas ‘TE’ determines the effects of adoption of a technology on or with respect to that firm only.

ii) In TA the effects or their quantification is not predefined, whereas in TE the technologies are to be compared on the basis of known/defined parameters.

iii) TA is often more concerned with secondary or unintended effects while TE is more concerned with direct or primary effects. Once again, like TA the difficulty lies in obtaining authentic data and information about a technology. Unlike TA one needs to obtain commercial data on the operation of a technology because the final decision in TE most likely to be a commercial one. There is thus not much academic or formal lit; nature on TE as it is a firm/organisation level exercise and seldom do firms seek to publicise the outcome of their studies. Most often the information pertaining to a technology is obtained from the licensors or other sources under a secrecy agreement and thus cannot be made public. The process of and interface between Technology Assessment and Technology Education; has been presented in Figure-1.

 Process of Technology Accessment and Evaluation

Figure-1 : Process of Technology Assessment and Evaluation 

TE PARAMETERS

The basis of evaluation of technology generally takes ‘into account factors such as ease/cost of technology acquisition, level of development and the sophistication of technology, size of capital investment and its scheduling, rate of return on investments, break even point, pay back period, sensitivity index with respect to diverse input/output parameters, nature of pollution caused and cost of control, type of capital goods/raw materials required, their availability and imports, manpower skills required and their availability, quality control, exportability of products/ technology, hazards and risks involved and their management costs, rate of obsolescence of technology and technology replacement costs etc. It is obvious that these factors are locale specific. The technology evaluation norms thus would vary from place to place. It technology evaluated by a firm in USA and India need not have the same weightages for specific evaluation parameters and sometimes may not involve even the same parameters. Consider for instance the evaluation of a technology for making S.G. Iron Castings in India and in USA. The nature and cost of pollution control measures would be vastly different in the two countries. In India it may not even be considered as a major parameter. The level of sophistication of technology would have different interpretations in the two countries. In India a semiautomatic system would be preferred whereas in USA the preference would be for a fully computer controlled and automated line.

An organisation at the initial stages of technology evaluation has to determine and decide on the parameters against which the competing technologies have to be evaluated and ascribe relative weightages to these parameters so as to arrive at a single value/index which could then be compared for different technologies. Even two firms in the same country evaluating technologies for the same final product may not ascribe the same weightages to different evaluation parameters. Once again let us take the case of S.G. Iron Casting technologies being evaluated by (a) an entrepreneur seeking to set up a small-scale unit (SSU), and (b) an existing largescale pump manufacturer (LSU). The SSU would seek technology that is more labour intensive, requires less of capital investment per tonne of output and is within the capital resources of SSU is simple and easy to operate by unskilled operators, is capable of quality control based on manual testing;, whereas the LSU would prefer a technology that is comparatively less labour oriented, has automated and instrumental quality control and high throughput. Thus the parameters and weightages for TE would differ between these two.

Thus every enterprise would need to evolve and lay down its own set of evaluation parameters and based on its resources endowments ascribe a weightage to them. It could then compare the available technologies based on these parameters and the relative weightages

Related Posts Plugin for WordPress, Blogger...

Add a Comment

Your email address will not be published. Required fields are marked *

Social Media Widget Powered by Acurax Web Development Company
Skip to toolbar