A metric framework for quantifying semantic reliability in shared ontology environments

Modern web based systems frequently exchange semantically rich messages in order to carry out their tasks. Such systems include Web Services, multi-agent systems, and the semantic web in general. For such message communication to be successful the terms used in messages must be interpreted according to agreed conventions on term meaning and hence usage. Ontologies can provide a common vocabulary to facilitate such meaning exchange. However semiotic constraints and application shortcomings, may still result in less than perfect semantic exchanges between applications. Where the variability of interpretation of messages is high (i.e. inconsistent or incomplete interpretation) we may have a semantically unreliable system. To date semantic reliability has only been examined in qualitative terms. This paper proposes a metric framework based on ontology test bed results to provide quantitative indices for likely semantic reliability. Such numeric indices are important, for as web applications assume more intelligent and complex tasks the potential for semantic variance in interactions only increases, and this must be carefully managed for next generation web systems to interact reliably. In order to manage anything it is invariably essential to be able to measure it; and the proposed quantitative indices provide an appropriate and viable approach for such measurement.