EXTENDED TUTORIAL ABSTRACT

EVALUATION, ASSESSSMENT AND CERTIFICATION OF

DEPENDABLE INFORMATION SYSTSTEMS

by Hans-Ludwig Hausen, Fraunhofer,

What is evaluation, assessment and certification? Why do we need it? We

ask for software assessment and certification because we want to be sure

that the product we want to apply provides the expected service correctly

with respect to both functional and non-functional requirements. If we are

assessing software we check whether the actual service we can get from the

present version of the product is (to some degree) equivalent to the

required service. We assume that the actual service is provided by a

program that has been coded under several conditions and constrains and

thus not being a one-to-one translation of the required service.

Complementary we introduce a third layer in our called specified service,

where we define what has to be done on the computer. The actual service

might be considered as the layer describing how the service is to be

accomplished. Why and what for the service is needed is already defined in

the required service layer.

In such a layered product environment assessment is performed using

assessment methods such as inspection, testing, verification and

measurement to check the actual service against the specified service and

the expected service. These assessment methods have to be supported by

appropriate tools. For the assessment we also need to know which

characteristics of the product have to be considered and what is the

threshold for them. In order to be able (i.e. ''to be allowed'') to

certify a product, i.e. to put a quality seal on the product, we have to

evaluate assess all product layers with the required characteristics using

appropriate methods and tools on both product documents and process

documents. As a consequence we have to handle product, process,

characteristics, methods and tools as wells as their interaction in a

defined, coherent procedure.

In an assessment and certification the product and process elements

are to be identified and evaluated with respect to selected, required quality

characteristics. Appropriate methods and tools have to be applied to the

product and process documents to check those characteristics. The

essential problem domains are: software product, software process, software

characteristics, software methods and software tools.

The tutorial will cover the methods and principles of information and

software system quality assurance (comprising test, measurement and

assessment) for procedural, object-oriented or agent-based dependable

software systems. Attendees will exercise proven techniques for goal-directed

measurement, scaling and assessment for software certification. Assessment

of both the software product as well as the software process will be discussed

with respect to its relevance for such acceptance assessments.

A standardized process model for measurement,

assessment and certification of dependable software will be used to make

the attendees familiar with this comprehensive assessment procedure and to

learn how to embed it into today's standardized or non-standardized

software processes. Basic knowledge in mathematics and some knowledge of

software methods and tools is required. Emphasis will be given to selected

advanced topics depending on the needs of participants.

Tentative Content List

General Overview

What, where and how?

Seminar Organisation.

Introduction

Refresh of State of the Art and Practise in Industry.

The Q question,

Information quality, Software Quality and Data Quality.

Norms, Standards and Terminology.

Software Process and Software Product

Definition and Representation of Development Processes.

Definition and Representation of Software Products,

Information Systems as Products.

Quality and Productivity Modelling

Test, Verification and Validation

with respect to Evaluation and Certification.

Characteristics, Metrics and Assessment.

Process Evaluation and Certification

Capability Assessment Procedures.

Process Assessment Norms and Standards.

CMM, CMMI, ISO9000,

TickIT, Trillium, ami, SPICE

Product Evaluation and Certification

Motivation and Justification of Evaluation and Certification.

The Tailorable Evaluation Procedure.

Evaluators Guide according ISO9126 and ISO 14598

Evaluation of COTS according ISO 12119

Summary and Conclusion

What did we hear and why?

What do we expect from the big Q discussion?

Final Remarks.

Focus will be on the topics selected according attendees' expression of interests,

Assumed background and Knowledge of Attendees

Background: Developers, Integrators, Testers, Quality managers,

technology interested End-user or technology oriented CIOs;

Knowledge: Elementary Programming Languages, K12 School Mathematics

Rae, Robert, Hausen

Software Evaluation for Certification: Principles, Practice and Legal Liability.

McGraw Hill, International Software Quality Assurance Series, London

Public source: http://www.scope.gmd.de/documents/EvalGuide/