EXTENDED TUTORIAL ABSTRACT
EVALUATION, ASSESSSMENT AND CERTIFICATION OF
DEPENDABLE INFORMATION SYSTSTEMS
by Hans-Ludwig Hausen, Fraunhofer,
What is evaluation, assessment and certification?† Why do we need it?† We
ask for software assessment and certification because we want to be sure
that the product we want to apply provides the expected service correctly
with respect to both functional and non-functional requirements.† If we are
assessing software we check whether the actual service we can get from the
present version of the product is (to some degree) equivalent to the
required service.† We assume that the actual service is provided by a
program that has been coded under several conditions and constrains and
thus not being a one-to-one translation of the required service.
Complementary we introduce a third layer in our called specified service,
where we define what has to be done on the computer.† The actual service
might be considered as the layer describing how the service is to be
accomplished.† Why and what for the service is needed is already defined in
the required service layer.
††††††††††† In such a layered product environment assessment is performed using
assessment methods such as inspection, testing, verification and
measurement to check the actual service against the specified service and
the expected service.† These assessment methods have to be supported by
appropriate tools.† For the assessment we also need to know which
characteristics of the product have to be considered and what is the
threshold for them.† In order to be able (i.e. ''to be allowed'') to
certify a product, i.e. to put a quality seal on the product, we have to
evaluate assess all product layers with the required characteristics using
appropriate methods and tools on both product documents and process
documents.† As a consequence we have to handle product, process,
characteristics, methods and tools as wells as their interaction in a
defined, coherent procedure.
††††††††††† In an assessment and certification the product and process elements
are to be identified and evaluated with respect to selected, required quality
characteristics.† Appropriate methods and tools have to be applied to the
product and process documents to check those characteristics.† The
essential problem domains are: software product, software process, software
characteristics, software methods and software tools.
††††††††††† The tutorial will cover the methods and principles of information and
software system quality assurance (comprising test, measurement and
assessment) for procedural, object-oriented or agent-based dependable
software systems.† Attendees will exercise proven techniques for goal-directed
measurement, scaling and assessment for software certification.† Assessment
of both the software product as well as the software process will be discussed
with respect to its relevance for such acceptance assessments.†
A standardized process model for measurement,
assessment and certification of dependable software will be used to make
the attendees familiar with this comprehensive assessment procedure and to
learn how to embed it into today's standardized or non-standardized
software processes.† Basic knowledge in mathematics and some knowledge of
software methods and tools is required.† Emphasis will be given to selected
advanced topics depending on the needs of participants.
Tentative Content List
††††††††††† What, where and how?
††††††††††† Seminar Organisation.
††††††††††† Refresh of† State of† the Art and Practise in Industry.
††††††††††† The Q question,
††††††††††† Information quality, Software Quality and Data Quality.
††††††††††† Norms, Standards and Terminology.
†Software Process and Software Product
††††††††††† Definition and Representation of† Development Processes.
††††††††††† Definition and Representation of Software Products,
††††††††††† ††††††††††† ††††††††††† Information Systems as Products.
Quality and Productivity Modelling
††††††††††† Test, Verification and Validation
††††††††††† ††††††††††† with respect to Evaluation and Certification.
††††††††††† Characteristics, Metrics and Assessment.†††††††††† ††††††††††† †††††††††††
†Process Evaluation and Certification
††††††††††† Capability Assessment Procedures.
††††††††††† Process Assessment Norms and Standards.
††††††††††† ††††††††††† CMM, CMMI, ISO9000,
††††††††††† ††††††††††† TickIT, Trillium, ami, SPICE
†Product Evaluation and Certification
††††††††††† Motivation and Justification of Evaluation and Certification.
††††††††††† The Tailorable Evaluation Procedure.
††††††††††† ††††††††††† Evaluators Guide according ISO9126 and ISO 14598
††††††††††† ††††††††††† Evaluation of† COTS according ISO 12119
Summary and Conclusion
††††††††††† What did we hear and why?
††††††††††† What do we expect from the big Q discussion?
††††††††††† Final Remarks.
††††††††††† Focus will be on the topics selected according attendees' expression of interests,
Assumed background and Knowledge of Attendees
Background: Developers, Integrators, Testers, Quality managers,
technology interested End-user or technology oriented CIOs;
Knowledge: Elementary Programming Languages, K12 School Mathematics
Rae, Robert, Hausen
Software Evaluation for Certification: Principles, Practice and Legal Liability.
††††††† McGraw Hill, International Software Quality Assurance Series, London
Public †source: http://www.scope.gmd.de/documents/EvalGuide/