Introduction
Purpose
· The purpose of this document is to define the System Requirements Specifications (SRS) that explains the informational, functional, behavioral and operational aspects a generic AI for health (AI4H) system.
· Revisions to SRS shall be performed in an iterative manner based on an incremental delivery process model to elicit the emergent requirements of the system under consideration as AI systems continue to evolve over time to attain progressive maturity levels
· System requirements analysis methodology follows a collaborative team oriented approach, involving all the working groups and topic groups of AI4GH FG, to help the project team identify, control and track various requirements and changes to those requirements during the AI4H system development lifecycle
SRS Scope
· SRS scope includes a requirements model that defines the informational, functional, behavioral and operational aspects of the AI4H system under consideration. Specific objectives include the following:
· Best practices for defining the AI software requirements and the task that the AI should solve without any ambiguity. (...) It may be modified, customized or extended appropriately to include the specific requirements and needs of the particular topic group under consideration
· Requirement specifications may be defined in terms of use cases, graphical methods, mathematical models, documentation, etc. or combination of these
Document Conventions
· This document shall conform to the following standard convention of specification language syntax for every requirement specifications statement to indicate its particular significance / compliance level
· This standard convention follows the ‘specific terms’ rules defined in the guidance document- FG-AI4H-F-103- ‘Data Acceptance and Handling Policy’
Term
Meaning
"Shall"
states a mandatory requirement of this policy
"Should"
states a recommended requirement of this policy
"May"
states an optional requirement
· “TBD” (to be determined) shall be used as a placeholder term for pending information
Definitions
Acronyms, Abbreviations
SRS Overview
· System requirements specifications are developed following a generic ‘requirements modeling framework’ to guide the process of organizing, promising and tracing the requirements
· System requirements specifications are broadly organized in terms of (a) Functional Requirements, (b) External Interface Requirements, and (c) Non-Functional Requirements
System Description
· Application domain / topic group affinity/ healthcare speciality
· AI system / product objective
· AI task / service (prevention, screening, diagnosis, treatment, education, self-management, etc.)
· AI system/product category( clinical , non-clinical (personal care & wellness) apps
· AI system /product grade( medical grade product, commercial grade product
· AI operation mode (fully automatic, semi-automatic)
· AI intervention mode ( assistive tool, augmentative tool, etc)
· AI deployment mode
System Functions
Functional Requirement 1
-------
-------
Functional Requirement N
User Types /Classes and Characteristics
· Primary user type
· Secondary user type
· User execution requirements
Operating Conditions / Environment
· Operations site integration requirements
· Operation site adaptation requirements
Design and Implementation Constraints
· Design considerations /constraints
· Data sharing / replication policy
· Technical accuracy to clinical effectiveness mapping
· Business model sustainability
· Areas of stakeholder conflict
· Regulatory and Policy issues
· Regulatory compliance with country/region specific data policies
· Data privacy, trust, ethics, and ownership considerations
· Internationalization requirements
· Localization requirements
· Application criticality
· Communication protocols
· Security considerations
· Hardware limitations
· Interfaces to other applications
· Parallel operation
· Language requirements
· Control functions
· Specific technologies to be used
· Specific tools to be used
· Specific databases to be used
· Non-clinical data availability
External Interface Requirements
· User Interfaces
· Hardware Interfaces
· Software Interfaces
· Communications Interfaces
· Memory Constraints
Non-functional Requirements
· Performance Requirements
· Safety Requirements
· Security Requirements
· System Quality Requirements
System Design Requirements
· System Architecture design
· Sub system design
· UI design
· Component level design
System Implementation Requirements
· Coding Principles
· Coding and Testing procedures
· System Testing
System Deployment Requirements
· System configuration
· Deployment /run-time environment
· Assembling and testing
· Delivery packaging
· Distributing Computing requirements
· High performance production environments
· AI service utilization metrics
· Service levels
· Service level compliance report for clinical deployment
User Documentation / Training Requirements
· User tutorial
· Technical guide
· User Safety Guide
· Online help
· User documentation delivery formats / standards
Assumptions and Dependencies
· Unintended consequences
· Third-party / commercial components / licenses used
· Components reused from other projects
· Vendor-neutral interoperability standards
Standards Process Compliance
· Project Management Process
· Data Management Process
· Software Delivery Process
· Regulatory Audit Process
· Quality Audit Process
· Regulatory, Quality and Security Certification Process
Risk Management Requirements
· Risk Assessment
· Risk Control
· Risk Communication
· Risk Review
Change Management Requirements
Requirements Validation Requirements
· Functional testing
· Performance testing
· System testing
· Hardware & Software platform testing
· Hardware & Software interface testing
· Data Interface / Interoperability testing
· Data Quality testing
· Data Access Control testing
· Workflow / Protocol Integration testing
· Safety and Security controls testing
· User Group testing
· Usability testing
· User-Interface testing
· Installation testing
· Stress testing
References
Appendix A -Glossary
Appendix B- SRS supporting information (e.g. system feasibility study reports, cost analysis study reports, patient safety reports, user surveys, etc)
Language:English
Score: 339112.4
-
https://www.itu.int/en/ITU-T/f...ocuments/all/FGAI4H-G-203.docx
Data Source: un
CI Forum 2012_Tunis_AKwan_S2_3
05/11/2012
1
Requirements for Accreditation Bodies and Testing Laboratories
Presented by Andrew Kwan ITU Consultant
ITU Forum on Conformance and Interoperability
for the Arab and African regions Tunis, Tunisia, 5-7 November 2012
Tunis, Tunisia, 5 - 7 November, 2012
1
PRESENTATION OVERVIEW
Conformance Testing
Requirements for Accreditation Bodies
Requirements for Testing Laboratories
Tunis, Tunisia, 5 - 7 November, 2012
2
05/11/2012
2
Conformance Testing
Accreditation body
Testing Laboratories
Assess competence
Products
Supplier
Assess conformity
Tunis, Tunisia, 5 - 7 November, 2012
3
Conformance Testing
ISO and IEC jointly develop international standards through the ISO Committee on Conformity Assessment (CASCO). (...) First surveillance on-site assessment no later than 12 months from initial accreditation
Tunis, Tunisia, 5 - 7 November, 2012
24
05/11/2012
13
Requirements for Accreditation Bodies (ISO/IEC 17011)
Accreditation process
Extending accreditation
Extension of scope of accreditation may require reassessment
Suspending, withdrawing or reducing accreditation
Accreditation of CAB can be suspended, withdrawn or reduced if the CAB persistently failed to meet the requirements of accreditation or to abide by the rules of accreditation
Tunis, Tunisia, 5 - 7 November, 2012
25
Requirements for Accreditation Bodies (ISO/IEC 17011)
Accreditation process
Responsibilities of the accreditation body
Give notice of changes to its requirements for accreditation
Make public information on the current status of the accreditations it has granted to CABs
Responsibilities of the CAB
Fulfill continually the requirements for accreditation
Cooperate with accreditation body to verify fulfillment of requirements for accreditation
Inform accreditation body of changes relevant to its accreditation
Tunis, Tunisia, 5 - 7 November, 2012
26
05/11/2012
14
Requirements for Testing Laboratories (ISO/IEC 17025)
General
Organization and Management
Management requirements
Technical requirements
27 Brasilia, Brazil, 12-15 June, 2012
27
Requirements for Testing Laboratories (ISO/IEC 17025)
General
ISO/IEC 17025 addresses both management system elements and technical competence in a systematic and consistent way
Quality systems elements in line with ISO 9001
Emphasis on responsibility of senior management
Explicit requirements for continued improvement of the management system
Emphasizes communication with customers
Tunis, Tunisia, 5 - 7 November, 2012
28
05/11/2012
15
Requirements for Testing Laboratories (ISO/IEC 17025)
Organization and Management
Organization
The laboratory must be part of an organization that is an entity which can be held legally responsible
Responsible for meeting requirements of:
ISO/IEC 17025
The needs of its clients
Regulatory authorities
The management system shall cover work at:
Permanent facilities
Sites away from permanent facilities
Associated temporary or mobile facilities Tunis, Tunisia, 5 - 7 November, 2012
29
Requirements for Testing Laboratories (ISO/IEC 17025)
Organization and Management
Lab requirements
Have managerial and technical personnel with authority and resources needed to carry out their duties including:
implementation, maintenance and improvement of the management system
Have policies and procedures for the protection of client’s confidential information, proprietary rights including:
Electronic storage and transmission of results
Tunis, Tunisia, 5 - 7 November, 2012
30
05/11/2012
16
Requirements for Testing Laboratories (ISO/IEC 17025)
Organization and Management
Lab requirements
Define organization and management structure including:
Relationship within a larger organization
Relationship between qualify management, technical operation and support services
Define the responsibility, authority and inter- relationship of personnel involved in management, performance or verification of work affecting the quality of tests and/or calibrations
Tunis, Tunisia, 5 - 7 November, 2012
31
Requirements for Testing Laboratories (ISO/IEC 17025)
Organization and Management
Lab requirements
Technical management with overall responsibility for:
Technical operation
Provision of resources needed to meet the required quality of laboratory operation
Tunis, Tunisia, 5 - 7 November, 2012
32
05/11/2012
17
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Management system
Document policies, procedures and instructions on the laboratory’s scope of activities
Quality policy defined in a quality manual and issued under authority of top management with:
Statement on standard of service
Quality objectives must be measurable
Roles and responsibility of technical management and quality manager defined in the qualify manual
Tunis, Tunisia, 5 - 7 November, 2012
33
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Document control
Establish and maintain procedures to control all documents as part of it management system
Document approval and issue:
Documents reviewed and approved by authorized personnel
A master list established and readily available
Authorized editions of documents available at all locations where operations essential to effective operation of the laboratory are performed
Tunis, Tunisia, 5 - 7 November, 2012
34
05/11/2012
18
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Document control
Document approval and issue:
Documents reviewed and revised periodically for suitability
Documents changes:
Changes to documents reviewed by the same function that performed the original review
Procedures and authorities for hand written amendment defined
Procedures to describe to make and control changes in computerized systems
Tunis, Tunisia, 5 - 7 November, 2012
35
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Review of request, tenders and contracts
Procedures for review to ensure that:
Requirements including methods to be used are adequately defined and understood
Capability and resources to meet the requirements
Appropriate tests or calibrations are selected capable of meeting the customer’s requirements
Any differences between the request or tender and the contract be resolved before commencing work
Maintain records of review and discussions with customers
Tunis, Tunisia, 5 - 7 November, 2012
36
05/11/2012
19
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Subcontracting
Subcontracting work be placed with a competent laboratory (e.g. one which is ISO/IEC 17025 compliant)
Advise in writing the customer of subcontracting of tests and, where appropriate obtain their approval
Responsible to the customer for the subcontractor’s work
Tunis, Tunisia, 5 - 7 November, 2012
37
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Purchasing services and supplies
Have procedures for selecting and purchasing of services and supplies affecting quality of work
Purchased supplies not to be used until they have been verified as complying with requirements
Purchasing documents adequately describe the services and products ordered and they are reviewed and approved for technical content before being released
Tunis, Tunisia, 5 - 7 November, 2012
38
05/11/2012
20
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Service to customer
Seek feedback both positive and negative from customers
Analyse feedback for improvement to the management system
Complaints
Procedure for resolution of complaints
Maintain records of all complaints, investigations and corrective actions taken
Tunis, Tunisia, 5 - 7 November, 2012
39
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Control of nonconforming testing
Have policy and procedures to be implemented in the case of the testing does not conform to its own procedures or customer’s requirements
Policy and procedures ensure that:
Responsibilities and authorities for taking action are designated
Corrective action is taken immediately with decision on the acceptability of the nonconforming work
Customer is notified and work is recalled
Tunis, Tunisia, 5 - 7 November, 2012
40
05/11/2012
21
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Corrective actions
Have procedures for corrective action and appropriate authorities designated to implement corrective action
Corrective action procedures include:
Root cause analysis
Selection and implementation of corrective actions
Additional audits to ensure compliance with own policies and procedures or compliance with ISO/IEC 17025
Tunis, Tunisia, 5 - 7 November, 2012
41
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Preventive actions
A proactive process to identify opportunities for improvement, not a reaction to the identification of problems
Plans to be developed, implemented and monitored to identify improvement opportunities or required preventive actions
Tunis, Tunisia, 5 - 7 November, 2012
42
05/11/2012
22
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Control of records (General)
Have procedures to identify, collect, index, access, file, maintain and dispose quality and technical records
Specify retention times for records
Records to be readily retrievable and stored in a manner to prevent damage and loss
Records to be held secure and in confidence
Procedures to back up records stored electronically and to prevent unauthorized access to or amendment of such records
Tunis, Tunisia, 5 - 7 November, 2012
43
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Control of records (Technical)
Records retained for a defined period of time including:
Original observations
Derived data
Sufficient information to establish audit trail
Staff records
Copy of test report or calibration certificate issued
Calibration records
Tunis, Tunisia, 5 - 7 November, 2012
44
05/11/2012
23
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Control of records (Technical)
Test or calibration records contain sufficient information to enable:
Identification of factors contributing to measurement uncertainty
Identification of personnel responsible for the sampling and checking of results
Mistakes in records be crossed through but still legible with the correction entered next to the original information
Alteration signed or initialled by person making the correction
Tunis, Tunisia, 5 - 7 November, 2012
45
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Internal audits
Conducted periodically and within a predetermined schedule and procedure
Quality manager responsible for planning and carrying out internal audits
Covers all elements of management system including testing activities
Records kept of the area of activity audited, the audit findings and corrective actions
Conduct follow up to the audit to verify the implementation and effectiveness of corrective actions taken
Tunis, Tunisia, 5 - 7 November, 2012
46
05/11/2012
24
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Management reviews
Management reviews are conducted periodically (typically once per year) with a predetermined schedule and procedure to:
Review the effectiveness and suitability of its management system and/or testing or calibration activities
Introduce necessary changes or improvement
Tunis, Tunisia, 5 - 7 November, 2012
47
Requirements for Testing Laboratories (ISO/IEC 17025)
Management requirements
Management reviews
Management reviews cover:
Reports from managerial and supervisory personnel
Outcome of recent internal audits
Corrective and preventive actions
Assessments by external bodies
Customer feedback
Complaints
Recommendations for improvement
Tunis, Tunisia, 5 - 7 November, 2012
48
05/11/2012
25
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
General
Factors which determine the correctness and reliability of tests include:
Human, environmental conditions, test method and method validation
Measurement traceability, sampling and handling of test items
Take into account of these factors when developing tests, procedures, training and qualification of personnel and selection of equipment
Tunis, Tunisia, 5 - 7 November, 2012
49
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Personnel
The laboratory shall:
Ensure the competence of all who operate equipment, perform tests, evaluate results and sign test reports and calibration certificates
Have policies and procedures for training programs and evaluation of their effectiveness
Management to authorize specific personnel:
To perform particular types of sampling and tests
To issue test reports and calibration certificates
Maintain records of relevant authorizations, competence, qualifications and experience for all personnel including those under contracts
Tunis, Tunisia, 5 - 7 November, 2012
50
05/11/2012
26
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Accommodation and environmental conditions
Laboratory facilities (energy source, lighting, environmental) be suitable for performing tests correctly
Document technical requirements for accommodation and environmental conditions that can affect results
Effective separation between incompatible activities
Control access to and use of areas affecting quality of testing
Tunis, Tunisia, 5 - 7 November, 2012
51
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Test and calibration methods and method validation
Use appropriate methods and procedures for all tests in its scope
Deviation from test methods documented, technical justified, authorized and accepted by the customer
Test methods published in international, regional or national standards are preferred
Use of non-standard method shall:
Be subject to agreement with the customer
Include a clear specification of the customer’s requirement and the purpose of the test or calibration
Be validated appropriately before use
Tunis, Tunisia, 5 - 7 November, 2012
52
05/11/2012
27
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Estimation of uncertainty of measurement
Procedures for estimation of uncertainty for types of tests and calibrations
If test method may preclude rigorous analysis, the laboratory shall:
Attempt to identify all components of uncertainty
Make a reasonable estimation based on knowledge of the performance of the method and on the measurement scope
Tunis, Tunisia, 5 - 7 November, 2012
53
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Control of data
Calculation and data transfers subject to appropriate checks
Computer software suitably validated
Procedures for protecting data including integrity and confidentiality of data collection, data storage, data transmission and data processing
Tunis, Tunisia, 5 - 7 November, 2012
54
05/11/2012
28
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Equipment
Furnished with equipment for correct performance of tests
Equipment used outside of its permanent control meets appropriate requirements of ISO/IEC 17025
Equipment which has been subject to overloading or mishandling or found to be defective shall be:
Taken out of service
Isolated or clearly marked as being out of service until it has been repaired and calibrated
Tunis, Tunisia, 5 - 7 November, 2012
55
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Equipment
Equipment to be labelled with its calibration status
When for whatever reason, equipment goes outside the control of the laboratory, check its function and calibration status before it is returned to service
Conduct intermediate checks when required according to defined procedure
Tunis, Tunisia, 5 - 7 November, 2012
56
05/11/2012
29
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Measurement traceability
Calibrate all equipment which have a significant effect on test results
For calibration, establish traceability of equipment to the International System of Units (SI) (Systeme international d’unites) by means of an unbroken chain of calibrations or comparison linked to relevant primary standards, typically through national metrology institutes
Tunis, Tunisia, 5 - 7 November, 2012
57
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Sampling
Have sampling plan and procedures available at locations where sampling is undertaken
Record deviations, additions or exclusion from the documented sampling procedure with appropriate sampling data
Record sampling procedure, identification of the sampler, environmental conditions, diagram and if appropriate statistics upon which sampling is based
Tunis, Tunisia, 5 - 7 November, 2012
58
05/11/2012
30
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Handling of test and calibration items
Procedures for identification, transportation, receipt, handling, protection, storage, retention and disposal of test items
Consult customer before proceeding when there is doubt about the suitability of an item for test or test required is not specified in sufficient detail
Tunis, Tunisia, 5 - 7 November, 2012
59
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Reporting the results
Report results accurately, clearly, unambiguously and objectively and in accordance with any specific instructions in the test method
Test reports include information requested by the customer and necessary for interpretation of the test results
Tunis, Tunisia, 5 - 7 November, 2012
60
05/11/2012
31
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Test reports and calibration certificates
Test report, as a minimum contains:
A title (e.g. “Test Report” or “Calibration certificate”)
A unique identification such as the serial number
Name and address of customer
Identification of test methods used
Description of condition and unambiguous identification of test item
Date of receipt of test item where this is critical to the validity and application of the results
Date of test or calibration
Test or calibration results with, where appropriate the unit of measurement
Tunis, Tunisia, 5 - 7 November, 2012
61
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Test reports
Test report also contains where necessary for interpretation of test results:
Deviations from, additions to, or exclusions from the test method
Where relevant, a statement of compliance/non- compliance with requirements
Where applicable, a statement on the estimated uncertainty of measurement
This statement is applicable when required for the validity or application of results, when the customer requires it or when it affects compliance to a specification limit
Tunis, Tunisia, 5 - 7 November, 2012
62
05/11/2012
32
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Test reports
In addition, if sampling is used, report includes:
Date of sampling
Unambiguous identification of samples
Location of sampling, including diagrams, sketches or photographs
Reference to sampling plan and procedures used
Details of any environmental conditions during sampling that may affect interpretation of results
Any standards or specifications for sampling methods
Any deviations, additions or exclusions from the specification concerned
Tunis, Tunisia, 5 - 7 November, 2012
63
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Calibration certificates
Calibration certificates contains:
Conditions under which calibrations were made
Uncertainty of measurement and/or statement of compliance
Evidence that measurements are traceable
Tunis, Tunisia, 5 - 7 November, 2012
64
05/11/2012
33
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Opinion and interpretation
When opinions and interpretations are included, the laboratory shall:
Document the basis on which they were made
Clearly mark as such in the report
Testing and calibration results obtained from subcontractors
When test report contains results of tests performed by subcontractors, these results shall be clearly identified
Tunis, Tunisia, 5 - 7 November, 2012
65
Requirements for Testing Laboratories (ISO/IEC 17025)
Technical requirements
Amendments to test reports & calibration certifications
Material amendments to test report after issue shall be made only in the form of a further document or data transfer
It includes the statement:
“Supplement to Test Report (or Calibration Certificate), serial number….” or equivalent wording
A new report shall be uniquely identified and contain a reference to the original that it replaces
Tunis, Tunisia, 5 - 7 November, 2012
66
05/11/2012
34
Requirements for Accreditation Bodies and Testing Laboratories
Thank you
Andrew Kwan
akwan70@gmail.com
Tunis, Tunisia, 5 - 7 November, 2012
67
Language:English
Score: 338476.37
-
https://www.itu.int/ITU-D/tech...um%202012_Tunis_AKwan_S2_3.pdf
Data Source: un
